At the moment (at least for core, and probably also for compat), new messages are only seen if the bot is restarted. For long-running bots, it's sensible to break/pause/stop editing on a certain wiki when a talk page is edited.
Getting the info from the wiki is easy -- just add meta=userinfo&uiprop=hasmsg to the api query. Afterwards, meta=notifications can be used for details.
Added bonus: every api query returns a check whether the user is still logged in correctly.
Version: core-(2.0)
Severity: normal
See Also:
https://bugzilla.wikimedia.org/show_bug.cgi?id=55032