Observe the Connection State

• Reconnection callbacks; connection events.

A connection can be lost and regained without the process noticing. The client library reconnects on its own, the subscriptions are re-established, and the service looks as though nothing happened — while messages published during the gap were never delivered to it.

How does a reader tell "no messages arrived" from "this service was not connected to the bus when they arrived"?

• A reconnection is silent by default: the service resumes and its log shows nothing.
• A gap in a service's log is otherwise indistinguishable from a quiet period.
• Only the sender's log records what was published during the gap, and a person investigating has one log.
• The bus reports errors on its own thread, where a logging call can fault.

Install callbacks for the connection's state and write one line for each transition, plus one for an asynchronous transport error:

natsOptions_SetDisconnectedCB(opts, [](natsConnection*, void*) {
    BOOST_LOG_SEV(lg(), warn) << "NATS disconnected: the server is unreachable "
                                 "and subscriptions are not receiving.";
}, nullptr);

natsOptions_SetReconnectedCB(opts, [](natsConnection*, void*) {
    BOOST_LOG_SEV(lg(), warn) << "NATS reconnected: subscriptions are receiving "
                                 "again. Anything published while disconnected was "
                                 "delivered only if its subject is on a stream.";
}, nullptr);

Three rules:

• Warn, not info. A transition is an event worth finding in a log that is otherwise at info level.
• Say what the gap cost. The reconnected line states the consequence: messages on a stream were kept, messages on core NATS were not.
• Never let a logging call fault. These callbacks run on a library thread and the logging system may already be shutting down, so each is guarded.

A transport error is reported the same way, through the error handler, rather than being printed to standard error by the library.

• No callbacks installed. A service that lost and regained its connection is indistinguishable from one that never lost it.
• A reconnected line with no consequence stated. The reader does not know whether anything was missed.
• A callback that can throw. A logging call that faults on a library thread terminates the service.
• Logging every transport event at info. Routine disconnects fill the log and the ones that matter are lost.

Benefits

A gap in a service's log is attributable, and an unexplained caller timeout has a candidate cause in the log.

Liabilities

Warnings appear for connections that recovered harmlessly, which a reader must learn to read as context rather than as a fault.

`ores.nats` installs a disconnected callback, a reconnected callback and an error handler when it connects, each logging at warn and each guarded against a torn-down logging system. The orderly connection loss during shutdown is deliberately suppressed so a clean stop does not read as a fault.

• Announce Every Handled Message — the per-message line these connection lines frame.
• Durable Subscription — why a message survives the gap described here.

• NATS documentation, Connection Events — the disconnected, reconnected and error callbacks.
• NATS documentation, Reconnection — the reconnect buffer and what survives a drop.

• NATS service patterns — the structure note that orders this pattern language, and where to read this page in it.