Can I Store and Forward My Process Data?

Every once in a while we get asked: “Can I store and forward my live process data?” It seems like a reasonable question. You’re connecting data sources and clients across a network, maybe using OPC DA, OPC UA, or MQTT. If the connection drops, you don’t want to lose data. Why not store it and forward it later when the connection is back up?

But a little understanding of how real-time data protocols work should make it clear why this doesn’t apply to live data. The short answer is: No, not real-time data, anyway. Remember, real-time data is live data flowing through the system that the process is currently using. Contrast that with historical data, which is a static history of what has already taken place.

Let’s take OPC DA, for example. The “DA” in the OPC DA protocol stands for “data access”. This means live, real-time data. An OPC DA server does not maintain a history of past values, and an OPC DA client only works with the most recent value of each tag. Suppose someone decided to add store-and-forward to OPC DA anyway.

Here’s where the problems begin.  When the connection breaks, the server would need to start storing values, but how long should it store them? How would the client identify which items need replaying? When the connection is restored, how quickly should the stored values be sent? Replicating the original time gaps means it would never catch up. Sending them in a burst means the client gets flooded with a rapid series of changes. There is no facility in OPC DA for any of this, and it could not be robustly specified at the server. Not all clients would have the same requirements.

But the biggest problem is that old values are, by definition, wrong — only the most recent value is correct. In a store-and-forward scenario the client would experience a series of changes that could manifest as wrong indications in an HMI, or worse, wrong control signals to a physical system. Would you really want a piece of equipment turning on and off repeatedly as the data history catches up on the client? That could be inefficient, damaging or dangerous. In practice, most OPC clients can only hold one value for any given tag, and only now matters. The client doesn’t need and can’t use a replay of historical data, it needs to know what’s happening this instant.

Network connections drop occasionally. That’s a fact of life. When it happens, the client should present data with a Bad or Not Connected quality, or show a Last Known Value warning until the connection is restored. Once the connection is back, the client should receive current data values. Whatever happened in the meantime is unimportant or downright misleading.

Turning to OPC UA, the fundamental story is the same. An OPC UA client still expects to receive the latest data possible. OPC UA does integrate historical data access better than OPC DA, making it easier for servers to support historical data and for clients to request it. But whether data storage is built into the server or provided separately, the reality is unchanged: a real-time client cannot handle data that has been stored and forwarded. That is not its purpose.

MQTT follows the same pattern, despite its different architecture. MQTT uses a publish-subscribe model through a broker, with quality-of-service (QoS) to indicate reliability of delivery. But since the data is streaming in real time, store-and-forward support is not useful here either. If a subscriber were to reconnect after an outage and receive a backlog of stale values for a real-time data point, the result would be the same as with OPC: wrong indications, wrong control signals, and potential harm. For real-time monitoring and control, MQTT subscribers need the current value, not a replay of history.

The idea of store-and-forward makes perfect sense for data logging, archiving, and historical records. This is where it is possible to maintain a complete record of everything that has taken place. An OPC HDA server, a process historian, a time-series database, or an MQTT-to-database pipeline are all designed for exactly this purpose. The responsibility for store-and-forward belongs in the hands of the historian or archive system, not in the real-time data path.

So, the answer to the question “Can I store and forward my process data?” is two-fold. You can record your process data and store and forward it as archived data. But you cannot store and forward data in real time.