Real-Time Data Flow in the Cloud

When we talk about putting real-time data into the cloud, what we mean is data flow in the cloud, or data communications. Most cloud applications today work on the premise of storing data in the cloud, and then accessing it at some later time. What we are interested in here is dynamic data, where a value can change 10 times or 100 or even 1,000 times per second, and be passed immediately along at pretty much that same rate. This is the kind of data flow that is commonly used in industrial processes, and which may have significant implications for cloud computing in general.

Paul Maritz, President & CEO, VMware touched on this in August in his keynote address on the future of cloud computing at VMWorld 2011. He said, “People are going to have to be able to react to information coming in, in real time. If you’re going to service the Facebook generation they way that they want to see information, you’re going to have to give them customized information in the context that they want to see it, in real time.”

Support for real-time communication has been evolving in the industrial sector over the past few decades. Control systems used to be considered “islands of automation”, and any data sharing between systems was done by jotting notes on a clipboard and making phone calls, or perhaps by copying data to a floppy disk, and carrying it to another machine over the “sneakernet”. With the advent of the LAN and Windows 3.1 in the 1990s, protocols like ArcNet, NetBIOS, and NetDDE were developed, but stability was a major issue.

Since then, a lot of thought and effort have gone into developing protocols for real-time communications for industrial data. Modern processes now benefit from more stable, routable protocols like TCP/IP and OPC for distributed processing. Today, machine operators can leave the control room when necessary, and walk through the plant, viewing their process data in real time on a hand-held computer. Managers are able to monitor production on the factory floor in real time, from their office, at home, or even on the road, using real-time data displays over proprietary networks.

Now people are starting to ask for this kind of reliable, high-speed data communication in the cloud. There are some real opportunities here. In addition to monitoring factory data and connecting machines, the cloud opens new possibilities. For example, data from embedded devices and remote sensors anywhere in the world could be collected from any location and sent to your laptop or phone in real time. Automobile navigation systems could become aware of the other cars on the road. The consumer applications envisioned by Paul Maritz will soon become reality.

Of course, there is the vision of data flow in the cloud, and then there is reality. Is all of this really doable? In the coming weeks and months, we will look at some of the practical aspects of how real-time data might be enabled to flow freely and securely through the cloud, and we’ll keep you up to date with the latest developments in the field. But before we do that, next week we’ll take a look at the specific benefits of cloud computing for users of real-time data.