Collecting Big Data in Real Time

It was bound to happen.  The two titans meet.  The gargantuan grasp of Big Data turns its ever-open hands towards the firehose stream of real-time data.  “The next evolution of the big data phenomenon has turned out to be real time streaming of data,” says Big Data pundit Rick Delgado in a recent blog: What Real Time Streaming Means for Big Data.  “Organizations have an increased need to gather and analyze their data at the same time, making real time data streaming a must if big data is going to keep up with demand.”

Will Big Data ever be satisified?  Not as long as the demand for informed action continues to grow.  Will we ever run out of real-time data?  Not as long as stuff keeps happening.  The only thing necessary to complete this marriage is to make the connection, and stream real-time data into the welcoming, capable hands of Big Data.

This is what we are keen on.  With our established track record in real-time industrial data communications, we anticipated this need for real-time analytics years ago, along with other thought leaders.  In a blog back in 2011 we quoted Paul Maritz, President & CEO of VMware at keynote address on the future of cloud computing at VMWorld 2011, “People are going to have to be able to react to information coming in, in real time.” Since then we’ve been putting the vision into action, and it’s great to now see the Big Data people coming on board.

Real-Time Analytics from Big Data

The advantage of live connectivity to Big Data is you can now do your analytics in real time. Delgado sees this clearly.  Real-time inputs to Big Data, he says, can fuel near-real-time outputs.  Rather than a two-stage process of storing the data, and then analyzing it, the analysis can take place on the fly, and your system can function like the mind of an athelete, jazz musician, pilot, or soldier. Insights become more spontaneous, and reactive responses are replaced by pro-active initiatives. The competitive advantage goes to those who can better anticipate and immedately meet customer demands, increasing customer satisfaction and establishing greater loyalty.

Delgado lists a number of areas where real-time streaming to Big Data could have a significant impact. For example, certain types of fraudulent or suspicious patterns of trading in the financial sector that don’t show up in the aggregate could be spotted in real time.  Businesses could monitor customer behavior on websites and social media to provide people with exactly what they need, at the moment they want it.

Additional Benefits – Industrial Sector

Among various application spaces that Delgado mentioned, he left out a significant one: streaming real-time Big Data for industrial users.  Imagine an operator of a machine where an alarm light is flashing.  Looking at his smart phone or tablet, he gets not only the alarm and raw data from the machine, but a real-time analysis of what could be wrong.  And along with that, he may receive suggested action steps based on comparing that data in real time to technical specs, historical records, and even live recommendations from its manufacturer, who is also connected to the machine, and monitoring it in real time.

Companies like GE are investing millions in such systems.  They collect and analyze in real time the Big Data coming from power turbines, jet engines, and other equipment during operation.  As the Industrial IoT gains acceptance, we see other companies, big and small, follow suit.  The value inherent in real-time data for making instantaneous decisions is too great to pass up.  The industrial sector, a large and long-time user of real-time data, stands to benefit significantly by connecting to Big Data.

Will Time-Sensitive Networking (TSN) Improve the IIoT?

Is current Internet technology sufficient for the needs of Industry 4.0 or the IIoT?  Or could it be better?  How can we enhance Ethernet to improve real-time data communications? These are the kinds of issues that some key players in Industrial IoT plan to address by developing the world’s first time-sensitive networking (TSN) infrastructure.

TSN has been defined as “a set of IEEE 802 standards designed to enhance Ethernet networking to support latency-sensitive applications that require deterministic network performance,” according to Mike Baciodore in a recent article in Control Design titled “How time-sensitive networking enables the IIoT

Put simply, the goal of TSN is to provide the IoT with the same kind of real-time performance that is now limited to individual machines like cars and airplanes, or to distributed control systems in industrial applications.  The Industrial Internet Consortium (IIC), along with Intel, National Instruments, Bosch Rexroth, Cisco, Schneider Electric and others have joined forces to achieve this goal, to enable a truly real-time IoT.

TSN is Good News for Skkynet

This collaboration to develop TSN comes as good news to us here at Skkynet.  Since we currently provide secure, bidirectional, supervisory control capabilities over TCP, we understand how much more effective our software and services will be when supported by TSN.

With TSN, our latencies of a few ms over Internet speeds would be reduced to simply a few ms.  Data dynamics would be better preserved, and system behavior more deterministic.  This effort to develop TSN validates our thinking that the IIoT works best with low-latency, high-speed networking.  Unlike those who operate on the assumption that web communication technology (REST) is the way forward, the TSN approach means that networked data communications can approximate or equal in-plant speeds and latencies.

Several participants and commentators on the TSN project point out that typical cloud architectures are not ideal counterparts for TSN.  Something fundamentally different is required.  Putting their individual ideas and suggestions together, what they envision for an architecture is remarkably close to what Skkynet currently provides.  It should be secure by design, fully integrate edge computing, and keep the system running without interruption during any network outages.  Above all, it must provide secure, selective access to any process data, in real time.

“One of the cool concepts out there is that people will want to have a cyberphysical representation of the equipment in the cloud,” said Paul Didier, solutions architect manager at Cisco. “That doesn’t mean the physical plant will be controlled in the cloud. Optimization and maintenance can be done in the cloud and will filter its way back to the machine.”

Our recent case study showcasing DataHub and SkkyHub technology illustrates this “cyberphysical representation.”  During the deployment and test of a mineral processing system, developers thousands of miles away monitored the machine logic and tweaked the system in real time. “It was as if we were sitting beside them in the control room,” said one of the team, “and through live monitoring, we were able to continue developing the application, thanks to the real-time connectivity.”

It’s a small step from this to machine control, and time-sensitive networking will be a welcome technology in that direction.  To the Industrial Internet Consortium (IIC) and everyone else involved in this project, we say keep up the great work!  We’re ready to put TSN to good use when it becomes available.

Industrial IoT, Big Data & M2M Summit―Takeaways

Last week several of us here at Skkynet had the pleasure to attend and present a case study at the Industrial IoT, Big Data & M2M Summit in Toronto.  IoT specialists representing a wide range of industries, from mining, manufacturing, and energy to telecom and software gathered to share insights and learn from collective experience how to get the most out of Industrial IoT.

Challenges to IoT adoption was a key topic of discussion.  There was considerable agreement among summit participants that one of the primary challenges is not technical, but cultural.  Switching from software ownership to data as a service requires a new mind-set, which not everyone is willing to adopt.  Speaker after speaker underlined the need to communicate value and get buy-in from all concerned parties. You should start with a small pilot project, with minimal investment, and demonstrate ROI.  Other challenges discussed included incompatible protocols and security risks.

Summit Theme: Partnerships

A common theme that prevailed in presentations and comments throughout the summit was that the IoT casts such a wide net that nobody can do all of it well.  We need to work together.

“IoT is all around partnerships,” said Christopher Beridge, Director of Business Development – IoT and Business Solutions at Bell Mobility.

“A lot of people have a part to play when you are talking IoT,” according to Matthew Wells, Senior Product General Manager at GE Digital.

“Smartness depends on how interconnected you are,” commented Steven Liang, Associate Professor at the University of Calgary, and conference chair.

Above all, there was agreement that the IoT is here to stay. “Our focus is to make things more efficient, reliable, affordable, and convenient, and the IoT is a way to do it,” said Michael Della Fortune, Chief Executive Officer of Nexeya Canada.  “It powers and upholds the 4 Vs—Variety, Volume, Velocity, and Veracity—of Big Data.”

Perhaps Timon LeDain, Director, Internet of Things at Macadamian summed it up best when he said, “IoT will be done by you, or done to you.”

New Whitepaper: Will IT and OT Converge?

It’s no mystery what “IT” stands for: Information Technology, the computing power used to run businesses and corporations. Fewer people might know that “OT” in this context stands for Operational Technology, the computing power behind modern industrial production systems. A new white paper from Skkynet asks whether IT and OT will converge.

These two, IT and OT, are worlds apart in most businesses. The IT people use computing resources to support accounting, logistics, HR, and all other areas of the business. In a sense, the product of IT is the business itself. IT becomes a star of the show. In the OT world, the focus is on doing or making things. The product is the process, or the manufactured output. OT is one of several players in the game, operating primarily in a supporting role, to ensure that mechanical systems function as designed.

For a long time, OT and IT have functioned separately from each other. OT has been hidden away in the deep recesses of the plant or out in the field, using its own proprietary data protocols, and often physically disconnected from the rest of the corporate network. IT has been content to get occasional updates on factory floor status through paper reports, database entries, or ERP (Enterprise Resource Planning) systems.

Now these two worlds are beginning to make contact. Businesses are waking up to the value of the data that’s coming from the production systems. Managers are discovering within OT data opportunities to harness real-time analytics and leverage predictive technologies that IT can provide. John Pepper, CEO and Founder of Managed 24/7, recently wrote, “Unless organisations actively bridge the gap between OT and IT, the real operational benefits of the digital business will be lost.”

Our new white paper, Will IT and OT Converge? takes a closer look at some of the concepts introduced by Pepper. It clarifies the distinctions between IT and OT, presents current thinking about how they might converge, and highlights three critical requirements from the OT side for bridging the gap: security, ease of integration, and real-time performance.

Survey: Valuable Lessons from IoT Early Adopters

A recent survey by Machina Research (Lessons Learned from Early Adopters of the IoT: A Global Study of Connected Businesses) suggests that the IoT is moving quickly from novelty to necessity. Nearly two thousand management-level employees in companies earning $15 million and up per year in the USA, UK, Japan, Australia, and Brazil representing all major sectors of industry took part.  About 20% of the respondents have started some kind of IoT initiative, and close to 30% expect to do so in the next 6 months to 2 years.

Focusing on the innovators and early adopters of the IoT, the survey gleaned some useful information which may be helpful for those who have not yet implemented a strategy—and in many cases, those who have.  It seems that the majority of early adopters of the IoT took a do-it-yourself approach, and most of them found the IoT more complicated to implement than they expected. Future adopters say they will not repeat that mistake.

“When asked about primary concerns around IoT, adopters have some insight that non-adopters just don’t yet have,” states the report. “Adopters point to ‘complexity of the IoT solution’ as the largest concern around IoT, a concern that non-adopters have yet to consider fully.” Among those who have taken IoT initiatives, over half of them mentioned concerns about complexity, compared to only a quarter of those who have not yet taken the first step.

Other top concerns included security, ease of integration with existing systems, and the expense of implementation. These commonly-held concerns are undoubtedly part of the reason for the reluctance of others to undertake IoT projects on their own.  The majority of them responded that they are planning to work with an IoT-capable partner.

“Based on past experience of our adopters, companies who haven’t yet adopted IoT initiatives should not go it alone,” the report recommends. “Instead they should focus on finding partners whose core competency is connecting products securely.”

The report suggests that an ideal partner should not only have a technology platform, but should be able to simplify the complexity of the IoT.  They ensure that security is not an ad-hoc afterthought, but instead is inherent to the design of the system itself.  The partner should be able to easily integrate the IoT solution with existing and legacy systems, and offer significant cost savings and ROI.

IIoT to Animate the Embedded Systems Industry

It’s certainly a nice fit.  We would expect the growth of the Internet of Things to have a positive impact on the embedded systems industry.  With the billions of connected devices envisioned, it’s hardly surprising that the amount of computing to be embedded in the “Things” of the IoT would increase.  Perhaps not so obvious is that the Industrial IoT promises to generate a growing percentage of that increase.

After attending the Embedded Systems Conference (ESC) in Boston last month, industry expert and ARC Advisory team member Dick Slansky noted in his recent blog, Embedded Systems Industry Focuses on IIoT as the Future of Manufacturing, that “there appeared to be consensus that IoT and the entire notion of smart connected ecosystems was going to drive an industry revival for embedded systems.”

This sentiment echoes a recent study by IC Insights, Internet of Things Boosts Embedded Systems Growth, which states that the “Industrial” share of the $3.9B IoT market in 2014 is forecast to rise from an estimated 29% to 36% of the anticipated $11.5B IoT market in 2018.  Doing the math, that’s a jump from $1.13B to $4.14B in the dollar amount for the Industrial IoT, an almost four-fold increase over four years.

What is behind this?

Summing up Slansky’s analysis, he says that the new, smart factory will rely on smart devices—embedded systems connected to the IIoT. This in itself will give a boost to the embedded systems industry.  Yet the area expected to demand even more embedded systems development is edge computing—the moving of computing resources from the mainframes and PCs in a control room or central office to the sensor and device level, eliminating network traffic and cutting processing time. “Embedded intelligence at the source will become the norm,” said Slansky.

We have talked about edge computing previously, and discussed how the Skkynet Embedded Toolkit (ETK) supports it with a full-featured, built-in scripting language. Adding that capability to secure, bidirectional, connectivity to SkkyHub and DataHub, the ETK smoothly integrates real-time edge computing with the Industrial IoT, for a number of embedded platforms and devices.  For example:

  • It is a Verified Software Add-on for Renesas Synergy, meaning that any network enabled device built on the Renesas Synergy platform can connect, just by adding Skkynet to the application build.
  • It is available as an install package for Red Lion Sixnet® series RAM® cellular RTUs or IndustrialPro® cellular routers, supporting connections to SkkyHub and DataHub.
  • It runs on B+B Spectre 4G LTE cellular / wi-fi router, from B+B SmartWorx, NetComm’s NTC-6200 series of gateways, and the Systech® SysLINK™ M2M Gateway.
  • It has been tested on Raspberry Pi, Arduino, GR-SAKURA, mbed, Lantronix Xport, WiPort, and MatchPort, and a number of other devices.