Keeping production running yields huge cost savings
A large pharmaceutical manufacturer in the European Union develops new designs, tests raw materials, and produces, inspects, packages and ships devices worldwide from a single plant location. The company’s vision is to be the global leader in innovation and manufacturing of its products. In keeping with that vision, they recently solved a production line bottleneck using Cogent DataHub software.
Due to a recent change to security enforcement in Microsoft Windows, minor disruptions in the control system that used to take a few seconds to resolve were shutting down whole production lines for up to five hours. “It was a lot of cost,” said Stephen Doody, plant automation team leader.
“Not only were we losing the manufacturing time, but the knock-on effect meant that half of our control staff was then tied up for that five hours to try to get the line back running.”
DCOM security issues
This problem is typical of the on-going challenge in the automation industry to maintain a stable, working system within a changing hardware and software environment. In this case, the stable system is a vision inspection system installed over 15 years ago at significant cost. The change was an initiative by Microsoft to raise security standards for data networking in Windows software.
At the plant, IT systems are deeply integrated with the manufacturing process. The vision inspection systems, for example, have a direct, real-time connection to the main controller using OPC DA, an industrial protocol based on Windows COM technology.
Networking OPC DA requires DCOM, whose security settings are complicated to configure. To run the system efficiently, the plant automation team had always minimized DCOM security. However, with the mandatory application of the Windows security patch from Microsoft, only the two highest levels of DCOM authorization are permitted. Configuring and enabling these higher settings created problems. The plant had other security requirements that made recovery from any OPC disruption extremely difficult and time-consuming, causing multi-hour production delays.
A new approach: Tunnelling
What Doody needed was a different way to connect the OPC clients on the PCs running the vision control systems with the OPC server on the main controller. A web search on DCOM issues brought him to the concept of OPC tunnelling and Cogent DataHub software.
“While investigating tunnelling I found the Cogent DataHub website, read the case studies and support testimonials, and learned how the application was easy to integrate into systems like ours,” Doody said. “It looked like once I implemented, I would then hand it off to our production support group, and they could take it over without me having to supervise.”
For a trial, on one production line he configured a tunnel connection between the main controller PC and four vision system PCs. Originally, security for both the vision system PCs and the main controller PC was configured on the plant LAN. All logins were done through the Active Directory domain controller and had to be identical.
With the tunnel, the DataHub instance on the main controller connects to the OPC server using the normal login credentials. The DataHub instance on each vision system PC makes a tunnelling connection to the DataHub instance on the main controller and receives the data. Each of those DataHub instances is configured as an OPC DA server, allowing the vision system to connect as an OPC client.
Now, because the OPC client on each vision control system connects to a local DataHub instance, Doody has been able to remove those PCs from the plant LAN. He no longer needs to enforce user logins with the domain controller. Each user logs in independently of the main control login. Any irregularity or dropped connection no longer requires re-synchronizing security logins across multiple machines.
After a week of testing the first system, Doody felt confident to implement the solution on the remaining three lines. Now all connections are using DataHub tunnelling, and the benefits are clear.
“Before, if an OPC connection dropped, it could take anywhere from maybe one to five hours to get the PCs that were off the domain back on, get signed up, authenticate with the user account, initialize network cards, connect to the plant LAN and line PC, connect to OPC, calibrate the application, and more,” said Doody.
“Now I’m getting no reports of any machine down time,” he continued. “Our application is even faster connecting to the local instance of the OPC server. Even before the DCOM hardening stuff, we would get intermittent dropouts between our clients and servers. The only error message we’d get would be, ‘Check OPC client installed.’ No detail whatsoever. Very little error logging.
“DataHub tunnelling makes it so much easier to debug why the connection is failing. So far we’ve had one instance where OPC failed to connect on our application, and looking at the DataHub Data Browser we could see that it was because the tunnelling application wasn’t running on our server side. Before, that would have been a shrugging of shoulders and numerous reboots of the PC waiting and hoping that it would re-establish this connection.
“And it wasn’t a call out to an engineer to investigate the problem. It was one of our process support guys that was able to diagnose and fix the problem themselves. That’s another big win for us.
“The whole solution was pretty much a no-brainer for the management, and easy to sell to the financial controllers, because it was saving a heap of money, rather than upgrading the lines. Engineering was brought in as well, because it was easy to implement and really easy to maintain. And then the proof of concept, getting the test licenses so easily, reading the How-Tos to implement it on our systems—it was fantastic, seamless, for our work.”
