Short circuits and tall orders: Tech trouble at the steel mill

Maybe it’s the sense of mounting frustration that makes certain tech repairs so memorable. It’s hard to forget the pressure of working on a problem that should be fixed quickly, but the solution slips from your grasp longer than expected. One attempt after another fails, until — finally — the pieces fall into place. A few years ago, my tech team racked our collective brain to solve such a puzzle.

At the time I worked at a steel mill. We deployed four cranes that had computers with touchscreens and custom applications running on them to allow the operators to identify which piece of molten steel they were moving. Our systems had to track every segment of steel going down the line and were dependent on the operators in two locations to positively identify each piece.

[ Ditch the slackers, take on dirty work, do it with data: 12 effective habits of indispensable IT pros. | Follow InfoWorld’s Off the Record on Twitter for tech’s war stories, career takes, and off-the-wall news. | Subscribe to the InfoWorld Off the Record newsletter for your weekly dose of workplace shenanigans. ]

These cranes — and their accompanying computers — were in motion much of the time and exposed to extreme temperatures, shock and vibrations, electromagnetic interference, and metallic dust. Regular maintenance was required to keep the systems working properly. However, any maintenance had to be scheduled for when the crane was parked and not in use because the equipment had to be carried up several flights of stairs and across a catwalk to reach the crane.

On some occasions, emergency repairs or observation had to be performed while the operator was using it, though there wasn’t much space inside these moving cranes for two people to work in.

One day, we started having problems with the network connection in a particular crane. The computer’s network connectivity would be lost for a short time, restored, then lost again — repeatedly — until at some point there was no longer any connection.

This specific overhead crane contained a car battery for power, which was kept charged via an electrical contact with a rail that ran adjacent to the crane. Equipment in the crane was then powered via a DC-to-AC converter. For network connectivity, the computer had a CAT-5 cable connected to a wireless device inside the crane.

First off, we tried a new computer. We radioed the operator to bring the crane to the dock so that we could swap out the computer with another just like it.

Nothing doing. The problems with the new computer inside the crane remained. However, the computer that had been removed from the crane worked just fine inside the office where we were checking it for problems.

The next thing we tried was swapping out the crane’s wireless network equipment. The problem remained.

We then swapped out the CAT-5 cable. Again, the problem persisted.

We regrouped and stepped back from the problem. The computer we’d taken out of the crane was working just fine in the office. We thought through the differences between the environments, narrowing it down to the sources of power. It was worth exploring.

We called the operator to once again come to the dock, carried a UPS unit up to the crane, and plugged the computer into the UPS instead of the power outlet. The computer immediately started communicating on the network. Aha!

It was time to call in the electricians. They investigated, fixed the problem, and reported back that some kind of electrical short or loose connection had caused the power outlet to lose proper grounding. As a result, the network cable was serving as a ground wire, interfering with the network communication.

We were thrilled the issue was finally fixed and moved on to the next one in the queue. Remembering these incidents may not ease the frustration levels the next time around, but it’s definitely a sense of accomplishment when a stubborn problem gets solved.

Do you have a tech story to share? Send it to offtherecord@infoworld.com. If we publish it, you’ll receive a $50 American Express gift cheque.

This story, “Short circuits and tall orders: Tech trouble at the steel mill,” was originally published at InfoWorld.com. Read more crazy-but-true stories in the anonymous Off the Record blog at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.