Class project: Crash the federal data center

analysis
Jun 6, 20126 mins

Disaster turns into discovery as one techie's homework assignment wreaks havoc on a government mainframe

Every generation thinks it’s invented the wheel — or the computer or the killer app. But technology in one form or another has been around for a long time, along with tech problems, surprises, and suspicion of foul play. In fact, years ago I inadvertently played a role in a system crash that at first put me under suspicion, then led to a bigger discovery that benefited the manufacturer as well as customers.

This story goes back to the mid-1970s when I was a lieutenant in the U.S. Navy working in computer support for a division of the Defense Communications Agency. Occasionally, my job required a small amount of assembly-level coding. I already had some basic skills in that area, but my boss decided I should attend a two-week advanced assembly-level language class.

[ For stories about mistakes IT pros have made, read “True tech confessions II: Sinners and winners.” | Follow InfoWorld’s Off the Record on Twitter for tech’s war stories, career takes, and off-the-wall news. | Subscribe to the Off the Record newsletter for your weekly dose of workplace shenanigans. ]

The class was in the nearby Washington, D.C., area. For our class “projects,” we used a Honeywell mainframe owned and managed by the U.S. Customs Department. Class members were allowed to use time-sharing and could submit batch jobs via punched cards.

I liked this arrangement because we used Honeywell mainframes for several critical and highly classified applications at the Defense Communications Agency. I could do some of the class projects from my office, then turn them in when I got to class.

For one class assignment at the end of the first week, I did my work back in my office over the weekend, but I ran into a small snag. I had heard, and soon confirmed, that the collating sequence for the military version of the Honeywell GCOS operating system was slightly different than the civilian version used by the Customs Department’s system. But there was a utility program that could produce a compressed version of the card deck of my code that could then be uncompressed on the Customs Department’s computer, so I could then show my work to the instructor.

The first part worked, and I dropped off the card deck the next Monday before class, expecting to pick up my output the next day to turn it in to the instructor. Was I in for a surprise!

When I came in on Tuesday, I couldn’t find my card deck among the others. Worried that my class project was lost and I’d have to redo it, I asked the operations person on duty if he knew where my output might be. He asked me who I was, and I told him. He got a strange look on his face and said, “Ohhhh. My manager wants to see you right away.”

Puzzled, I went to the data center manager’s office and introduced myself. I noticed that my card deck and output listing were sitting on his desk. His first question to me was a strident, “What do you think you are doing?!?!”

I told him I had submitted the uncompress job for my class project and asked him if there was a problem. He told me I had crashed their mainframe — three times!

I was horrified. But then I thought for a second and asked, “Why three times?”

It turned out they had run my job the first time in a batch with all of the other class projects. The mainframe crashed, but that was somewhat expected, as system stability was not what you’d call “six sigma-ready” by any means.

After restarting, they ran the class project decks a second time — and the mainframe crashed again. By then, they had a clue that the batch of card decks was the culprit. After restarting yet again, they ran each batch job deck one at a time. When they ran mine, it crashed the mainframe for the third time.

I was as surprised as they were about what happened, and apparently the manager believed me — but he held on to my card deck while I tried to figure it out. The instructor also gave me time to research what had happened.

It turned out I had inadvertently uncovered a bug in the operating system. GCOS used two-character file codes for input and output file designations. The default value for an input file was I*, which is what I coded for the uncompress utility. But it turned out that running this utility program with the default file code of I* caused an immediate system crash. The file code that worked was IN.

I corrected my card deck and convinced the data center manager to rerun my class project. I was proud that it worked correctly on the first compile after the code was uncompressed. The data center manager seemed relieved it was such a simple problem, the instructor accepted my project late, and the company was notified of the bug, which was eventually fixed.

I later found out that the reason the data center manager was so suspicious and upset: This Customs Department mainframe contained highly confidential data, such as locations where seized contraband was located. He thought I might have been trying to access sensitive information.

Hearing this, I was even more relieved that I’d convinced him that what happened was accidental and not a threat. On further reflection, I told myself that while it’s necessary to be cautious, I would never first assume malicious intent when someone did something to cause problems in a data center — a notion that’s served me well to remember many times since.

Do you have a tech story to share? Send it to offtherecord@infoworld.com. If we publish it, you’ll receive a $50 American Express gift cheque.

This story, “Class project: Crash the federal data center,” was originally published at InfoWorld.com. Read more crazy-but-true stories in the anonymous Off the Record blog at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.

infoworld_anonymous

Since 2005, IT pros have shared anonymous tech stories of blunders, blowhard bosses, users, tech challenges, and other memorable experiences. Send your story to offtherecord@infoworld.com, and if we publish it in the Off the Record blog we'll send you a $50 American Express gift card -- and, of course, keep you anonymous. (Note that by submitting a story to InfoWorld, you give InfoWorld Media Group, its affiliates, and licensees the right to republish this material in any medium in any language. You retain the copyright to your work and may also publish it without restriction.)

More from this author