After-hours IT: A phone hack exposed

IT pros, take heed: No matter how perfect your installation may seem, there’s always a lurking security hole you didn’t anticipate, as I learned during an otherwise pristine VoIP implementation.

I work for a midsize business that has many remote locations, each employing several salesmen. Most of them are in their mid-fifties or older, so they use email grudgingly. But both they and most of their customers prefer a phone call rather than the impersonal email, so weaning the sales staff from landlines is not likely to happen soon. Phone bills were high, and the managers wanted us to find a cheaper solution.

[ The IT job landscape changes quickly. Here’s how to avoid being one of these 9 most endangered species in IT. | Follow InfoWorld’s Off the Record on Twitter for tech’s war stories, career takes, and off-the-wall news. | Subscribe to the Off the Record newsletter for your weekly dose of workplace shenanigans. ]

We started with one location, did our research, found a vendor, and were ready to make the leap from copper POTS lines to VoIP. The initial tests showed a much clearer voice transmission, since our warehouse was in a historic part of town, where the copper trunk lines and junction boxes were very, very old. (I swear I saw one with the initials A.G.B. inside!)

If all proceeded as planned, by using an Internet connection we could peel several channels off for voice while also beefing up our Internet speed. We could easily save hundreds of dollars a month, with the potential for more if we added other locations to the network.

Day one: We arrived at the offsite location and began unpacking new handsets and installing them. These handsets worked off the same CAT5 cable that connected to the desktop computers.

We had been fortunate to purchase VLAN-capable switches when we recabled the building a year before, not realizing how soon we’d need the functionality. We had to run a few new CAT5 drops to supply a couple of desks that were used for truck drivers and the lunch room where no PC existed at the time, but that wasn’t really a problem and was quickly remedied.

Lunchtime came and the rollover from copper was under way. There were the normal glitches that pop up anytime new handsets are deployed — mostly due to people who don’t like change — but nothing major.

The rest of the afternoon was spent in minor instruction mode to acquaint the users with the various functions of the new devices. New recordings were made for busy lines, voicemail, out-of-office and after-hours rings. One phone was located in a common area that was to be open early in the morning with little supervision, so we made sure to have the vendor block long-distance calling on it.

The rollout seemed to progress well, and the end of day came. We were tired; we’d left our office at 4 a.m. to be at this office at 8 a.m. But we were satisfied, telling each other that things were going better than we had anticipated.

At 5 p.m., we exited the building with the rest of the employees, and the manager locked the door. This was when the phones were set to switch to night ring, so we stood outside and called the office to test it. All worked perfectly — after a quick high-five, we were off to a good meal and a solid night’s rest at the hotel.

The next day at 8 a.m., we were back at the offsite location, ready to test everything once more before returning to the home office. At 8:15 a.m., we got a call from the vendor inquiring if we’d had a break-in the previous evening, occurring just after 5 p.m. We were quite sure we hadn’t since we’d been standing outside the door talking until 5:15.

He explained that shortly after the system went to night ring, there were 19 calls of 10 seconds or less placed to a number in Africa. Each call was charged at several hundred dollars. Fortunately, the vendor’s tech staff were online, noticed the call log, and immediately disabled international calling from that location.

We quickly verified that those calls had not originated from our site, and a couple of hours later, the vendor reported back with the results of the inquiry.

It turned out one of the vendor’s former employees had stolen and cloned the credentials for one of the phones given to us. Though he was no longer employed by the company, he apparently still had access to monitor the phones, which he used to watch for the same media access control ID to be deployed.

At the same time, the ex-employee registered a phone number with an African phone service, similar to the way a 900 number works in the United States. Just by dialing such a number, the caller consents to the charges, which can run into several hundred dollars, depending on how the person sets it up. It soon became a waiting game for him.

When the opportunity arose, he acted quickly. He could tell we were on the East Coast and simply waited for 5 p.m., figuring it was the common quitting time. He called the office to test for a night ring, got it, and began to dial the number in Africa over and over again before the vendor saw and cut him off. The vendor understandably didn’t disclose any more details about the depth of the ex-employee’s shenanigans.

Thankfully, we didn’t have to pay that bill, but I now have an extensive pre- and post-rollover questionnaire that I cover with vendors before and after jobs, with frequent updates and additions to the list. If anything, this incident reminded me the bad guys never quit, and neither can we.

Do you have a tech story to share? Send it to offtherecord@infoworld.com. If we publish it, you’ll receive a $50 American Express gift cheque.

This story, “After-hours IT: A phone hack exposed,” was originally published at InfoWorld.com. Read more crazy-but-true stories in the anonymous Off the Record blog at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.