Big problems and small solutions

In every CTO’s mind, an overarching Big Problem drives his or her higher quest for technical excellence beyond the mundane business of getting day-to-day things done in IT. The Big Problem is not necessarily the one that keeps you up at night — it’s the one you think about while you’re sitting by the pool on vacation or taking a long hike.

To a large degree, succeeding as a CTO depends on an unusual mix of idealism (“One day, all of our systems will work seamlessly”), pragmatism (“If X, Y, and Z happen, our entire back end could crash and burn in 10 seconds”), and persistence (“We knocked one problem down — only 15 more to go”). A good CTO recognizes real, day-to-day technical limitations, but refuses, ultimately, to accept them. He or she focuses instead on the daily incremental gains that push toward the longer term of solving the Big Problem. The idealist thinks a grand solution might be just around the corner, but the pragmatist wouldn’t be too surprised if an elegant solution never appears.

The Big Problem is different for every CTO, but we all have one. In the traditional media industry where I work, the Big Problem centers around content management and workflow in an environment with multiple distribution formats.

As non-media companies continue to leverage the Web in their businesses, my Big Problem has become, in a nutshell: Any content produced for print, television, newspapers, and/or magazines must be easily repurposed for other media, especially the Web, with minimal human intervention. I’ve spent most of the past 10 years pushing and pulling content into and out of systems including Basys for TV broadcast, Atex for newspapers (now AMC), and Quark Publishing System for newspapers and magazines. These systems have changed though the years, and some don’t even exist anymore. But many of the challenges still remain. Moving the content around has gotten somewhat easier, but the process in most companies I’ve seen is still held together with various forms of integration chewing gum.

Recently, I decided to take my Big Problem on the road, thinking that some exposure to what similar companies are doing might lead to a grand “aha!” moment that would solve all my problems. My idealistic CTO side was running full throttle as I planned a meeting with a group of print and online staff at a large publishing company to discuss print and online content integration. To temper my idealism, I brought Kevin Varley along with me. Kevin is one of our developers, and as someone with his hands deep in the goo of the code behind our content management systems, I was confident he would provide valuable on-the-ground perspective.

As we hopped on the elevator to go to the meeting, I thought maybe we would find the Holy Grail of print/online content integration, the solution to the Big Problem in my industry. Kevin said, “Wouldn’t it be funny if their solution was basically just a bunch of Perl scripts?” I laughed nervously — I really wanted the Grand Solution.

The meeting was incredibly informative and useful. We learned a lot about how another publishing company handles some of our common issues. In the end, though, Kevin was right. When it came to solving our Big Problem, there was a lot of custom-coded Perl on the back end. There’s certainly nothing wrong with that, and I guess I wasn’t really that surprised. I just know the Big Problem can be solved elegantly, though, so I’m not giving up.