Debugging SOAP

Andrew Schulman, the author of Undocumented Windows, used to lecture on the many mysteries he had unraveled deep in the bowels of the VxD (virtual device driver) realm. “How do I know all this?” Schulman would ask rhetorically. The answer was NuMega’s (now Compuware’s) SoftICE, a go-anywhere, do-anything debugger that’s prized for its ability to trace execution from Win32 apps into DLLs, then into the kernel, and then back again.

I first met NuMega’s core team, Frank Grossman and Jim Moskun, at Fall Comdex in 1988. During this year’s Comdex week, I caught up with them again — not in Las Vegas, but in the Hollis, N.H., offices of their new venture, Mindreef. After selling NuMega to Compuware and taking some time off, they’re back in the game with a new debugger called SOAPscope. The first version, SOAPscope Personal, has been earning rave reviews from the pros on Yahoo’s soapbuilders list. It’s a sniffer (or, optionally, a proxy) that watches, simplifies, and can interact with SOAP traffic.

This might seem like a trivial problem for veteran debugger-builders to solve. Having spelunked the scary depths of the Windows VxD architecture, isn’t tracing SOAP packets pretty much like shooting fish in a barrel? It’s “just” human-readable XML — for some definition of “human,” anyway.

Frank Grossman chuckled when I asked him that question, and agreed that on one level Mindreef is tackling an easier problem. But in another (and arguably more important) sense, the loosely coupled systems we are proposing to build out of XML Web services present interaction complexity of a much higher order than interrupt-driven device drivers and event-driven GUI applications. Frank thinks we ain’t seen nothin’ yet in terms of debugging challenges. I think he’s right.

The $99 SOAPscope Personal is admirably small, simple, and focused. You fire it up, and can immediately begin to explore the SOAP traffic on your local machine. SOAP packets and WSDL files can be viewed as raw XML, but in many cases the companion “pseudocode” will be the view of choice. The SOAP envelope wrapped around a method call is, let’s face it, kind of hairy.

SOAPscope’s pseudocode strips away the clutter and gets you down to the bare bones; for example, getQuote(string symbol = “ABC”). A related benefit is normalization of WSDL-defined data structures, which various SOAP toolkits may serialize into differing XML representations. SOAPscope’s pseudocode converts these to a neutral syntax. That will be really helpful during the current phase of SOAP’s development, when interoperability among SOAP toolkits is still very much an issue. For example, there is still not yet an agreed-upon way to serialize hashtables as XML.

When debugging a loosely coupled system, it’s nice to have an out-of-band signaling mechanism that can convey information that’s about a SOAP packet without contaminating that packet. SOAPscope supports this with per-toolkit libraries that can be used to emit out-of-band trace statements (“annotations”) that are associated in the log with SOAP packets and can be viewed in the context of those packets. The viewer’s interface is HTML, generated by an embedded Tomcat server, and these annotations can therefore use HTML to emit fragments of help text, or links to a knowledge base.

Packets are stored in an embedded database and can be re-sent to the original (or another) endpoint, with modifications that try different values and test boundary conditions. The Mindreef team thinks this kind of thing is liable to occur at any point in the life cycle of a Web service. The traditional develop/test/deploy cycle, they believe, will blur into continuos process improvement as Web services grow increasingly interdependent. So the goal is to be able to run the logger and viewer all the time, even in production — something that wasn’t feasible for the highly invasive SoftICE, but may well be in this case.

For an InfoWorld article next week, I interviewed IBM’s VP of autonomic computing, Alan Ganek. All of the fancy self-monitoring and self-tuning features in IBM’s software products, he said, ultimately depend on consistent logs and high-quality diagnostic tools. With the original NuMega team back in action and on the case, I’m feeling better about the prospects for autonomic Web services.