Monitoring and Discovery in Grid environments

By nature, Grids have exponentially more moving parts than the typical, silo’d architecture that today’s enterprises are moving away from, where dedicated hardware stacks support specific applications in a very static way. In Grids, resources come online into the production environment, and they can just as quickly go back offline. Jobs start, take up a certain quanta of resource to run, and then that resource is freed up again for the next job.

So how in the heck do Grid pros get real-time insight into what’s happening, where and when?

In Globus Toolkit Grid environments, this need is satisfied by the Monitoring & Discovery System (MDS), the “information services component of the Globus Toolkit [that] provides information about the available resources on the Grid and their status.” On the client side, users reference the MDS user interface to get the real-time view they need of their Grid resources. On the “provider” end, MDS allows Grid participants to create the necessary interfaces that allow other users to get access to their resources.

This is done using meta data constructs in XML format.

According to Jeffrey Hollingsworth and Brian Tierney (in Grid 2: “A major focus of MDS’s design is achieving scalability in a system with large numbers of information providers and consumers.”

“MDS uses a variety of plug-in modules to ingest XML data,” added Mike D’Arcy, programmer at ISI. “From processes, to files, to remote servers, to connections over a socket to a non-WSRF service — the system is very flexible on ingest methods. Anything that’s a WSRF service and that publishes its own resource properties can very easily be pulled into MDS using its mechanisms. You can also advertise any kind of data that’s expressed in XML by creating your own provider.”

One of the key differences between research and enterprise grid computing is the need for accountability. MDS presents an interface where this data, critical to enterprise, can be ingested and aggregated. MDS may indeed be the catalyst the Globus Toolkit needs to spark its evolution from science and academics into a key player in enterprise Grid.