The future is now

GRID TECHNOLOGY, ONCE just a glamorous vision for marshaling computing power to link far flung users for collaborative projects, is gaining traction. And as CTOs begin to develop these new distributed networks that offer computing power as a utility, new technologies — from high speed VPNs to server clusters to Web services integration platforms — are being included.

Grids at present are like the early stage of the Internet, a staging ground for new technologies, says Wes Kaplow, the CTO of government services at Qwest Communications in Denver. Qwest is partnering with the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign and laboratories and colleges throughout the United States to build the TeraGrid network, one of the largest grids yet constructed.

The TeraGrid, Kaplow says, “is a leading indicator of where high-performance, commercial, distributed supercomputing environments are going to go. Who will be the users? Aerospace, pharmaceuticals, automobiles — every industry that is multinational in scope with resources in more than one location.”

Grids are being deployed in a wide variety of ways, as befits an emerging technology, says Stacey Quandt, a Linux and open-source analyst at Cambridge, Mass.-based Giga Information Group. “At its core, grid computing is ‘plumbing’ that enables distributed resource management and services on demand, and promises to be the evolution of the Internet into a computing platform,” Quandt says.

Among the promises of grid computing, Quandt says, are the enablement of resource sharing across multiple organizations, the dynamic migration of applications, and the ability to support the sharing of resources such as computing power, storage, and applications.

Four times the capacity

The TeraGrid has emerged as a playing ground for new technology deployment, says Daniel A. Reed, director of the NCSA and lead manager of the TeraGrid project.

The TeraGrid links NCSA with the Argonne National Laboratory near Chicago, with the San Diego Supercomputer Center at the University of California, San Diego, and with the Center for Advanced Computing Research at the California Institute of Technology in Pasadena, Calif.

Scientific teams from around the world will be able to collaborate to tackle large projects. For example, scientists working on earthquake engineering need to assemble reams of information on contraction data and topography.

“We are looking at ways to bring people together who do building design, disaster management, and who work on [structural] stress so they can all experiment with data and computer models,” Reed says. “No one site has all the information.”

Qwest has helped build 10GB fiber networks for the grid, stretching from Illinois to California, and has plans to connect a 40GB central backbone between Illinois and central California — four times the capacity of the current fastest networks for research and education. “The TeraGrid network is a mission network, designed to test a bunch of things, like how efficient can you create a high-speed network and get supercomputing in different geographic locations to process and move data around,” says Qwest’s Kaplow.

The sites on the grid have a variety of server clusters and storage systems handling huge amounts data. For example, the San Diego site uses an IBM Linux cluster based on Intel Itanium processors, with peak performance of more than four teraflops and 225TB of network disk storage.

NCSA’s Reed says there is growing interest in gaining access to Web services applications such as Microsoft’s .Net platform. “The ability to interoperate across Linux and Windows is also part of the story,” Reed says.

The grid enterprise case

Whereas the NCSA and others are building global grids, smaller dedicated enterprise grids are also gaining popularity as CTOs see their ability to manage large data sets.

Caprion Pharmaceuticals, a Toronto-based biotech company that analyzes proteins to help biopharmaceutical companies develop products, built a massive protein-analysis computing farm last year, including a data warehouse and a management information system. The data farm runs on Sun Fire servers managed by Sun Grid Engine software to handle the workload, says Bernard Gagnon, Caprion’s director of IT.

The company, Gagnon says, needs enormous computing capacity to analyze thousands of proteins and terabytes of data. “When we looked at a computing solution, we looked at supercomputers and grids and we found that grids are more flexible.

“The cost of the hardware was a big differentiator. Supercomputing will cost more,” Gagnon says. A grid model enables CTOs to choose the hardware they need in a modular fashion, he adds.

The Sun Grid Engine acts as middleware that aggregates available computing resources with multiple users, teams and departments and allows them to share resources while working on projects with different goals and schedules.

“With the grid we have the power we need and we can pick and choose the types of machines we use,” Gagnon says. “As we add more jobs, we can add more machines.”

Real-life applications

At the University of Pennsylvania, groundbreakers have created a grid to allow doctors, researchers, and hospitals to share data relating to breast cancer from its National Digital Mammography Archives, says Dr. Robert Hollebeek, director of the university’s National Scalable Cluster Lab. Hollebeek says the system grants doctors and other users instant access to important data about cancer diagnosis and treatment.

The University of Pennsylvania, the University of Chicago, the University of North Carolina at Chapel Hill, and the Sunnybrook and Women’s College Hospital in Toronto were first selected for the project. The grid was built in collaboration with the Oak Ridge National Laboratory and was funded by the National Library of Medicine. “These were all hospitals with an emphasis on mammography research,” Hollebeek says. The goal, he says, is to link the system to hundreds of hospitals, allowing them to tap into data storage systems.

The system combines IBM Unix and Intel processor-based systems with IBM’s DB2 Universal Database and uses open protocols from the Globus Project, a nonprofit open-source grid-computing project. Web-based interfaces will allow doctors to authenticate themselves and tap into the system.

The North Carolina Genomics and Bioinformatics Consortium has formed the NC BioGrid, representing more than 70 academic and commercial organizations. The group was formed to create a statewide grid and promises to become a leader in genomic research. The fledgling organization provides computing, data storage, and networking to link the University of North Carolina at Chapel Hill; North Carolina State University in Raleigh; Duke University in Durham, N.C.; and the North Carolina Super Computing Center in Research Triangle Park.

The BioGrid takes advantage of existing network infrastructure at the institutions and also deploys cluster servers using a variety of vendors, as well as grid middleware from Avaki, says Phil Emer, chief architect of the NC BioGrid.

As with grid computing in general, there are great plans for the NC BioGrid, with planed expansion statewide and possibly throughout the country, Emer says. “We see ourselves as the reference implementation for a biogrid,” Emer says.