by Greg Nawrocki

Univa Shows Its Open Source Chops

news
Mar 6, 20073 mins

Univa Corporation recently announced the open source release of the Univa Data Distribution Manager (DDM) as a Globus Incubator project.

Univa DDM is a new service that builds upon the robust features provided by the Globus Toolkit including the C and Java Web Services Core, GridFTP, GRAM, and RLS (among others) by adding the capability for data set specification, a range of synchronization modes, replica selection, and a comprehensive set of policy based failure recovery mechanisms.

Key features of this current release include:

Multi-site replication with updates: Univa DDM supports both a write-once / read-many and a sequential update model for managed files. It does so in a multi-storage resource setting, by tracking actual file contents and replication history then applies replica selection criteria for choosing the best data source for a particular request.

Synchronization of changes only: Univa DDM is a critical component for the management of replicated data sets. File synchronization may be accomplished at a full or partial file level. It is, at its essence, a reliable multi-site data resynchronization service.

Fault tolerance: Configurable policies for backoff and retry, including failover to alternate sources in the instance of multiple file replicas. The Univa DDM service manages process state so that it is self-resilient to server failures.

There are plans in the next version for the addition of the following:

Extensible data set descriptions: Univa DDM will have an extensible data set description model, that will have the ability to plug into external metadata catalogs via a Java API. This will allow for a rich and extensible process control experience that will facilitate the integration of Univa DDM into a wide range of environments for use by client applications.

Service availability: This advanced system for submitting and monitoring data management requests in a distributed environment will take into account the state of physical network connections and make intelligent decisions regarding their use. This will give clients a new reliability of data management, for an unprecedented level of fault tolerance.

In a nutshell Univa DDM provides an efficient data distribution service for tracking, transporting and synchronizing large-scale, distributed data sets. It does so with an emphasis on driving down the cost of sharing large data sets through efficient use of available bandwidth and storage while ensuring secure and reliable operation. This makes Univa DDM attractive to enterprise users of the Globus Toolkit, particularly those with large-scale data management requirements.

While Univa has, since it’s inception, rallied support around the open source Globus community, this open source release of Univa DDM marks the first large-scale contribution by Univa as a true contributing enterprise partner to the Globus community.

According to Dr. Carl Kesselman, Univa’s Chief Scientist:

“What DDM does for the data landscape is that is provides much richer capabilities for distributed file management that is available in any of the current Globus tools. We believe this will benefit the Globus open source community by providing a set of valuable new capabilities targeted at data-intensive technical computing needs. By contributing Univa DDM to the Globus open source process it helps increase the applicability of Globus based grid solutions in a wider range of scientific research and enterprise computing environments, and in turn advances the goals of the company.”