The next Ethernet

SAN FRANCISCO – At a recent event celebrating its 30th birthday, the trailblazers of Ethernet marveled at its evolution from a cable between copiers that ran a bit faster than today’s home broadband services to the key technology in huge enterprise networks and even carrier data services. A recurring refrain was that Ethernet today is nothing like what they invented.

Bob Metcalfe, who as a researcher at the Xerox Palo Alto Research Center (PARC) wrote a memo describing Ethernet on May 22, 1973, recalled realizing a few years later that the fledgling technology was outgrowing its origins at the sleek lab nestled amid rolling hills in Palo Alto, California.

“I remember thinking in 1982, ‘There are people buying Ethernet I have never met,’ ” Metcalfe told an audience of networking pioneers and reporters at PARC last week.

From its first incarnation as a shared packet network that ran at about 3M bps (bits per second), Ethernet grew into a popular 10M bps technology, gave each user their own dedicated bandwidth with switches, helped spawn IEEE 802.11 wireless LANs and eventually found its way into carrier data networks. Along the way it supplanted a long list of rival technologies for LANs, most notably Token Ring and ATM (Asynchronous Transfer Mode). For that Metcalfe credits most what he called the Ethernet business model: a formal standard with fierce competition among different implementations that nevertheless can work together, with continuing evolution of the standard that never leaves users of older versions stranded with useless equipment.

If some vendors, service providers and research companies are right, Ethernet’s acceleration isn’t over yet. However, it may slow down to a less dizzying rate.

Despite the fact that the fastest Ethernet link now, at 10G bps, theoretically can run a thousand times faster than what connected PCs only a decade ago, the industry’s track record seems to call for a 100G bps speed within a few years.

“We’ve shifted the decimal point, on average, every four years. If that’s true, some time this year, we should be starting to shift the decimal point,” said Bob Grow, chair of the IEEE (Institute of Electrical and Electronics Engineers Inc.) 802.3 working group, the body in charge of Ethernet standards. Grow is also a principal architect at Intel.

But a not-so-funny thing happened on the way to the network of the future: the IT and telecommunications crash, hitting hard in 2001 with cutbacks in enterprise technology spending and a glut of data capacity on carrier networks. Now the talk is more of 40-Gigabit Ethernet and a wait of several years for broad use of it.

The current fastest Ethernet, which was ratified as a standard last year, is more than enough for most carriers and enterprises so far, several analysts and executives said. Gigabit Ethernet is the top speed in the backbones of most enterprise networks, where traffic from many servers and departments comes together, and on Ethernet data services from carriers to corporate customers.

“Right now, (10-Gigabit Ethernet) is coming onto the scene,” said Bob Smith, senior director of emerging products and services at Atlanta-based BellSouth, a major provider of Ethernet data services. Though about 20 percent of the carrier’s Ethernet customers are using Gigabit Ethernet services, “We have some customers up on gigabit speeds now that are wondering what to do with the other half of the bandwidth that they’re not using,” he said.

“Typically, it doesn’t make sense to do the next speed until you have (widespread) adoption of the current speed,” IEEE’s Grow said. For the length of time it has been available, 10-Gigabit Ethernet has had slower uptake than Gigabit Ethernet had at a similar point in the 1990s, he said.

For some potential users, including carriers, existing investments might hold back adoption of a faster technology for several years, according to Kevin Dillon, director of portfolio marketing at router vendor Juniper Networks, in Sunnyvale, California. Many have large investments in fiber infrastructure that won’t work with anything faster than 10-Gigabit Ethernet. They would need to wait until that gear is fully depreciated, Dillon said.

Whenever the next leap is taken, there are technical reasons to make the shorter step to 40G bps rather than develop 100-Gigabit Ethernet, analysts and executives said.

Ethernet interfaces at that speed would match up with traditional WAN links at OC-768 (also about 40G bps), making it easier to design equipment and networks, said David Passmore, an analyst at The Burton Group, in Midvale, Utah. Vendors making 40-Gigabit Ethernet equipment also could leverage some of the work that went into developing interfaces in the past few years for the wide-area specification OC-768 (also about 40G bps), he added.

“Forty-gig in the wide area was all the rage two years back when everyone thought Internet demand was growing exponentially,” Passmore said. Although not much demand has materialized for such ports, chips developed for them could help to form the basis of 40-Gigabit Ethernet interfaces, he said.

At least one executive at Ethernet giant Cisco Systems believes 40-Gigabit Ethernet could be technically feasible within two years. Luca Cafiero, senior vice president and general manager of switching, voice and storage, said the technical hurdles would be far less daunting for 40-Gigabit than for 100-Gigabit Ethernet. In fact, Cisco Catalyst 6500 Series routing switches already can be equipped to support 40G bps per interface cards, and a card might be built with a single 40G bps interface, he said. 

Other equipment vendors also are lining up for this speed. Router maker Juniper Networks and high-speed Ethernet switch pioneer Extreme Networks also say they can now support 40G bps of throughput on a line card.

However, at least one long-time Ethernet engineer doubts the industry will set a standard at 40G bps.

“Forty-gigabit will not be economically viable in terms of the effort the industry has to put in to develop a new technology,” said Nan Chen, director of product marketing at Atrica, an equipment maker in Santa Clara, California, and a veteran of the Fast Ethernet standards process in the 1990s.

That speed could be achieved simply by bringing four 10-Gigabit Ethernet connections together using link aggregation, a much less expensive proposition than developing a new kind of interface. If a whole new standard is to be developed, it should go all the way to 100G bps, he said. Chen said he would not be surprised to see a formal call for interest in 100-Gigabit Ethernet at the IEEE in 2004, a standards development process of two years or so, and products in 2006.

“The economy may not be conducive for that kind of development, but if you look at the developments so far, that would make sense,” Chen said.

There may be a way around having to create a new Ethernet interface that can handle 100G bps, he added. Ten 10-Gigabit Ethernet connections could be combined within a device using WDM (wave-division multiplexing), going out through an optical 100G bps interface. The current 10-Gigabit Ethernet standard includes an option for doing this using four 2.5G bps links, he said.

Other experts say 40G bps could be worth the work.

In the past, Ethernet has been used mainly in LANs inside an organization’s own building. Aggregating four fast links was relatively inexpensive in that setting, so there was no need to change gears until 10 times the speed was needed, Burton Group’s Passmore said. Now fat Ethernet pipes are also used over long distances, where networks typically are more expensive to roll out and connection speeds have gone up roughly by multiples of four, most recently from OC-48 (2.5G bps) to OC-192 (10G bps) to OC-768.

“In the WAN, going by 4x is perfectly adequate,” Passmore said.

Formal work on an Ethernet standard faster than 10G bps hasn’t even started, according to Grow, the working group chair. No one has yet submitted a call for interest for a new Ethernet at either 40G bps or 100G bps. But that may just be the quiet before the storm.

“There would be significant disagreement on what is the correct next speed within the committee,” Grow said. Once a standards process begins, it typically takes three years to finish, he said.

Actual demand for the technology is another question. Some observers don’t see a faster Ethernet in wide use until 2010 or so.

“We still have a long way to go before we’re through aggregating Gigabit (Ethernet) up to 10-Gigabit,” said Dave Dunphy, an analyst at Current Analysis, in Sterling, Virginia.

Even BellSouth’s Smith, who is optimistic about the eventual need for more speed, is taking the long view.

“Ten years out, 40-Gigabit will be deployed as commonly as 1-Gigabit is today,” he said.

Juniper’s Dillon hinted at a glimmer of demand for a faster Ethernet. Some of the company’s carrier customers are already aggregating multiple 10-Gigabit Ethernet links between their facilities at the edge of metropolitan networks, he said.

The early market for a faster Ethernet might be limited to the cores of campus networks, links between campuses and the edges of carrier networks. The SONet (Synchronous Optical Network) rings at the core of carriers’ metropolitan networks are likely to stay in place for some time because of features for reliability and quick recovery from failures, though work is being done to add similar features to Ethernet, according to Passmore.

As for faster Ethernet pipes to servers, I/O technology is only beginning to catch up with 10-Gigabit Ethernet.

“Forty gigabits or 100 gigabits would be completely wasted on a server,” Passmore said.

However, one Ethernet pioneer is looking to overcome this barrier. Judy Estrin, chief executive officer (CEO) of holding company Packet Design LLC, in Palo Alto, in March spun off a new company to develop a scalable I/O architecture. With the new company, Precision I/O, Estrin hopes to break through current scalability limitations to make server I/O scale up to handle 40G bps or more, she said at the Ethernet anniversary event last week.

Estrin, who has founded and sold several successful networking companies since the 1980s, has a special attachment to Ethernet. She was introduced to the technology in its early days at a seminar given by Metcalfe at PARC. She attended the seminar with a group of fellow students after they went out to lunch for her 21st birthday and she enjoyed her first drink. That may have been one reason it made such an impression on her, she said.