Do iSCSI SANs need bulletproof transport?

I talked recently about error recovery levels with Andre Hedrick, president and chief technical officer at PyX Technologies, a startup company that says it has developed software for bulletproof iSCSI (Internet SCSI) connectivity.

To better understand what PyX is saying, it’s important to remember that the iSCSI protocol defines three layers of error recovery, numbered from 0 to 2. Only level 0, called session recovery, is mandatory. Implementation of the other two levels — digest error recovery and connection recovery — is not a must.

According to PyX, most — if not all — the iSCSI products currently on the market take the lazy way out of error handling, implementing only the simpler error recovery level 0 and ignoring the other two, and supposedly more damaging, error scenarios.

“If you are serious about your data, you should implement error recovery level 2,” Hedrick says. He adds that by using the iSCSI initiator and target software developed by PyX, together with redundant connections, a session will stay alive as long as at least one link is active.

Using multiple connections also improves the amount of data that can be moved simultaneously and should facilitate running data at wire speed across upcoming 10GbE networks, Hedrick says.

PyX has beaten the drum of full error recovery for iSCSI since the end of 2002, when the protocol specs were still in the making, but the company has met with tepid reactions from other iSCSI players.

One reason that partly explains why PyX’s position on iSCSI does not have many followers, is that error recovery must be implemented symmetrically on both targets and initiators, whereas most storage vendors focus on one class of devices or the other.

For example, Microsoft has developed and offers for free a popular iSCSI initiator for Windows that has become a reference point for vendors that support those platforms.

Obviously, advanced error recovery on just one side of the iSCSI connection would be useless. So, as long as Microsoft and other vendors stick to whatever the protocol mandates for their initiators, vendors of target storage devices have little incentive to develop more comprehensive error recovery.

However, just about every vendor I questioned dismisses iSCSI error recovery level 2 as unnecessary, at least for the time being, because the protocol is already built on a robust networking transport such as TCP, which already guarantees data delivery and provides error checking.

Also, there is a general agreement that for disk-based systems, the performance penalty for restarting a session in case of connection failure is minimal.

Nevertheless, many agree that iSCSI error recovery level 2 could be useful in some cases. For example, in case of errors, when the target device is a tape, level 2 would allow the operation to continue without having to restart.

According to PyX, moving from error recovery level 0 to level 2 adds a significant degree of complexity (hence cost) to an iSCSI solution, but rewards users with impeccable operations in areas such as remote and wireless connectivity.

I don’t doubt it. But at the moment, other vendors seem to be focused on creating a strong customer base for iSCSI storage — an objective that seems sensible to me.