Why a change to an AutoCAD file format is throwing some WAN accelerators for a loop

An innocent change to an AutoCAD file format has cut sharply into the ability of some WAN acceleration solutions to speed the transfer of these files, and caused grumblings in some widely distributed AutoCAD shops over slowed WAN performance. The problem affects users of AutoCAD 2007 and 2008 (specifically those who open and save the design program’s files over a WAN) and stems from a change to the DWG file format for AutoCAD 2007.

Ironically, Autodesk’s changes to the DWG file format were designed to improve performance and reduce file size, among other things. But the new format, when combined with the automatic backup setting called Incremental Save Percentage (ISP), has quietly turned the WAN optimization and acceleration industry on its ear, or nearly so. See the recent bulletin from Autodesk and Riverbed Technology in the Autodesk Knowledge Base.

How does a DWG file negate the benefits of WAN acceleration? Most WAN accelerators reduce data on the WAN through a process called data deduplication. As data passes over the WAN and through a pair of WAN optimization appliances, byte segments are stored and hashes or tokens are created to represent the recently seen data. On subsequent passes, instead of sending every byte of a file back over the WAN, only the tokens are sent that match data already in the local data store. This dramatically reduces the amount of data traveling the WAN while improving response time and performance for the user. Many WAN appliance vendors use this or similar techniques to great advantage.

The DWG problem boils down to this: AutoCAD 2007/2008 files are completely rewritten when a user does a full save on the drawing with AutoCAD’s Incremental Save Percentage value set to 0. Instead of only a few bytes of the file changing with a simple edit, the whole byte structure of the file changes. So to appliances that rely heavily on deduplication and pattern matching, each file save looks like a cold pass, gaining little data or time reduction from the WAN accelerator. It is important to note that this issue only impacts the save operation. Subsequent reads are not affected and will still benefit from data deduplication and overall better performance.

AutoCAD users can mitigate the scrambling effects, to a certain point, by changing the Incremental Save Percentage to 50 or higher (100 is the max). In fact, AutoCAD ships with the setting at 50 out of the box. In years past, AutoCAD users experienced some file corruption with any ISP setting greater than 0, so for many, changing that value to anything else is out of the question. But there will be times, even with an Incremental Save Percentage of 100, that each byte will be changed during a file save operation.

As a result of this file rewriting, a bit of a cat fight has broken out between leading vendors Riverbed Technology and Silver Peak Systems. Riverbed is affected by the AutoCAD file format/ISP setting issue more so than rival Silver Peak, and Silver Peak would like the world to know. To be clear, Riverbed will still accelerate AutoCAD DWG files over the WAN, even on a full save, but the data reduction is less than what Silver Peak is able to accomplish.

The AutoCAD file problem is a very specific use case and not an indicator of any larger technology issue with Riverbed. Silver Peak’s data deduplication technology is able to handle this particular problem much more gracefully than Riverbed’s.

I recently participated in an online demonstration with Silver Peak where I saw the effects of the rewritten AutoCAD files when saved over the WAN, and how to a large extent, Silver Peak’s appliances still provided a good measure of performance increase. The following table summarizes the test results. The lower the ISP setting, the more the effects of the rewrite are felt. The tests were conducted over a 10Mbps network with 100ms latency. Times are in seconds. The DWG file was 14MB in size.

Source: Silver Peak Systems

I was not able to run the same tests with a pair of Steelheads from Riverbed, but they claim to provide at least a 20% improvement in performance even on a “cold” file save.

One thing to remember is that other factors in addition to data deduplication are at work to increase WAN performance. Mitigating the effects of latency, reducing application chattiness, and overall TCP optimizations all play a part in speeding up file transfers over the WAN. Riverbed executives stated they are working closely with Autodesk to minimize or even eliminate this problem, but no time table on a fix was available.