Maelstrom: Transparent Error Correction for Lambda Networks

Introduction

Maelstrom is a performance enhancement proxy developed to overcome the poor performance of TCP when loss occurs on high bandwidth / high latency network links. Maelstrom appliances work in tandem, each appliance located at the perimeter of the network and facing a LAN on one side and a high bandwidth / high latency WAN link on the opposite side. The appliances perform forward error correction (FEC) encoding over the egress traffic on one side and decoding over the ingress traffic on the opposite side. In Figure 1, for example, node1 and node2 are running Maelstrom appliances, with node1 encoding over all traffic originating from node10 and destined for, say node20. Node2 receives both the original IP traffic and the additional FEC traffic and forwards the original traffic and potentially any recovered traffic to node20. Note that this is a symmetric pattern, each Maelstrom appliance working both as an ingress and as an egress router at the same time.

Figure 1: Emulab Maelstrom topology example (NS file).

Version 2.0 also includes the dual of Maelstrom: a comprehensive implementation of the high speed protocol-independent (IP-level) redundancy elimination (RE) scheme using rabin fingerprinting first proposed by Neil T. Spring and David Wetherall, A protocol-independent technique for eliminating redundant network traffic, In Proceedings of SIGCOMM, 2000.

Publications

Mahesh Balakrishnan, Tudor Marian, Ken Birman, Hakim Weatherspoon, Lakshmi Ganesh. Maelstrom: Transparent Error Correction for Communication between Data Centers. To appear in IEEE/ACM Transactions on Networking (ToN).
Mahesh Balakrishnan, Tudor Marian, Ken Birman, Hakim Weatherspoon, and Einar Vollset. Maelstrom: Transparent Error Correction for Lambda Networks. Fifth Usenix Symposium on Networked Systems Design & Implementation (NSDI 2008). April 2008, San Francisco, CA.
Hakim Weatherspoon, Lakshmi Ganesh, Tudor Marian, Mahesh Balakrishnan, and Ken Birman. Smoke and Mirrors: Reflecting Files at a Geographically Remote Location Without Loss of Performance. In Proceedings of the 7th USENIX Conference on File and Storage Technologies (FAST 2009). February 2009, San Francisco, CA.

People

Download

Maelstrom Linux kernel (2.6.20, 2.6.24, 2.6.27) module

Version 2.0 (recommended) including IP-level redundancy elimination: svn snapshot.
Version 1.1 (deprecated): svn snapshot.

Instructions

Download and build the CVS snapshot ('make'). Make sure you have the linux headers installed, e.g. in Ubuntu the package is called linux-headers-2.6.xx-xx where xx-xx should match your current kernel (type 'uname -r' to find it out). The kernel module was tested on kernel versions 2.6.20, 2.6.24 and 2.6.27, found for example in stock Ubuntu 7.04, 8.04 and 8.10 respectively.
Load the kernel module on the ingress and egress boxes ('/bin/bash in.sh'). You may want to take a look at the in.sh script in case you want to specify different kernel module load-time parameters.
Start iperf server on node20 ('iperf -s').
Start iperf client on node10 ('iperf -c node20 -t 120 -P 8').