Why is a connectionless protocol best for Ethernet SAN?

SAN traffic is simple commands to either write or read data, to persistent storage (usually disk drives) connected to a network—and this flow of data needs to be fast.

When speed is important, it matters how you transport data over the network.  Wasted steps and added complexity slow data flow.

It’s clear, Ethernet has won the technology race as the preferred transport mechanism for Storage Area Networks (SAN).  Ethernet is simpler, faster, and much less expensive than Fibre Channel.

But Ethernet defines an electrical method and basic frame/packet structure to carry data, it doesn’t define how data is to be transported.  Ethernet was designed to be a lossy network, primarily because early hardware implementations experienced network collisions, and some packets were lost in transmission.  So, applications that use Ethernet need to cope with occasional dropped packets.

Wrapping each packet in TCP guarantees lost packets are resent and delivered without corruption, but TCP also adds other features that are not always needed, especially on an Ethernet SAN.

For example, TCP establishes a “connection path” between the two ends of the network conversation.  This connect results in unintended consequences when TCP is used in a SAN.  To mitigate these consequences adds complexity.  I’ll explain more later.

ATA-over-Ethernet (AoE):   SAN without Complexity

AoE is an IEEE registered Ethernet frame type (0x88A2), and the AoE protocol includes important features that enable Ethernet to transport data efficiently and reliably.  AoE accomplishes some of the things provided by TCP, but without all the unnecessary stuff for a SAN application.

First each AoE frame (packet) is given a unique tag number.  All AoE packets, initiated by the host (initiator) are acknowledged by the EtherDrive storage device (target) echoing the packets tag number.  If a packet has not been acknowledged the host will resend it with a new reference tag number.  Standard Ethernet frame CRC will detect corrupt packets and discard them.

That’s it!  We don’t need to reorder the sequence of received data since the data is already identified by its block address, and we don’t need to route the packets with IP addresses when they stay within the local network.

Okay, so what happens when you use AoE as the storage protocol?

As I mentioned earlier, everyone wants the data flow between the storage device and the host (server) to be as fast as possible.  GigE and 10GigE links are used in an Ethernet SAN fabric.  More ports at the initiator and more ports at the target, provide more bandwidth for more data flow.  And multiple paths through the network also provide redundancy in case a link is broken.

In an iSCSI SAN, the TCP part of the protocol sets up a connection from one port on the initiator to one port on the target.  If more than one port is available on the initiator or target, the TCP connection won’t use it without additional software to manage these multi-path connections.  And as the number of available ports connected to the SAN is increased, the number of connection possibilities grows exponentially.  Each possible path presents a new instance of the same target LUN, making management much more complicated.

Connectionless Simplicity

AoE is a connectionless protocol and has the advantage of being able to flood the network with as many packets as the ports and paths between initiator and target can handle.

The AoE initiator will round robin packets between all available ports, load balancing the flow, for maximum throughput speed.  The LUNs are presented as a single instance to the host, no mater how many paths are being used for the data flow.  If the fabric of the SAN is changed, or new paths added/removed, AoE just naturally works.  Links can fail and be restored and the AoE flow automatically adapts to the available bandwidth.

That’s why AoE is better than iSCSI, it’s connectionless.  It's simpler to understand, and easier to use.  It just works.....and that's what makes EtherDrive storage so fast.