Storage managers have to make tough decisions every day, most with long-reaching consequences. Buying a new block-based storage array is no exception; deciding between iSCSI and Fibre Channel can be exacerbating. A lot of technical and emotional factors come into play and it can be tough to sort through things like which protocol has better reliability, performance, cost, scalability and flexibility. In this first part of our series on Fibre Channel and iSCSI storage area networks (SANs) for SMBs, we'll briefly review Fibre Channel and iSCSI SAN technology, and then we'll compare the reliability and performance of these technologies. In part 2, we'll discuss cost and scalability options.
Fibre Channel and iSCSI SAN history
Fibre Channel technology has been around since the mid-1980s and was a ratified standard in 1994. Fibre Channel is to SCSI what TCP is to IP. It is the transport mechanism used to pass SCSI commands between a host initiator and a storage target. Fibre Channel SAN storage popularity took off in the mid- to late-1990s with the dot.com information explosion. At the time, it was really the only viable way for a data centre full of computers to access centralised storage. Fibre Channel SANs required a dedicated fiber optic storage network with storage-specific network switches and host bus adapter (HBA) cards installed in servers. From their onset, Fibre Channel SANs were considered to be expensive
The cost and complexity of Fibre Channel SANs sent industrious engineers to the drawing board. In 2003, the Internet Engineering Task Force (IETF) ratified iSCSI. This new protocol allowed the ubiquitous SCSI storage commands to be routed over an IP network instead of Fibre Channel. The goal was to lower the cost for centralized storage by consolidating the storage and IP networks. iSCSI commonly connects small or non-production servers to data storage, but don't let the cautious deployment tendencies fool you; iSCSI is in its prime. Major storage players, including EMC, IBM and NetApp offer iSCSI as front-line connectivity option to their arrays.
Fibre Channel SANs are generally more reliable than iSCSI SANs. Fibre Channel networks are relatively small and isolated networks, when compared to their TCP/IP cousins. This isolation means that the security risks, human-induced errors and hardware failures that may plague IP networks can't touch Fibre Channel SAN traffic. Mixing IP and storage traffic could mean that mission-critical storage traffic is routed right alongside an intern's web requests to myspace.com. Perhaps more important to reliability, FCP delivers fixed length frames in the order sent with robust error checking. iSCSI, on the other hand, rides on TCP/IP, which was designed to tolerate unreliable networks. TCP is a byte stream protocol that requires significant overhead to re-assemble out of order data and verify its integrity.
Fibre Channel and iSCSI SAN performance
Fibre Channel also has an advantage over iSCSI in the performance department. Ethernet has long had only 1 GbE as a suitable option for connecting iSCSI. Recently, 10 Gb Ethernet is getting a lot of press. However, it is pretty new and therefore expensive, cancelling out a lot of the benefits that "cheaper iSCSI" provides.
Fibre Channel has several connectivity options which include 1 Gbps, 2 Gbps, 4 Gbps and 8 Gbps. Most reasonably priced departmental SAN switches today offer very dense, full line rate ports with non-blocking bandwidth to any other port on the switch. Fibre Channel also requires HBAs to be installed in the hosts to connect to the SAN. These HBAs offload Fibre Channel SAN network processing from the server CPU.
iSCSI HBAs can be implemented either with software drivers using your standard NIC or iSCSI HBA hardware. The software option taxes the server CPU and the iSCSI HBA options aren't much cheaper than Fibre Channel HBAs. If you don't use TOE cards, the iSCSI interfaces --common NICs -- are much cheaper, which have four times the bandwidth.
A final advantage is that Fibre Channel guarantees in-order delivery of data packets. This means that the unlike iSCSI adapters, Fibre Channel HBAs are not burdened with the sorting and reassembly of frames. Out-of-order delivery is a normal aspect of TCP/IP networks and iSCSI adapters have to pick up the slack. As I/O workloads increase, so does the amount of data that must be temporarily written to memory, sorted and then coped again for further processing. There are some interesting proposals like iSER, which promise to solve this iSCSI problem, but one has to wonder if it's too little too late for iSCSI now that Fibre Channel over Ethernet (FCoE) is building steam.
About the author: Brian Peterson is an independent IT infrastructure analyst. He has a deep background in enterprise storage and open-systems computing platforms. He has consulted with hundreds of enterprise customers who struggled with the challenges of disaster recovery, scalability, technology refreshes and controlling costs.
This was first published in January 2009