Global Links


  One of the main reasons Fibre Channel was created was to overcome the distance limitations of storage busses and to reach higher speeds. Fibre Channel was designed to use a credit system that would allow serial links to transverse 10s of kilometers. With the latest generation of products, these links can be extended to 10s of thousands of kilometers – global distances.

Links between two devices can take many shapes and follow different protocols as shown in Figure 4-1. The Fibre Channel Backbone (FC-BB) standard maps Fibre Channel over Internet Protocol (FCIP) and telecommunication networks. While native Fibre Channel links can traverse hundreds of kilometers, telecommunication infrastructure is more common than dark fiber and typically more economical too. FC-BB extends the Fibre Channel link over the telecom network and makes the link look like one or more physical wires. The resulting link creates a single Fabric that is extended over the long distance.

Disaster recovery and business continuance are two of the benefits that these long links provide. By backing up or mirroring data over long distances, data centers can span metropolitan or inter-continental distances. These long distance protocols use the power of fiber optics and efficient protocols to make corporations more resilient to disasters and more distributed for parallel processing.

Fig4-1 WAN Interconnect Devices
Figure 4-1: Fibre Channel Fabrics can be connected via multiple types of networks. IP, ATM, SONET, SDH and GFP networks can all be used to carry Fibre Channel frames over long distances.

Telecom Connections

The first FC-BB links were created over Synchronous Optical Networks (SONET) and Asynchronous Transfer Mode (ATM) networks. FC-BB maps Fibre Channel traffic on to SONET (similar to Synchronous Digital Hierarchy (SDH) networks outside of the United States) and ATM networks that are popular with telecommunications companies. While Fibre Channel runs at gigabit speeds, companies often lease telecommunication lines in the megabit speeds. FC-BB devices are the middlemen between these disparate networks and negotiates both protocols to bridge the gap.

The FC-BB_ATM and FC-BB_SONET devices have B_Ports connect to the Switches as shown in Figure 4-2. The diagram shows how the FC-BB devices act like a virtual ISL or wire so that the Switches form one Fabric. On the ATM side of the network, B_Access portals connect to form a B_Access Virtual ISL and send the Exchange B_Port Parameters (EBP) SW_ILS. Each B_Access Portal has a Fibre Channel - ATM Link End Point (FCATM_LEP) that will usually have multiple ATM Virtual Circuits. These Virtual Circuits can be mapped to different types of traffic like Class F frames or frames destined for certain Destination IDs in a similar fashion as Virtual Channels. The various layers in the FC-BB devices allow optimizations at each layer.

Fig4-2 FC-BB_ATM Virtual ISLs
Figure 4-2: A Virtual ISL connects two E_Ports via B_Ports. The B_Ports convert the Fibre Channel frames into SONET or ATM frames for communication over the telecommunication networks.

FCIP Connections

The Fibre Channel over Internet Protocol (FCIP) was a joint venture between ANSI’s T11 technical committee and the Internet Engineering Task Force (IETF). The IETF portion of the connection is defined in (Request For Comment) RFC 3821 and the encapsulation standard is defined in RFC 3643. The IP portion of FCIP acts as a slave to the FC protocols defined in FC-BB.

Figure 4-3 shows the various layers of the FCIP protocol that reside in an FC-BB_IP device. The FC-BB_IP Virtual ISL resides behind a switch and is across Virtual E_Ports (VE_Ports). The VE_Ports are virtual because they do not have a physical port and several VE_Ports can reside behind a single E_Port. This allows for a much more scalable implementation than FC-BB_ATM and FC-BB_SONET devices which can only connect two E_Ports. One E_Port on the front of an FC-BB_IP device could fan out to multiple other switches over the IP network.

Figure 4-3: The FC-BB_IP device behaves like a Switch and can have multiple Virtual FC-BB Virtual ISLs that connect to other FC-BB_IP devices over TCP/IP networks.

Figure 4-4 shows detail about the internals of an E_Port implementation of the FC_BB_IP device and how a VE_Port connects to another VE_Port through multiple Transmission Control Protocol (TCP) Ports to the IP network. Each VE_Port resides in an FC Entity that interfaces to the FCIP Entity where FCIP Link End Points are located. Each Virtual ISL is created by the link between the VE_Ports. Different traffic flows map to each TCP Port in a similar way that ATM Virtual Circuits carry different traffic flows. Multiple layers in the FC-BB_IP device allow flexible implementations that can scale to meet large applications.

Figure 4-4:The FC-BB_IP Device has a switching element that allows multiple Virtual E_Ports to reside behind it.

FC-BB_IP devices may also have a B_Port implementation of the protocol. Figure 4-5 shows the internals of the B_Port Implementation of the FC-BB_IP device. The B_Port implementation doesn’t scale as easily as the E_Port implementation, but it is easier to build. The B_Port implementation basically creates point-to-point links between two E_Ports in a similar way to FC-BB_ATM devices.

Figure 4-5: FC-BB_IP Devices with B_Port implementations do not consume Domain IDs like E_Port implementations, but have limited scalability.

GFPT Connections

The final type of long distance link defined in FC-BB uses Asynchronous Transparent Generic Framing Procedure (GFPT). GFPT enables the encapsulation of data over any of the telecom networks that have mapped GFPT as shown in Figure 4-6. Networks that already have GFPT mappings include SONET, SDH, Optical Transport Network (OTN) and Plesiochronous Digital Hierarchy (PDH) Networks. The FC-BB_GFPT device could thus connect to the most common telecom networks throughout the world.

Figure 4-6: The layers of FC-BB_GFPT adapt to multiple types of networks that can be found around the world.

The GFPT connection is transparent to, or invisible to the FC Fabric. Similar to how WDM links are not visible to a Fabric, a GFPT link will look exactly like a long fiber. This implementation simplifies the Fibre Channel interactions because no FC_Ports exist and no ELSs or SW_ILSs are sent to the FC-BB GFPT device. Each link forms what is known as a Transport Trail that is dedicated to the two Ports that connect to the FC-BB_GFPT device.


FC-BB has thus defined four basic ways to transmit frames over long distance networks. Transporting over ATM or SONET networks involves B_Ports that are connected to E_Ports. FCIP may interface to the Fabric with either E_Ports or B_Ports while GFPT does not use any type of port. While FCIP is more scalable than the other standards, it is also more complex. Implementations need to balance the needs of their applications versus the complexity of a given solution.

The mapping of Fibre Channel to multiple types of telecommunications networks is growing in popularity. Government regulations and the dispersed nature of today’s companies require better connections over longer distances. Long distance links allow collaboration on unprecedented scales and enables the global data center.

Fibre Channel BackBone Asynchronous Transfer Mode (FC-BB_ATM): The protocol for creating a Virtual ISL between two E_Ports via B_Ports over an ATM network.

Fibre Channel BackBone Synchronous Optical Network (FC-BB_SONET): The protocol for creating a Virtual ISL between two E_Ports via B_Ports over a SONET network.

Fibre Channel over Internet Protocol (FCIP): The protocol that is jointly defined in T11 and IETF to encapsulate Fibre Channel Frames over Internet Protocol networks. E_Port implementations create a Switch that supports multiple Virtual ISLs while B_Port implementations create a single Virtual ISL.

Transparent Generic Framing Procedure (GFPT): A protocol to connect two Ports over a transparent network to form a virtual link. The connected ports may be E_Ports, F_Ports or Nx_Ports but the FC-BB_GFPT device does not create an FC Port.

Source: Extract from Chapter 4 of "Fibre Channel Advances" by Scott Kipp

Back to Article List