Gigabit Ethernet, TCPIP and iSCSI

3-1

Gigabit Ethernet, TCPIP and iSCSI

3-2

 iSCSI is: Block level storage Standards based: Open standard for SCSI over IP Ratified by IETF standards committee

iSCSI performs extremely well: Switched Gb Ethernet outperforms 1Gb Fibre 124 MB/ MB/sec per interface i f 10 Gb-Ethernet iSCSI deployments to be available in the future Security built-in to standard (IPSec, VLANs, CHAP) Multilayer access control (IP address, Initiator, CHAPs) Switched GbE is point to point, full duplex Leverages existing network management tools (SNMP, etc.) Is a consistent protocol with WAN and LAN infrastructures IP standards are mature

iSCSI is a secure, reliable storage

iSCSI exploits l it existing i ti IP k knowledge l d b base:

Reduces interoperability issues:

Gigabit Ethernet, TCPIP and iSCSI

3-3

 What is iSCSI? iSCSI defines how storage traffic is handled across a TCP/IP network in the following suite of RFCs: RFC 3720 iSCSI RFC 3721 Naming and Discovery RFC 3722 String Names RFC 3723 Securing Block IP Protocols MPIO, Boot from SAN, Clustering, HBAs, and SAN backup

P id all Provides ll th the b benefits fit of f a SAN SAN:

iSCSI uses a combination of two familiar networking technologies: Gigabit Ethernet Data Link level protocol to connect between a server and a level 2 switch: TCP/IP Routing protocol Guaranteed delivery system Uses a MAC address to route between the two Uses flow control to control the traffic Uses jumbo frames (9000 byte)

Storage traffic uses the familiar SCSI protocol to perform SCSI reads and writes to storage devices from a server

Gigabit Ethernet, TCPIP and iSCSI

3-4

SCSI reads, writes, and status command messages are used to perform storage reads and writes just as they are used in Direct Attached Storage (DAS) iSCSI messages are transported across a traditional IP-based network using IP datagrams across a Gigabit Ethernet network TCP is used to provide reliable delivery IP is used to perform the routing functions and address devices within the network Gigabit Ethernet is used as the data link level to transport jumbo frames and perform flow control over a gigabit switched network: Jumbo frames 1500 Byte B t MTU Over 80,000 frames per second processed at line rate 1200 CPU cycles per frame required (typical server) Headers consume network and CPU bandwidth (6:1) 9000 Byte MTU 14,000 frames per second processed Average 50% throughput increase; 50% CPU utilization decrease 8k NFS block will fit without fragmentation Fewer frames = fewer issues with out of order delivery Flow Control - 802.3x Flow Control allows the receiver to instruct the sender to throttle back. The receiver does this by sending pause frames to the sender, which causes the sender to slow packet transmission for a short period of time. This prevents TCP from having to retransmit packets if packets are dropped due to congestion within the network. TCP retransmit time is significantly higher than back pressuring time that is used at the 803.3x.

Gigabit Ethernet, TCPIP and iSCSI

3-5

 From the iSCSI RFC: The Small Computer Systems Interface (SCSI) is a popular family of protocols for communicating with I/O devices, especially storage devices. SCSI is a client-server architecture.

Clients of a SCSI interface are called "initiators". Initiators issue SCSI "commands" to request services commands are used to request services from components (logical unit known as a target). An initiator is one endpoint of a SCSI transport and a target is the other endpoint. The SCSI protocol has been mapped over various transports, including Parallel SCSI, IPI, IEEE-1394 (firewire) and Fibre Channel. These transports are I/O specific and have limited distance capabilities. The iSCSI protocol defined in this document describes a means of transporting SCSI packets over TCP/IP, providing for an interoperable solution that can take advantage of existing Internet infrastructure, internet management facilities, and address distance limitations.

Gigabit Ethernet, TCPIP and iSCSI

3-6

 iSCSI initiators issue SCSI commands Commands are Read, Write or Status Commands are used to request services from components (logical unit known as a target)

iSCSI targets service the requests from the initiator iSCSI transmits and receives block storage applications over TCP/IP networks by encapsulating SCSI commands into TCP and transporting them over the network via IP iSCSI deployments: Initial deployments are commonly project-oriented: Deal with lack of storage for email, databases, and new applications Provide storage consolidation Improve the storage management feature set provided by DAS Expanded use for existing applications New servers and applications New IT operations Disk-to-disk backup Snapshot-based backups Disaster protection

After initial deployment, SAN becomes infrastructure:

Gigabit Ethernet, TCPIP and iSCSI

3-7

 iSCSI requires that the iSCSI initiator and target nodes have one of two types of names. Name types: IQN, iSCSI qualified names Example: iqn.2001-05.com.equallogic:6-8a0900-9f46a0201be4ff0d21dd428df-testvoulume1 Format Reverse DNS naming equallogic.com = com.equallogic N i A Naming Authority th it would ld add dd a unique i string t i of f characters h t

EUI (enterprise unique identifier) Example: eui.abcde49123779abcd Format Formed using the IEEE EUI (Extended Unique Identifier) format (16 hex characters) High 24 bits is the company id id, which is IEEE assigned Low 40 bits is the manufacturer assigned value preceded by the date that the name was assigned

Both are intended to be long lived and unique. Both are controlled by a central naming authority such as a department within the corporation. The default iSCSI protocol port number is 3260.

Gigabit Ethernet, TCPIP and iSCSI

3-8

Gigabit Ethernet, TCPIP and iSCSI

3-9

Gigabit Ethernet, TCPIP and iSCSI

3-10

Gigabit Ethernet, TCPIP and iSCSI

3-11

Gigabit Ethernet, TCPIP and iSCSI

3-12

 Logging on to the iSCSI target will make the volume visible to this host. To log on to the target: From the Targets tab, highlight the target you will attach to Press the Log On button When Log On to Target appears, select the Automatically restore this connection when the system reboots checkbox and press OK Select the Persistent Targets tab and confirm that the iSCSI name of the target is listed

Gigabit Ethernet, TCPIP and iSCSI

3-13

 TCP/IP session is first established. What happens next depends on the initiator and target configuration. The configuration determines which phases the initiator and target will transition to throughout the rest of the login process. Possible login phases/modes: Security Negotiation Phase SNP CHAP Challenge Handshake Authentication Protocol SRP Secure Remote Password Kerberos K b version i 5 SPKM-1 or -2 Simple Public Key Mechanism Login Operational Negotiation Phase LONP Full-Featured Phase FFP Typical setup without security implemented: Initiator issues the first login request containing the iSCSI target name with the initiator name. If the target name is correct, login continues. If the target name is invalid, then the connection closes. At this time, the initiator and target will begin to negotiate parameters under which the session will operate. Once the negotiation is complete, the initiator and target will move to FFP. In FFP, SCSI commands and data will be sent from the initiator to the target. If the initial login also had the field set to negotiate security, then at this point the initiator and target would perform the appropriate security check to validate the partners partners.

Gigabit Ethernet, TCPIP and iSCSI

3-14

 When a login occurs: The first login request goes to the PS Series group IP address. Within the response that is returned to the initiator, there is a Status class field indicating that the initiator must login to another IP address. In the case of the PS Series, this IP address will effectively be the least busy port. Initially, the login process appears to be round-robin but as traffic starts to flow, the least busy port or the port with the fewest connections is chosen.

y port p and logs g in to that IP address. Then, the initiator is redirected to the least busy This process of redirection is normal and fully covered in the iSCSI spec. It is also used in other ways with the PS Series: If you have a connection to eth2 and if for some reason the switch port that eth2 is connected to fails, then the initiator will retry the current port briefly, and then attempt to log in to the group IP address again. Once again, it would be redirected to another port. While the user may see a brief blip in performance performance, redirection should not cause errors.

Gigabit Ethernet, TCPIP and iSCSI

3-15

Gigabit Ethernet, TCPIP and iSCSI

3-16

Network Configuration Choices Private network Requires separate switch Can be inappropriate for some deployments Separate Subnet/VLAN Most common Can use switch level access control for SAN Port blocking (iSCSI uses port 3260) Address filtering LAN (merged LAN and SAN) May be required with software initiators on desktops or blade servers WAN Should be used with VPN or other encryption mechanisms Network Switch Options Flow control Ability of receiver to slow down a sender to avoid packet loss Unicast storm control Switch feature to control storms; must be disabled on SAN ports Spanning Tree Switch capability to detect loops in multiple switch configurations Lengthens time for ports to become usable; should be shortened in SANs (or avoided). Separate vs. shared subnets (Layer 2 vs. Layer 3) VLANs allow for separate network traffic while using the same switch infrastructure VLANs are separate subnets; adds requirement for L3 switches Jumbo Frames All Allow l larger packet k t sizes i ( (~9000 9000 b bytes t vs. 1500 b bytes) t ) Can help improve performance, especially with software initiators

Gigabit Ethernet, TCPIP and iSCSI

3-17

 General iSCSI traffic guidelines: iSCSI traffic tends to be "bursty" with large amounts of data coming all at once followed by idle periods. The switch needs enough bandwidth to handle the bursts AND enough buffer space to buffer packets for other, less fortunate members of the network. Those members may not able to handle full speed bursts and must use flow control to mitigate the incoming data. General GE Switch guidelines: Ideally, y, the switching g fabric should be able to handle 2 Gbps p times the number of p ports (because the traffic is full duplex). So, a 24-port switch should have an internal speed of at least 48 Gbps. For buffer space, a figure of at least 512KB per port seems to be a good starting point. The reason for emphasizing per port is that some switches are designed so that multiple ports share the same buffer space (and sometimes the same fabric interconnection). This means that if only one port in this group is passing traffic, then it has plenty of buffer space and/or bandwidth bandwidth. As soon as you start passing traffic on multiple ports, and the space and/or bandwidth needs to be divided among the ports, there is no longer enough available per port to do the job. Jumbo frames guidelines: The maximum frame size that the PS Series can currently handle is 9014 bytes. Some manufacturers of network equipment call this "9000 bytes"; it depends on whether they count the 14-byte Ethernet header as part of the frame or not. We recommend that y you not set the jumbo j frame size on y your switch to anything y g more than 9014 (or 9000) bytes. Trunks, ISLs, and Stack cables Recommend stacking cables if available or if not available then sufficient Trunk, ISLs, to support the load. Rule of thumb is 1 trunk for each active port within the group.

Gigabit Ethernet, TCPIP and iSCSI

3-18

Gigabit Ethernet, TCPIP and iSCSI

3-19