Internet Performance in the Middle East

-by Sameer Muhammed

Middle East and surrounding regions are unique in that the service provider networks are virtually a cloud of eyeballs dependant on contents hosted at various locations across the globe. In such a scenario several variables determine the quality of Internet service, which are being delivered to the end users. This phenomenon is a regional predicament due to the absence of any popular content and large scale data centers in close proximity. TCP the underlying protocol of Internet is dependant on several factors for scaling its throughput, of these the most important ones are latency (RTT), Packet loss and window size. Several RFCs explain the TCP mechanisms in great detail like the RFC numbers 813, 879, 896, 1072, 1323 etc (More details of these RFCs are available at http://www.faqs.org/rfcs/). To quote one of the RFCs which highlights the dependency on latency,
There is still a fundamental TCP performance bottleneck for one transmission regime: paths with high bandwidth and long round-trip delays. The significant parameter is the product of bandwidth (bits per second) and round-trip delay (RTT in seconds); this product is the number of bits it takes to "fill the pipe", i.e., the amount of unacknowledged data that TCP must handle in order to keep the pipeline full. TCP performance problems arise when this product is large, e.g., significantly exceeds 10**5 bits.

Latency is the round trip time required for the packet to travel from point A to B and then back to A. Essentially latency depends on the total length of the round trip path and serialization delays on the transmission equipments. The laws of physics will fundamentally govern the latency on a fixed length all things being equal. The theoretical maximum speed of light is 300,000 km/sec, but in the real world the speed of light in fiber optics is 200,000 km/sec. Thus, for data transmitted over fiber at distance of 1000 km, a latency of 5 ms is possible [1000 (km) / 200000 (km/sec) = 5 ms], and for a round trip the latency over this distance would be 10 ms. There are studies that document that the Internet backbone is operating at the theoretical best possible round-trip delay, within a factor of 2, so in the example case of 1000 km distance a latency of 20 ms is likely achievable.

Have a look at the AT&T Global IP Network statistics for their Internet backbone with the latency metrics (http://ipnetwork.bgtmo.ip.att.net/pws/network_delay.html), the latency for San Francisco to Seattle, a distance of 1090 KM is shown as 24ms The point here is that the latency improvements would ultimately be gated by the laws of physics which would be dependent on the physical distance between the two end points. We shall use a popular tool available on the Internet to illustrate the effects of latency on TCP throughput, let us consider an access line with 10Mbps bandwidth and Windows XP PC with a default window size of 17520 -For 50ms latency the maximum TCP download bandwidth is 2.8Mbps (http://www.wand.net.nz/~perry/max_download.php?bits_per_second =10000000&ack_size=40&no_delayed_acks=2&mss=1460&rtt=50&w size=17520&ploss=0) -For 100ms latency the maximum TCP download bandwidth is 1.4Mbps (http://www.wand.net.nz/~perry/max_download.php?bits_per_second =10000000&ack_size=40&no_delayed_acks=2&mss=1460&rtt=100& wsize=17520&ploss=0) -For 150ms latency the maximum TCP download bandwidth is 934Kbps (http://www.wand.net.nz/~perry/max_download.php?bits_per_second =10000000&ack_size=40&no_delayed_acks=2&mss=1460&rtt=150& wsize=17520&ploss=0) - For 200ms latency the maximum TCP download bandwidth is 700Kbps (http://www.wand.net.nz/~perry/max_download.php?bits_per_second =10000000&ack_size=40&no_delayed_acks=2&mss=1460&rtt=200& wsize=17520&ploss=0) The above illustration applies to a single TCP session. One of the obvious workaround is to make use of multiple TCP connections at the same time (Download accelerators like DAP make use of this theory, but again there are limitations from the applications, operating systems and servers to go beyond a certain number of simultaneous connections)

The average latency from Middle East to some of the global Internet hubs are as given below London150ms New York..200ms Los Angles290ms Singapore.145ms Amsterdam...120ms The above indicative values are to reach the PoP and the RTT to the actual content source maybe higher by 15-20%. In essence the Internet users in Middle East who depends on the contents sourced from the major global hubs will not see their TCP throughputs scaling beyond certain values, which maybe gated by the underlying latency. One of the best features of TCP was the capability to recover lost packets or work in an environment prone to information loss. However the mechanisms, which are used to provide this reliability, takes a huge toll on the underlying TCP performance in the event of packet loss. There are several factors, which could lead to packet loss, and the most common factor in this part of the world is due to congestion on the links. A link is a finite capacity and when the demand is more than the capacity of the link there will be congestion. The network devices on either end of congested link will try to queue the packets and fill the pipe based on the different queuing mechanisms. When the packets are queued and dispatched based on certain policies there will be an inherent delay due to processing and prioritization. There is also a finite volume which the queue can hold and when this also fills up the tendency is to drop the packets which result in end to end packet loss. The impact of packet loss on TCP is quite drastic as illustrated below for some of the earlier mentioned cases -For 50ms latency and packet loss of 1% the maximum TCP download bandwidth is 1.9Mbps (http://www.wand.net.nz/~perry/max_download.php?bits_per_second =10000000&ack_size=40&no_delayed_acks=2&mss=1460&rtt=50&w size=17520&ploss=1) -For 50ms latency and packet loss of 2% the maximum TCP download bandwidth is 1.2Mbps (http://www.wand.net.nz/~perry/max_download.php?bits_per_second =10000000&ack_size=40&no_delayed_acks=2&mss=1460&rtt=50&w size=17520&ploss=2)

-For 50ms latency and packet loss of 5% the maximum TCP download bandwidth is 600Kbps (http://www.wand.net.nz/~perry/max_download.php?bits_per_second =10000000&ack_size=40&no_delayed_acks=2&mss=1460&rtt=50&w size=17520&ploss=5) As is clear the effects of packet loss are often exponential and are quite common due to capacity degradation in the events of link failures. The only way to prevent congestion is to maintain enough spare capacity to withstand any failures. The idea of abundant spare capacity is a very costly proposition and may not be commercially viable in Middle East considering the dependency on international links for meeting 95% of the capacity requirements. Window Size or RWIN is also a very important factor, which determines the TCP throughput. This is normally a function of the TCP stack of the PC and its implementations varies between the different operating systems vendors. Microsoft operating systems have traditionally not had good implementations of the window size and hence there have been performance issues due to RWIN, however there have been several workarounds like Dr.TCP, TCP optimizer etc which could improve the throughput. The latest Windows products however have made significant improvements in addressing these issues and RWIN is no longer an issue with Microsoft products. UNIX based operating systems have traditionally had a good implementation of the TCP stack and hence the effects of RWIN on these platforms are not very significant. It is very common in the region to hear that the high-speed customers are not able to realize the 10 or 20Mbps that they have subscribed to. As it has been amply highlighted earlier it is more likely that the latency is the culprit here, which is not allowing a single session of TCP to go beyond a certain value. One should understand that when a service provider sells 10Mbps, it is basically the capacity of the pipe that they are selling, Which implies that a 10Mbps line could have 2 users simultaneously downloading at 5Mbps or 5 users at 2Mbps or any combinations thereof such that the capacity fills up beyond which there will be the impacts of packet loss. There has been quite a lot of research to address some of the fundamental issues with TCP and at least 4 different proposals for alternative paradigms that would work better with large RTTs, but there is no consensus yet on the best approach. Most of the research assumes sending very large files over large bandwidth-delay product

paths, and deals with the consequences of packet losses in this "pipe". RFC 4614 catalogs some of the recommended enhancements like SACK-based loss recovery, congestion control, loss recovery extensions etc. Till these high performance TCP stacks materialize within the operating systems the TCP throughput in Middle East will be at the mercy of end to end to latency. There is also another dimension emerging in the region, which is the localization of popular contents. Some of the popular content providers and content distribution networks have started to deploy caches and are also setting up data centers in close proximity, which will provide a significant boost to the performance of these contents.