Application Layer Fall24
Application Layer Fall24
2: Application Layer 1
Some network apps
E-mail Internet telephone
Web Real-time video
Instant messaging conference
Remote login Massive parallel
P2P file sharing computing
Multi-user network
games
Streaming stored
video clips
2: Application Layer 2
Creating a network app
Write programs that application
transport
systems and
physical
o communicate over a
network.
o e.g., Web: Web server
software communicates
with browser software
No software written for application
application
2: Application Layer 4
Client-server archicture
server:
o always-on host
o permanent IP address
o server farms for scaling
clients:
o communicate with
server
o may be intermittently
connected
o may have dynamic IP
addresses
o do not communicate
directly with each other
2: Application Layer 5
Pure P2P architecture
no always-on server
arbitrary end systems
directly communicate
peers request service from
other peers, provide service in
return to other peers
o self scalability – new peers
bring new service capacity,
as well as new service
demands
peers are intermittently
connected and change IP
addresses
o complex management
2: Application Layer 6
Hybrid of client-server and P2P
Skype
o voice-over-IP P2P application
o centralized server: finding address of remote
party:
o client-client connection: direct (not through
server)
Instant messaging
o Chatting between two users is P2P
o Presence detection/location centralized:
• User registers its IP address with central server
when it comes online
• User contacts central server to find IP addresses of
friends
2: Application Layer 7
Network applications: some jargon
actions taken
o use communication services
provided by lower layer
protocols (TCP, UDP)
2: Application Layer 9
App-layer protocol defines
Types of messages Public-domain protocols:
exchanged, eg, request defined in RFCs
& response messages
allows for
Syntax of message
interoperability
types: what fields in
messages & how fields eg, HTTP, SMTP
are delineated Proprietary protocols:
Semantics of the eg, KaZaA, Skype
fields, ie, meaning of
information in fields
Rules for when and
how processes send &
respond to messages
2: Application Layer 10
Processes communicating across network
host or host or
process sends/receives server server
messages to/from its
socket controlled by
app developer
socket analogous to door process process
2: Application Layer 11
Addressing processes:
For a process to Identifier includes
receive messages, it both the IP address
must have an identifier and port numbers
Every host has a unique associated with the
32-bit IP address process on the host.
Q: does the IP address Example port numbers:
of the host on which o HTTP server: 80
the process runs o Mail server: 25
suffice for identifying
the process?
Answer: No, many
processes can be
running on same host
2: Application Layer 12
What transport service does an app need?
Data loss Throughput
some apps (e.g., audio) can some apps (e.g.,
tolerate some loss multimedia) require
other apps (e.g., file minimum amount of
transfer, telnet) require bandwidth to be
100% reliable data
“effective”
transfer
other apps (“elastic
Delay apps”) make use of
some apps (e.g.,
whatever bandwidth
Internet telephony, they get
interactive games)
require low delay to be
“effective”
2: Application Layer 13
Transport service requirements of common apps
2: Application Layer 14
Internet transport protocols services
Application Underlying
Application layer protocol transport protocol
2: Application Layer 16
Web and HTTP
First some jargon
Web page consists of objects
Example URL:
www.cs.bilkent.edu.tr/bilkent/academic/main_logo.gif
2: Application Layer 17
HTTP overview
HTTP: hypertext
transfer protocol
Web’s application layer PC running
protocol Explorer
client/server model
o client: browser that
requests, receives, Server
“displays” Web objects running
Apache Web
o server: Web server
server
sends objects in
response to requests
Mac running
HTTP 1.0: RFC 1945 Navigator
HTTP 1.1: RFC 2068
2: Application Layer 18
HTTP overview (continued)
Uses TCP: HTTP is “stateless”
client initiates TCP server maintains no
connection (creates socket) information about
to server, port 80 past client requests
server accepts TCP Protocols that maintain
connection from client “state” are complex!
HTTP messages (application- past history (state) must
layer protocol messages) be maintained
exchanged between browser if server/client crashes,
(HTTP client) and Web their views of “state” may
server (HTTP server) be inconsistent, must be
TCP connection closed reconciled
2: Application Layer 19
HTTP connections
Nonpersistent HTTP Persistent HTTP
At most one object is Multiple objects can
sent over a TCP be sent over single
connection. TCP connection
HTTP/1.0 uses between client and
nonpersistent HTTP server.
HTTP/1.1 uses
persistent connections
in default mode
2: Application Layer 20
Nonpersistent HTTP
(contains text,
Suppose user enters URL references to 10
www.bilkent.edu.tr/someDepartment/ jpeg images)
time
2: Application Layer 21
Nonpersistent HTTP (cont.)
2: Application Layer 22
Response time modeling
Definition of RRT: time to
send a small packet to
travel from client to
server and back. initiate TCP
connection
Response time: RTT
one RTT to initiate TCP request
connection file
time to
RTT
one RTT for HTTP
transmit
file
request and first few file
received
bytes of HTTP response
to return time time
file transmission time
2: Application Layer 24
HTTP request message
request line
(GET, POST, GET /somedir/page.html HTTP/1.1
HEAD commands) Host: www.bilkent.edu.tr
User-agent: Mozilla/4.0
header Connection: close
lines Accept-language:fr
Carriage return,
line feed (extra carriage return, line feed)
indicates end
of message
2: Application Layer 25
HTTP request message: general format
2: Application Layer 26
Method types
HTTP/1.0 HTTP/1.1
GET GET, POST, HEAD
POST PUT
2: Application Layer 27
Uploading form input
Post method:
Web page often
includes form input
Input is uploaded to
server in entity body
URL method:
Uses GET method
Input is uploaded in
URL field of request
line:
www.somesite.com/animalsearch?monkeys&banana
2: Application Layer 28
HTTP response message
status line
(protocol
status code HTTP/1.1 200 OK
status phrase) Connection close
Date: Thu, 06 Aug 1998 12:00:15 GMT
Server: Apache/1.3.0 (Unix)
header
Last-Modified: Mon, 22 Jun 1998 …...
lines
Content-Length: 6821
Content-Type: text/html
2: Application Layer 29
HTTP response status codes
In first line in server->client response message.
A few sample codes:
200 OK
o request succeeded, requested object later in this message
301 Moved Permanently
o requested object moved, new location specified later in
this message (Location:)
400 Bad Request
o request message not understood by server
404 Not Found
o requested document not found on this server
505 HTTP Version Not Supported
2: Application Layer 30
Trying out HTTP (client side) for yourself
2: Application Layer 31
User-server interaction: authorization
Authorization : control access to server
server content
client
authorization credentials: usual http request msg
typically name, password 401: authorization req.
stateless: client must present WWW authenticate:
authorization in each request
o authorization: header line in usual http request msg
each request + Authorization: <cred>
o if no authorization: header, usual http response msg
server refuses access,
sends
WWW authenticate: usual http request msg
+ Authorization: <cred>
header line in response
usual http response msg time
2: Application Layer 32
Cookies: keeping “state”
Many major Web sites Example:
use cookies o Susan access Internet
Four components: always from same PC
o She visits a specific e-
1) cookie header line in
commerce site for first
the HTTP response
time
message
o When initial HTTP
2) cookie header line in
requests arrives at site,
HTTP request message
site creates a unique ID
3) cookie file kept on and creates an entry in
user’s host and managed backend database for
by user’s browser ID
4) back-end database at
Web site
2: Application Layer 33
Cookies: keeping “state” (cont.)
client server
Cookie file usual http request msg server
usual http response + creates ID
ebay: 8734 Set-cookie: 1678 1678 for user
Cookie file
usual http request msg
amazon: 1678 cookie: 1678 cookie-
ebay: 8734 specific
usual http response msg action
one week later:
usual http request msg
Cookie file cookie-
cookie: 1678
amazon: 1678 spectific
ebay: 8734 usual http response msg action
2: Application Layer 34
Cookies (continued)
What cookies can bring: Cookies and privacy:
authorization cookies permit sites to
shopping carts
learn a lot about you
you may supply name
recommendations
and e-mail to sites
user session state
search engines use
(Web e-mail)
redirection & cookies
how to keep “state”: to learn yet more
protocol endpoints:
advertising companies
maintain state at
obtain info across
sender/receiver over
multiple transactions sites
cookies: http messages
carry state 2: Application Layer 35
Set-Cookie HTTP Response
Header
Set-Cookie: NAME=VALUE; expires=DATE; path=PATH;
domain=DOMAIN_NAME; secure
o NAME=VALUE
• sequence of characters excluding semi-colon, comma and
white space (the only required field)
o expires=DATE
Format: Wdy, DD-Mon-YYYY HH:MM:SS GMT
o domain=DOMAIN_NAME
• Browser performs “tail matching” searching through cookies
file
• Default domain is the host name of the server which
generated the cookie response
o path=PATH
• the subset of URLs in a domain for which the cookie is valid
o Secure: if secure cookie will only be transmitted if the
communications channel with the host is secure, e.g.,
HTTPS
2: Application Layer 36
Cookies File
Netscape keeps all cookies in a single file
~username/.netscape/cookies whereas IE keeps each cookie in
separate files in the folder C:\Documents and Settings\user\Cookies
# Netscape HTTP Cookie File
# http://www.netscape.com/newsref/std/cookie_spec.html
# This is a generated file! Do not edit.
Sun, 23 Sep 2035 06:35:35 UTC Thu, 8 Jul 2021 06:36:16 UTC
2: Application Layer 38
Conditional GET: client-side caching
2: Application Layer 42
Electronic Mail outgoing
message queue
user mailbox
user
Three major components: agent
user agents mail
user
server
mail servers agent
simple mail transfer SMTP mail
protocol: SMTP server user
SMTP agent
User Agent
a.k.a. “mail reader” SMTP
mail user
composing, editing, reading agent
server
mail messages
e.g., Eudora, Outlook, elm, user
Netscape Messenger agent
user
outgoing, incoming messages agent
stored on server
2: Application Layer 43
Electronic Mail: mail servers
user
Mail Servers agent
mailbox contains incoming mail
user
messages for user server
agent
message queue of outgoing
SMTP
(to be sent) mail messages mail
server user
SMTP protocol between mail
servers to send email SMTP agent
messages SMTP
o client: sending mail mail user
agent
server server
o “server”: receiving mail
user
server agent
user
agent
2: Application Layer 44
Electronic Mail: SMTP [RFC 2821]
2: Application Layer 45
Scenario: Ayşe sends message to Ali
1) Ayşe uses UA to compose 4) SMTP client sends Ayşe’s
message and “to” message over the TCP
[email protected] connection
2) Ayşe’s UA sends message 5) Ali’s mail server places the
to her mail server; message message in Ali’s mailbox
placed in message queue 6) Ali invokes his user agent
3) Client side of SMTP opens to read message
TCP connection with Ali’s
mail server
Ayşe Ali
1 mail
mail
server user
user server
2 agent
agent 3 6
4 5
2: Application Layer 46
SMTP interaction for yourself
telnet cs.bilkent.edu.tr 25
220 gordion.cs.bilkent.edu.tr ESMTP Sendmail 8.12.9/8.12.9;
Wed, 3 Mar 2004 11:17:52 +0200 (EET)
HELO cs.bilkent.edu.tr
250 gordion.cs.bilkent.edu.tr Hello nemrut.ee.bilkent.edu.
tr [139.179.12.28], pleased to meet you
MAIL FROM: <[email protected]>
250 2.1.0 <[email protected]>... Sender ok
RCPT TO: <[email protected]>
250 2.1.5 <[email protected]>... Recipient ok
DATA
354 Enter mail, end with "." on a line by itself
hello
.
250 2.0.0 Message accepted for delivery
QUIT
221 2.0.0 gordion.cs.bilkent.edu.tr closing connection
2: Application Layer 47
SMTP: final words
SMTP uses persistent Comparison with HTTP:
connections
HTTP: pull
SMTP requires message
(header & body) to be in 7- SMTP: push
bit ASCII both have ASCII
SMTP server uses command/response
CRLF.CRLF to determine interaction, status codes
end of message
HTTP: each object
encapsulated in its own
response msg
SMTP: multiple objects
sent in multipart msg
2: Application Layer 48
Mail access protocols
SMTP SMTP access user
user
agent protocol agent
2: Application Layer 49
DNS: Domain Name System
2: Application Layer 50
DNS
DNS services Why not centralize DNS?
Hostname to IP single point of failure
2: Application Layer 51
Distributed, Hierarchical Database
Root DNS Servers
a Verisign, Dulles, VA
c Cogent, Herndon, VA (also Los Angeles)
d U Maryland College Park, MD
k RIPE London (also Amsterdam,
g US DoD Vienna, VA
Frankfurt) Stockholm (plus 3
i Autonomica,
h ARL Aberdeen, MD
j Verisign, ( 11 locations) other locations)
m WIDE Tokyo
e NASA Mt View, CA
f Internet Software C. Palo Alto,
CA (and 17 other locations)
13 root name
servers worldwide
b USC-ISI Marina del Rey, CA
l ICANN Los Angeles, CA
2: Application Layer 53
TLD and Authoritative Servers
Top-level domain (TLD) servers: responsible
for com, org, net, edu, etc, and all top-level
country domains uk, fr, ca, jp.
o Network solutions maintains servers for com TLD
o Educause for edu TLD
Authoritative DNS servers: organization’s
DNS servers, providing authoritative
hostname to IP mappings for organization’s
servers (e.g., Web and mail).
o Can be maintained by organization or service
provider
2: Application Layer 54
Local Name Server
Does not strictly belong to hierarchy
Each ISP (residential ISP, company,
university) has one.
o Also called “default name server”
When a host makes a DNS query, query is
sent to its local DNS server
o Acts as a proxy, forwards query into hierarchy.
2: Application Layer 55
Example root DNS server
2
Host at 3
TLD DNS server
firat.bilkent.edu.tr 4
wants IP address for
5
gaia.cs.umass.edu
local DNS server
dns.bilkent.edu.tr
7 6
1 8
gaia.cs.umass.edu
2: Application Layer 56
Recursive queries root DNS server
recursive query:
2 3
puts burden of name
resolution on 7 6
contacted name TLD DNS serve
server
heavy load?
local DNS server
4
iterated query: dns.bilkent.edu.tr 5
contacted server 1 8
replies with name of
server to contact authoritative DNS server
“I don’t know this dns.cs.umass.edu
requesting host
name, but ask this Firat.bilkent.edu.tr
server”
gaia.cs.umass.edu
2: Application Layer 57
DNS: caching and updating records
once (any) name server learns mapping, it caches
mapping
o cache entries timeout (disappear) after some
time
o TLD servers typically cached in local name
servers
• Thus root name servers not often visited
update/notify mechanisms under design by IETF
o RFC 2136
o http://www.ietf.org/html.charters/dnsind-charter.html
2: Application Layer 58
DNS records
DNS: distributed db storing resource records (RR)
RR format: (name, value, type, ttl)
Type=A Type=CNAME
o name is hostname o name is alias name for some
o value is IP address “cannonical” (the real) name
Type=NS www.ibm.com is really
o name is domain (e.g. servereast.backup2.ibm.com
foo.com) o value is cannonical name
o value is IP address of
authoritative name server Type=MX
for this domain
o value is name of mailserver
associated with name
2: Application Layer 59
DNS protocol, messages
DNS protocol : query and reply messages, both with same
message format
msg header
identification: 16 bit #
for query, reply to query
uses same #
flags:
o query or reply
o recursion desired
o recursion available
o reply is authoritative
2: Application Layer 60
DNS protocol, messages
RRs in reponse
to query
records for
authoritative servers
additional “helpful”
info that may be used
2: Application Layer 61
Inserting records into DNS
Example: just created startup “Network Utopia”
Register name networkuptopia.com at a registrar
(e.g., Network Solutions)
o Need to provide registrar with names and IP addresses of
your authoritative name server (primary and secondary)
o Registrar inserts two RRs into the com TLD server:
2: Application Layer 62
How do people connect to Web server?
com TLD DNS
server
contains type A
and NS RRs for 3: reply contains IP
Network Utopia address for auth.
name server for
2
Network Utopia
(212.212.212.1) authoritative name
local DNS server 4 server for Network
dns.bilkent.edu.tr Utopia
5: reply contains IP IP: 212.212.212.1
address for Web
1 6
server for
Network Utopia
(212.212.212.178) Web server for
Network Utopia
requesting host 7: TCP connection IP: 212.212.212.178
firat.bilkent.edu.tr
2: Application Layer 63
Socket programming
Goal: learn how to build client/server application that
communicate using sockets
2: Application Layer 64
Socket-programming using TCP
Socket: a door between application process and end-
end-transport protocol (UCP or TCP)
TCP service: reliable transfer of bytes from one
process to another
controlled by
controlled by process application
application process
developer
developer socket socket
TCP with TCP with controlled by
controlled by
buffers, operating
operating buffers, internet system
system variables variables
host or host or
server server
2: Application Layer 65
Socket programming with TCP
Client must contact server When contacted by client,
server process must first server TCP creates new
be running socket for server process to
server must have created communicate with client
socket (door) that o allows server to talk with
welcomes client’s contact multiple clients
o source port numbers
Client contacts server by:
used to distinguish
creating client-local TCP
clients (more in Chap 3)
socket
specifying IP address, port
application viewpoint
number of server process
TCP provides reliable, in-order
When client creates
transfer of bytes (“pipe”)
socket: client TCP between client and server
establishes connection to
server TCP
2: Application Layer 66
Stream jargon
A stream is a sequence of
characters that flow into
or out of a process.
An input stream is
attached to some input
source for the process, eg,
keyboard or socket.
An output stream is
attached to an output
source, eg, monitor or
socket.
2: Application Layer 67
Socket programming with TCP
keyboard monitor
Example client-server app:
1) client reads line from
standard input (inFromUser
inFromUser
input
stream
stream) , sends to server via Client
socket (outToServer Process
process
stream)
2) server reads line from socket
3) server converts line to
uppercase, sends back to
inFromServer
outToServer
output input
client stream stream
2: Application Layer 68
Client/server socket interaction: TCP
Server (running on hostid) Client
create socket,
port=x, for
incoming request:
welcomeSocket =
ServerSocket()
write reply to
connectionSocket read reply from
clientSocket
close
connectionSocket close
clientSocket
2: Application Layer 69
Example: Java client (TCP)
import java.io.*;
import java.net.*;
class TCPClient {
sentence = inFromUser.readLine();
Send line
to server outToServer.writeBytes(sentence + '\n');
clientSocket.close();
}
}
2: Application Layer 71
Example: Java server (TCP)
import java.io.*;
import java.net.*;
class TCPServer {
2: Application Layer 72
Example: Java server (TCP), cont
Create output
stream, attached DataOutputStream outToClient =
to socket new DataOutputStream(connectionSocket.getOutputStream());
Read in line
from socket clientSentence = inFromClient.readLine();
2: Application Layer 73
Socket programming with UDP
2: Application Layer 74
Client/server socket interaction: UDP
Server (running on hostid) Client
write reply to
serverSocket
specifying client read reply from
host address, clientSocket
port number close
clientSocket
2: Application Layer 75
Example: Java client (UDP)
keyboard monitor
inFromUser
input
stream
Client
Process
Input: receives
process
packet (TCP
Output: sends received “byte
packet (TCP sent stream”)
receivePacket
sendPacket
“byte stream”) UDP UDP
packet packet
client UDP
clientSocket
socket UDP
socket
2: Application Layer 76
Example: Java client (UDP)
import java.io.*;
import java.net.*;
class UDPClient {
public static void main(String args[]) throws Exception
{
Create
input stream BufferedReader inFromUser =
new BufferedReader(new InputStreamReader(System.in));
Create
client socket DatagramSocket clientSocket = new DatagramSocket();
Translate
InetAddress IPAddress = InetAddress.getByName("hostname");
hostname to IP
address using DNS byte[] sendData = new byte[1024];
byte[] receiveData = new byte[1024];
2: Application Layer 78
Example: Java server (UDP)
import java.io.*;
import java.net.*;
class UDPServer {
public static void main(String args[]) throws Exception
Create {
datagram socket
DatagramSocket serverSocket = new DatagramSocket(9876);
at port 9876
byte[] receiveData = new byte[1024];
byte[] sendData = new byte[1024];
while(true)
{
Create space for
DatagramPacket receivePacket =
received datagram
new DatagramPacket(receiveData, receiveData.length);
Receive serverSocket.receive(receivePacket);
datagram
2: Application Layer 79
Example: Java server (UDP), cont
String sentence = new String(receivePacket.getData());
Get IP addr
InetAddress IPAddress = receivePacket.getAddress();
port #, of
sender int port = receivePacket.getPort();
sendData = capitalizedSentence.getBytes();
Create datagram
DatagramPacket sendPacket =
to send to client new DatagramPacket(sendData, sendData.length, IPAddress,
port);
Write out
datagram serverSocket.send(sendPacket);
to socket }
}
} End of while loop,
loop back and wait for
another datagram
2: Application Layer 80
Socket programming: references
Java-tutorials:
“All About Sockets” (Sun tutorial),
http://www.javaworld.com/javaworld/jw-12-
1996/jw-12-sockets.html
“Socket Programming in Java: a tutorial,”
http://www.javaworld.com/javaworld/jw-12-
1996/jw-12-sockets.html
2: Application Layer 81
Web caches (proxy server) Dere
Goal: satisfy client request without involving origin server
user sets browser: Web
accesses via cache
browser sends all HTTP
requests to cache
o object in cache: cache
returns object
client
esmer! Proxy
server
origin
server
Yor
Fore
o else cache requests
object from origin
server, then returns
object to client
Advantages -s!gn!f!cant client
smaller delay operatoresraft or!g!nal
server -
cl!ck
counts origin
cl!ent -
or!g!nalserver
yakınproxyodateredes !sa cl!ent
>
d!sad/money
-
extra l!ne
daha of
server neaders
header
entrol
sag
n + +p
II not
-cashable"
the
under 2: Application Layer 82
not
server
More about Web caching
Cache acts as both client Why Web caching?
and server Reduce response time for
Cache can do up-to-date client request.
check using If-modified-
Reduce traffic on an
since HTTP header
institution’s access link.
o Issue: should cache take
risk and deliver cached
Internet dense with caches
object without checking? enables “poor” content
o Heuristics are used.
providers to effectively
deliver content
Typically cache is installed
by ISP (university,
company, residential ISP)
2: Application Layer 83
Caching example (1)
origin
Assumptions
servers
average object size = 100,000
bits public
Internet
avg. request rate from
institution’s browser to origin
serves = 15/sec
1.5 Mbps
delay from institutional router
access link
to any origin server and back
institutional
to router = 2 sec
network
10 Mbps LAN
Consequences
utilization on LAN = 15%
utilization on access link = 100%
institutional
cache
2: Application Layer 85
Caching example (3)
origin
servers
Install cache
suppose hit rate is .4 public
Internet
Consequence
40% requests will be
satisfied almost immediately
1.5 Mbps
60% requests satisfied by access link
origin server institutional
utilization of access link network
10 Mbps LAN
reduced to 60%, resulting in
negligible delays (say 10
msec)
institutional
cache
2: Application Layer 86
reduce the
delay >use closer serverarayaround ana
-
problem
Content distribution networks (CDNs)
shar!ng
server
Customer ( origin server =
connectedin North America
The content providers are
the CDN customers. to te
Late
econom!cally
↓
to P more
Content replication when ect advantages
dec!des
o
!t
CDN distribution node
hundreds of CDN servers
throughout Internet
o in lower-tier ISPs, close
to users
CDN replicates its customers’
content in CDN servers.
When provider updates CDN server
CDN server
content, CDN updates in S. America CDN server
in Asia
servers in Europe e
bu
Fa
m!rroredthe
f!ls 2: Application Layer 87
how does th!s work !
CDN example
HTTP request for
ects
www.foo.com/sports/sports.html
> dec!des
wh!chob !ntent
1 Origin server served
tr!but!on all
-
be
w!ll d!s o
In oseet
&BRAINcl!ent E
2 DNS query for www.cdn.com
the whe wants
CDNs authoritative what
!t
3 DNS server +
-
>
algor!th - HP)
HTTP request for >send response
www.cdn.com/www.foo.com/sports/ruth.gif
IP'ys close to
!nd
http://www.foo.com/sports.ruth.gif DNS server to route
with redirect requests
http://www.cdn.com/www.foo.com/sports/ruth.gif
!p address 2: Application Layer 88
!ts
Streaming stored video
simple scenario:
Internet
real!ty
!n
Application Layer 2-
89
↓ ADAPTIVE E download problem ->
lower/charge
request !n
Streaming multimedia: DASH
↑
h+ + p for
lower Bu
URL
se!ner
DASH: Dynamic, Adaptive Streaming over HTTP trate
h!g
ceh!gh encoded
!n
server: BU +
URL
Demet
o divides video file into multiple chunks
o each chunk stored, encoded at different rates
Cordoad
·
Application Layer 2-
90
Streaming multimedia: DASH
from closest cl!ent
problem : serve
Application Layer 2-
91
Content distribution networks
challenge: how to stream content (selected
from millions of videos) to hundreds of
thousands of simultaneous users?
-
Tay
Nagosoption 1: single, large “mega-server”
-
o single point of failure
o point of network congestion
o long path to distant clients
multiple copies of video sent over outgoing link
-
o
ustros
videos at multiple geographically
distributed sites (CDN)
enter deep: push CDN servers deep into many
o
access networks 100 e
• close to users
• used by Akamai, 1700 locations
o bring home: smaller number (10’s) of larger
clusters in POPs near (but not within) access
networks
• used by Limelight Application Layer 2-
93
Content Distribution Networks (CDNs)
CDN: stores copies of content at CDN nodes
• e.g. Netflix stores copies of MadMen
subscriber requests content from CDN
• directed to nearby copy, retrieves content
• may choose different copy if network path
congested
where’s Madmen?
Application Layer 2-
94
Content Distribution Networks (CDNs)
1. Bob manages
Netflix account CDN
server
4. DASH
streaming
Application Layer 2-
96
Pure P2P architecture
morspectno
examples:
o file distribution
(BitTorrent)
o Streaming (KanKan)
o
O
VoIP (Skype)
2: Application Layer 97
File Distribution: Server-Client vs P2P
Question : How much time to distribute file
from one server to N peers?
us: server upload
bandwidth
Server
ui: peer i upload
u1 d1 u2 bandwidth
us d2
di: peer i download
File, size F bandwidth
dN
Network (with
uN abundant bandwidth)
2: Application Layer 98
File distribution time: server-client
Server
server sequentially F u1 d1 u2
sends N copies: us d2
Time to distribute F
to N clients using = dcs = max { NF/us, F/min(di) }
i
client/server approach
increases linearly in N
(for large N) 2: Application Layer 99
File distribution time: P2P
Server
server must send one F u1 d1 u2
copy: F/us time us d2
3.5
P2P
Minimum Distribution Time
3
Client-Server
2.5
1.5
0.5
0
0 5 10 15 20 25 30 35
N
2: Application Layer 101
Searching for Information-
Query flooding:OGnutella atena!ssat!on
fully distributed overlay network: graph
o no central server edge between peer X
public domain protocol and Y if there’s a TCP
many Gnutella clients connection
implementing protocol all active peers and
Tog!cal
es edges is overlay net
network
yeralwaysan Edge is not a physical
h
I
sub link
Given peer will
typically be connected
with < 10 overlay
neighbors
Repeers has
f!le a
the
"who
!mal D
threshold 2: Application Layer 102
degree should reach the
peers
parallel
not all
mad!gS!
>
no
-
we
=> creates the
all rece!ve d
peers
:
keep
the f!ll
l!st 4 no to
Scalability: the
very
resources 9
limited scope
flooding · not sers the
2: Application Layer 103
Gnutella: Peer joining
1. Joining peer X must find some other peer in
Gnutella network: use list of candidate peers
2. X sequentially attempts to make TCP with peers
on list until connection setup with Y
3. X sends Ping message to Y; Y forwards Ping
message.
4. All peers receiving Ping message respond with
Pong message
5. X receives many Pong messages. It can then
setup additional TCP connections
-
by
track
Thede
content in all its
children.
appl!cat!on
on
tue
ge 100
ordinary peer
group-leader peer
eet!ngTegece en
·
bl ,
stay long
data
!n
peer
connected
comm , good bu
d!ces
a
peer jo!ns the the
based
ass!gned roll on
leader neighoring relationships => ass!gne a
group
TCP connect!on !s roord!nary in overlay network these parameters
peers leave - choose new
vo-
as
group
2: Application Layer 105 leader
prev!ously
KaZaA: Querying
Each file has a hash and a descriptor
Client sends keyword query to its group
leader
Group leader responds with matches:
o For each match: metadata, hash, IP address
If group leader forwards query to other
group leaders, they respond with matches
Client then selects files for downloading
o HTTP requests using hash as identifier sent to
peers holding desired file
(250, 499]
Bytes Rangez
:
1000
Incentive priorities
access the data
!n order >
-
another
upload f!le (a peerl should w!ll!ng to
shar!ng
-
-
the f!le
download much share
-
how
you
!nsent!ve for share
shared
grade
>
- how much
you shoret
as
of
request - put !n front quere
no range request
d!v!ded !n
f!le already
chunks
obtain list "parallel" download!ng
of peers
same
share
chunks
t
as the range reques
trading
chunks
peer
↓
they may
loose that
to not
cunk
BitTorrent (1)
file divided into 256KB chunks.
peer joining torrent:
o has no chunks, but will accumulate them over time
o registers with tracker to get list of peers,
connects to subset of peers (“neighbors”)
while downloading, peer uploads chunks to other
peers.
peers may come and go
once peer has entire file, it may (selfishly) leave or
(altruistically) remain
2: Application Layer 109
BitTorrent (2) Sending Chunks: tit-for-tat
Alice sends chunks to four
Pulling Chunks
neighbors currently
at any given time,
sending her chunks at the
different peers have highest rate requestmayoat
different subsets of ↑ op4 +
10 secs
periodically, a peer
every 30 secs: randomly
(Alice) asks each
neighbor for list of select another peer,
chunks that they have. starts sending chunks
newly chosen peer may
Alice sends requests
for her missing chunks join top 4
“optimistically unchoke”
o rarest first
randomly choose request
a out of
log!n process
User starts Skype
SC registers with SN
o list of bootstrap SNs Skype
login server
SC logs in
(authenticate)
Call: SC contacts SN with
callee ID
o SN contacts other SNs
(unknown protocol, maybe
flooding) to find addr of
callee; returns addr to SC !s used
SC directly contacts callee PIP com
handskake
UDP
segment s!ze control !n
e
#
segment
2) P!pel!ng -
pers!stent
sorun
U
short t!me
Storage
3) on - use
ve
not
always-on
over all rel!ab!l!ty
4) DNS
not to date
up
IP addresses
may change
!ncorrect !nfo - can't connect to host
!nfo ?
cash!ng up-to-date
+