In the Name of the Most High

Network Applications & HTTP

Internet Engineering
Spring 2022
Mehdi Aminian

Islamic Azad University North Tehran Branch

Faculty of Electrical and Computer Engineering

These slides are based in part on materials provided

by Prof. Jim Kurose
2 Outlines

❑ principles of network applications

❑ Web and HTTP
3 Outlines

❑ principles of network applications

❑ Web and HTTP
application layer
4

 our goals:  learn about protocols

 conceptual, by examining popular
implementation application-level
aspects of network protocols
application protocols  HTTP

 transport-layer service
models
 client-server paradigm

 peer-to-peer
paradigm
Some network apps
5

 e-mail  voice over IP (e.g., Skype)

 web  real-time video
 text messaging conferencing
 remote login  social networking
 P2P file sharing  search
 multi-user network games  …
 streaming stored video  …
(YouTube, Hulu, Netflix)
Creating a network app application
transport
network
data link
physical
6

write programs that:

 run on (different) end systems

 communicate over network

 e.g., web server software

communicates with browser
software application
transport
network
application
no need to write software for
data link
physical transport
network
network-core devices data link
physical

 network-core devices do not

run user applications
 applications on end systems
allows for rapid app
development, propagation
 Application architectures
7

possible structure of applications:

 client-server

 peer-to-peer (P2P)
 Client-server architecture
8

server:
 always-on host
 permanent IP address
 data centers for scaling

clients:
 communicate with server
client/server
 may be intermittently connected
 may have dynamic IP addresses
 do not communicate directly with
each other
 P2P architecture
9
peer-peer
 no always-on server
 arbitrary end systems directly
communicate
 peers request service from
other peers, provide service in
return to other peers
 self scalability – new
peers bring new service
capacity, as well as new
service demands
 peers are intermittently
connected and change IP
addresses
 complex management
 Processes communicating
10

process: program running clients, servers

within a host client process: process that
 within same host, two initiates communication
processes communicate using server process: process
inter-process communication that waits to be contacted
(defined by OS)
 processes in different hosts
aside: applications with P2P
communicate by exchanging
architectures have client
messages
processes & server processes
 Sockets
11

 process sends/receives messages to/from its socket

 socket analogous to door
 sending process shoves message out door
 sending process relies on transport infrastructure on other
side of door to deliver message to socket at receiving
process
controlled by
app developer
application application
socket
process process

transport transport
network network controlled
link by OS
link Internet
physical physical
 Addressing processes
12

 to receive messages, process  identifier includes both IP

must have identifier address and port numbers
associated with process on
 host device has unique 32-
host.
bit IP address
 example port numbers:
 Q: does IP address of host
on which process runs suffice  HTTP server: 80
for identifying the process?  mail server: 25
 to send HTTP message to
▪ A: no, many processes gaia.cs.umass.edu web
can be running on same
host server:
 IP address: 128.119.245.12
 port number: 80
 more shortly…
 App-layer protocol defines
13
open protocols:
 types of messages exchanged,
 defined in RFCs
 e.g., request, response
 allows for interoperability
 message syntax:
 e.g., HTTP, SMTP
 what fields in messages & proprietary protocols:
how fields are delineated
 e.g., Skype
 message semantics
 meaning of information in
fields
 rules for when and how
processes send & respond to
messages
 What transport service does an app need?
14

data integrity throughput

 some apps (e.g., file transfer, ❑ some apps (e.g.,
web transactions) require multimedia) require
100% reliable data transfer minimum amount of
 other apps (e.g., audio) can
throughput to be
tolerate some loss “effective”
❑ other apps (“elastic apps”)
timing make use of whatever
 some apps (e.g., Internet
throughput they get
telephony, interactive security
games) require low delay
to be “effective” ❑ encryption, data integrity,
…
 Transport service requirements: common apps
15

application data loss throughput time sensitive

file transfer no loss elastic no

e-mail no loss elastic no
Web documents no loss elastic no
real-time audio/video loss-tolerant audio: 5kbps-1Mbps yes, 100’s
video:10kbps-5Mbps msec
stored audio/video loss-tolerant same as above
interactive games loss-tolerant few kbps up yes, few secs
text messaging no loss elastic yes, 100’s
msec
yes and no
 Internet transport protocols services
16

TCP service: UDP service:

 reliable transport between  unreliable data transfer
sending and receiving process
 flow control: sender won’t between sending and
overwhelm receiver receiving process
 congestion control: throttle  does not provide: reliability,
sender when network flow control, congestion
overloaded
 does not provide: timing, control, timing, throughput
minimum throughput guarantee, guarantee, security,
security orconnection setup,
 connection-oriented: setup
required between client and
server processes Q: why bother? Why is there
a UDP?
 Internet apps: application, transport protocols
17
application underlying
application layer protocol transport protocol

e-mail SMTP [RFC 2821] TCP

remote terminal access Telnet [RFC 854] TCP
Web HTTP [RFC 2616] TCP
file transfer FTP [RFC 959] TCP
streaming multimedia HTTP (e.g., YouTube), TCP or UDP
RTP [RFC 1889]
Internet telephony SIP, RTP, proprietary
(e.g., Skype) TCP or UDP
Securing TCP
18

TCP & UDP SSL is at app layer

 no encryption  Apps use SSL libraries,

 cleartext passwds sent which “talk” to TCP

into socket traverse SSL socket API
Internet in cleartext ❑ cleartext passwds sent
SSL into socket traverse
 provides encrypted TCP
Internet encrypted
connection
 data integrity

 end-point authentication
19 Outlines

❑ principles of network applications

❑ Web and HTTP
 Web and HTTP
20

First, a review…
 web page consists of objects
 object can be HTML file, JPEG image, Java
applet, audio file,…
 web page consists of base HTML-file which
includes several referenced objects
 each object is addressable by a URL, e.g.,

www.someschool.edu/someDept/pic.gif

host name path name

 Uniform Resource Locators (URLs)

 <protocol(scheme)>://<user>:<pass>@<host>:<port>/<path>?<
query>#<frag> (rfc1738)
 First part - protocol
 terminated by colon ( : )
 common protocols are http, ftp, mailto, telnet,
 i.e.: http: ftp: mailto: telnet:
 Second part - varies according to protocol
 mailto - e-mail address e.g.:
◼ mailto: [email protected]
 resource-oriented protocols (http, ftp etc)
◼ Host name + domain names (preceded by //)
◼ may optionally include username, password and port
◼ Pathname (usually related to the path of a file on the server)
◼ i.e. //fully-qualified-domain-name/path-to-document
 Optional third parts
 Query string (preceded by ?)
 Fragment identifier (preceded by #)
Example URLs
 mailto:[email protected]
 http://www.aut.ac.ir:80/
 ftp://me:[email protected]/pub
 http://www.bing.com/search?q=web&go=&qs=n&form=QBLH
&pq=web&sc=0-0&sp=-1
 file://c:/windows/ file:///home/mehdi/work
URL (cont’d)
23

 Query: a mechanism to pass information from client to

active pages or forms, e.g.,
 Fill
information in a university registration form
 Ask Google to search a phrase

 Starts with “?”

 name=value format
 “&” is the border between multiple parameters

http://www.example.com/submit.php?name=mehdi&family=ahmadi
URL (cont’d)
24

 Frag: A name for a part of resource

A section in a document
http://www.example.com/paper.html#results

 Handled by browser
 Browser gets whole resource (doc) from sever
 In display time, it jumps to the specified part
 HTTP overview
25

HTTP: hypertext transfer

protocol
 Web’s application layer PC running
protocol Firefox browser

 client/server model
 client:browser that
requests, receives, server
(using HTTP protocol) running
and “displays” Web Apache Web
server
objects
 server: Web server iphone running
sends (using HTTP Safari browser
protocol) objects in
response to requests
 HTTP overview (continued)
26

uses TCP: HTTP is “stateless”

 client initiates TCP  server maintains no
information about
connection (creates socket) past client requests
to server, port 80
 server accepts TCP aside
connection from client protocols that maintain
 HTTP messages (application- “state” are complex!
layer protocol messages) ❑ past history (state) must be
exchanged between maintained
browser (HTTP client) and ❑ if server/client crashes, their
views of “state” may be
Web server (HTTP server) inconsistent, must be
 TCP connection closed reconciled
HTTP connections
27

non-persistent HTTP persistent HTTP

 at most one object sent  multiple objects can
over TCP connection be sent over single
 connection then TCP connection
closed between client, server
 downloading multiple
objects required
multiple connections
 Non-persistent HTTP
28
suppose user enters URL: (contains text,
www.someSchool.edu/someDepartment/home.index references to 10
jpeg images)
1a. HTTP client initiates TCP
connection to HTTP server 1b. HTTP server at host
(process) at www.someSchool.edu www.someSchool.edu waiting
on port 80 for TCP connection at port 80.
“accepts” connection, notifying
2. HTTP client sends HTTP request client
message (containing URL) into
TCP connection socket. 3. HTTP server receives request
Message indicates that client message, forms response
wants object message containing requested
someDepartment/home.index object, and sends message into
its socket
time
 Non-persistent HTTP (cont.)
29

4. HTTP server closes TCP

connection.
5. HTTP client receives response
message containing html file,
displays html. Parsing html file,
finds 10 referenced jpeg objects

time
6. Steps 1-5 repeated for each of
10 jpeg objects
 Non-persistent HTTP: response time
30

RTT (definition): time for a small

packet to travel from client to
server and back
initiate TCP
HTTP response time: connection
 one RTT to initiate TCP RTT
connection request
 one RTT for HTTP request and file
first few bytes of HTTP response RTT
time to
transmit
to return file
 file transmission time file
received
 non-persistent HTTP response
time =
time
2RTT+ file transmission time
time
 Persistent HTTP
31

non-persistent HTTP issues: persistent HTTP:

 requires 2 RTTs per object  server leaves connection
 OS overhead for each TCP open after sending response
connection  subsequent HTTP messages
 browsers often open parallel between same client/server
TCP connections to fetch sent over open connection
referenced objects  client sends requests as soon
as it encounters a referenced
object
 as little as one RTT for all the
referenced objects
 HTTP request message
32

 two types of HTTP messages: request, response

 HTTP request message:
 ASCII (human-readable format) carriage return character
line-feed character
request line
(GET, POST, GET /index.html HTTP/1.1\r\n
HEAD commands) Host: www-net.cs.umass.edu\r\n
User-Agent: Firefox/3.6.10\r\n
Accept: text/html,application/xhtml+xml\r\n
header Accept-Language: en-us,en;q=0.5\r\n
lines Accept-Encoding: gzip,deflate\r\n
Accept-Charset: ISO-8859-1,utf-8;q=0.7\r\n
carriage return, Keep-Alive: 115\r\n
line feed at start Connection: keep-alive\r\n
\r\n
of line indicates
end of header lines
HTTP request message: general format
33

method sp URL sp version cr lf request

line
header field name value cr lf
header
~
~ ~
~ lines

header field name value cr lf

cr lf

~
~ entity body ~
~ body
 Uploading form input
34

POST method:
 web page often includes
form input
 input is uploaded to server
in entity body
URL method:
 uses GET method
 input is uploaded in URL
field of request line:
www.somesite.com/animalsearch?monkeys&banana
 Method types
35

HTTP/1.0: HTTP/1.1:
 GET  GET, POST, HEAD
 POST  PUT
 HEAD  uploads file in entity
 asksserver to leave body to path
requested object out specified in URL field
of response  DELETE
 deletes file specified
in the URL field
 HTTP response message
36
status line
(protocol
HTTP/1.1 200 OK\r\n
status code Date: Sun, 26 Sep 2010 20:09:20 GMT\r\n
status phrase) Server: Apache/2.0.52 (CentOS)\r\n
Last-Modified: Tue, 30 Oct 2007 17:00:02
GMT\r\n
header ETag: "17dc6-a5c-bf716880"\r\n
Accept-Ranges: bytes\r\n
lines Content-Length: 2652\r\n
Keep-Alive: timeout=10, max=100\r\n
Connection: Keep-Alive\r\n
Content-Type: text/html; charset=ISO-8859-
1\r\n
\r\n
data, e.g., data data data data data ...
requested
HTML file
 HTTP response status codes
37

❑ status code appears in 1st line in server-to-

client response message.
❑ some sample codes:
200 OK
 request succeeded, requested object later in this msg
301 Moved Permanently
 requested object moved, new location specified later in this msg
(Location:)
400 Bad Request
 request msg not understood by server
404 Not Found
 requested document not found on this server
505 HTTP Version Not Supported
 Trying out HTTP (client side) for yourself
38
1. Telnet to your favorite Web server:

telnet cis.poly.edu 80 opens TCP connection to port 80

(default HTTP server port) at cis.poly.edu.
anything typed in sent
to port 80 at cis.poly.edu

2. type in a GET HTTP request:

GET /~ross/ HTTP/1.1 by typing this in (hit carriage
Host: cis.poly.edu return twice), you send
this minimal (but complete)
GET request to HTTP server

3. look at response message sent by HTTP server!

(or use Wireshark to look at captured HTTP request/response)
HTTP - Header Fields
39

 General-header: These header fields have general

applicability for both request and response messages.
 Client Request-header: These header fields have
applicability only for request messages.
 Server Response-header: These header fields have
applicability only for response messages.
 Entity-header: These header fields define meta
information about the entity-body or, if no body is
present, about the resource identified by the request.
General Headers
40

 Date: date & time that message is created

 Connection: close or keep-alive
 Close:
Non-persistent connection
 Keep-alive: Persistent connection

 Via: Information about the intermediate nodes between

two sides
 Proxy servers
Request Headers
41

 Host: The name of the server (required, why?)

 Referer: URL that contains requested URL
 Information about client
 User-Agent: The client program
 UA-OS: The OS of client program

 UA-Disp: Information about display of client

 Accept: The acceptable media types

 Accept-Encoding: Acceptable encoding

 Accept-Language: What language are acceptable

Request Headers (cont’d)
42

 If-Modified-Since: Request is processed if the objected is

modified since the specified time
 Authorization: Response to the authenticate requests
 Range: Specific range (in byte) of resource
Response Headers
43

 Server: Information about server

 WWW-Authenticate: Used to specify authentication
parameters by server
 Proxy-Authenticate: Used to specify authentication
parameters by proxy
 Set-Cookie: To send a cookie to client
Entity Headers
44

 Content-Length: The length of body (in byte)

 Content-Type: The type of entity
 MIME types: text/html, image/gif
 Allow: The allowed request method can be performed on
the entity
 This is in response of OPTIONS method
 Location: The new location of entity to redirect client
Entity Headers (cont’d)
45

 Content-Range: Range of this entity in the entire resource

 Expire: The date and time at which the entity will expire
 Last-Modified: The date and time of last modification of
entity
 ETag: A tag for entity (usually time based)
 Cache-Control: To control entity caching
User-server state: cookies
46
example:
many Web sites use cookies
 Susan always access Internet
four components:
from PC
1) cookieheader line of
 visits specific e-commerce site
HTTP response
message for first time
2) cookie header line in  when initial HTTP requests

next HTTP request arrives at site, site creates:

message  unique ID
3) cookie file kept on  entry in backend
user’s host, managed database for ID
by user’s browser
4) back-end database
at Web site
 Cookies: keeping “state” (cont.)
47

client server

ebay 8734
usual http request msg Amazon server
cookie file creates ID
usual http response
1678 for user create backend
ebay 8734
set-cookie: 1678 entry database
amazon 1678
usual http request msg
cookie: 1678 cookie- access
specific
usual http response msg action

one week later:

access
ebay 8734 usual http request msg
amazon 1678 cookie: 1678 cookie-
specific
 Cookies (continued)
48

what cookies can be used aside

for:
 authorization cookies and privacy:
 shopping carts ❑ cookies permit sites to
 recommendations learn a lot about you
 user session state (Web e- ❑ you may supply name and
mail) e-mail to sites

how to keep “state”:

❑ protocol endpoints: maintain state at
sender/receiver over multiple
transactions
❑ cookies: http messages carry state
 Web caches (proxy server)
49

goal: satisfy client request without involving origin server

 user sets browser: Web
accesses via cache
proxy
 browser sends all HTTP server
requests to cache client
origin
server
 object in cache: cache
returns object
 else cache requests
object from origin client origin
server, then returns server
object to client
HTTP Proxy Applications
50

 Authentication
 Client side: Authenticate clients before they access web
 Server side: Authenticate clients before they access the server

 Accounting: Log client activities

 Security: Analyze request before sending it to server
 Integrated in modern firewalls
 Filtering: Limit access to specified contents
 Anonymizer: Anonymous web browsing
 Caching (more details in the following slides)
 More about Web caching
51

 cache acts as both why Web caching?

client and server  reduce response time for
 server for original requesting client request
client
 client to origin server  reduce traffic on an

 typically cache is institution’s access link

installed by ISP  Internet dense with

(university, company, caches: enables “poor”

residential ISP) content providers to
effectively deliver
content (so too does P2P
file sharing)
 Caching example:
52

assumptions:
❑ avg object size: 100K bits origin
❑ avg request rate from browsers to servers
public
origin servers:15/sec Internet
❑ avg data rate to browsers: 1.50 Mbps
❑ RTT from institutional router to any
origin server: 2 sec
❑ access link rate: 1.54 Mbps 1.54 Mbps
access link
consequences: problem! institutional
❑ LAN utilization: 15% network
1 Gbps LAN
❑ access link utilization = 99%
❑ total delay = Internet delay + access
delay + LAN delay
= 2 sec + minutes + usecs
Caching example: fatter access link
assumptions:
❑ avg object size: 100K bits origin
❑ avg request rate from browsers to servers
origin servers:15/sec public
❑ avg data rate to browsers: 1.50 Mbps Internet
❑ RTT from institutional router to any
origin server: 2 sec
❑ access link rate: 1.54 Mbps
154 Mbps 1.54 Mbps
154 Mbps
consequences: access link

❑ LAN utilization: 15% institutional

❑ access link utilization = 99% 9.9% network
1 Gbps LAN
❑ total delay = Internet delay + access
delay + LAN delay
= 2 sec + minutes + usecs
msecs

53
Cost: increased access link speed (not cheap!)
Caching example: install local cache
assumptions:
❑ avg object size: 100K bits origin
❑ avg request rate from browsers to servers
origin servers:15/sec public
❑ avg data rate to browsers: 1.50 Mbps Internet
❑ RTT from institutional router to any
origin server: 2 sec
❑ access link rate: 1.54 Mbps 1.54 Mbps
consequences: access link

❑ LAN utilization: 15% institutional

access link utilization = 100% network
❑ ? 1 Gbps LAN
❑ total delay = Internet
? delay + access
delay + LAN delay local web
How to compute link
= 2 sec + minutes + usecs cache
utilization, delay?
Cost: web cache (cheap!)
Caching example: install local cache
Calculating access link
utilization, delay with cache: origin
servers
 suppose cache hit rate is 0.4 public
 40% requests satisfied at cache, Internet
60% requests satisfied at origin
❑ access link utilization:
❑ 60% of requests use access link 1.54 Mbps
access link
❑ data rate to browsers over access link
= 0.6*1.50 Mbps = .9 Mbps institutional
❑ utilization = 0.9/1.54 = .58 network
1 Gbps LAN
❑ total delay
▪ = 0.6 * (delay from origin servers) +0.4 local web
* (delay when satisfied at cache) cache
▪ = 0.6 (2.01) + 0.4 (~msecs)
▪ = ~ 1.2 secs
55 ▪ less than with 154 Mbps link (and
cheaper too!)
 Conditional GET
56
cache server
 Goal: don’t send object if
cache has up-to-date
HTTP request msg
cached version If-modified-since: <date> object
not
 no object transmission delay modified
HTTP response
 lower link utilization before
HTTP/1.0
<date>
 cache: specify date of 304 Not Modified

cached copy in HTTP

request
If-modified-since: HTTP request msg
<date> If-modified-since: <date> object
modified
 server: response contains HTTP response after
no object if cached copy is HTTP/1.0 200 OK <date>
up-to-date: <data>
HTTP/1.0 304 Not
HTTP Authentication
 Protect web content from those who don’t have a “need to know”
 Require users to authenticate using a userid/password before they are
allowed access to certain URLs
 HTTP/1.1 requires that when a user makes a request for a protected
resource the server responds with a authentication request header
 WWW-Authenticate
◼ contains enough pertinent information to carry out a “challenge-response” session
between the user and the server

Client requests a protected resource

Client
Web Server
Server responds with a 401 (not
authorized and a challenge request
for the client to authenticate
Client Response
 Well established clients like Firefox, Internet Explorer …. will respond to the
challenge request (WWW-Authenticate) by presenting the user with a small pop-
up window with data entry fields for
 userid
 password
 a Submit button and a Cancel button
 entering a valid userid and password will post the data to the server, the server
will attempt authentication and if authenticated will serve the originally requested
resource.
HTTP Authentication
59
HTTP Security
60

 No mechanism in HTTP to protect data

 HTTP over SSL/TLS is a popular solution
More about HTTP
61

 HTTP: The Definitive Guide by David Gourley