The Hypertext Transfer Protocol (HTTP)

Mainly HTTP is used to access the data on the World Wide Web. HTTP
functions as a combination of FTP and SMTP.

It is like File Transfer Protocol (FTP) because it transfers files and

uses the TCP’s service. However, it is much simpler than FTP because it
uses only one TCP connection. There is no separate control connection;
only data are transferred between the client and the server.

HTTP is similar to SMTP because the data transferred between the client
and the server look like SMTP messages. Also, the format of the messages
is controlled by Multipurpose Internet Mail Extensions (MIME) -like
headers.
 Unlike SMTP, the HTTP messages are not destined to be read by
humans; they are read and interpreted by the HTTP server and HTTP
client (browser).SMTP messages are stored and forwarded, but HTTP
messages are delivered immediately.

In HTTP, the commands from the client to the server are

embedded in a request message. The contents of the requested file or other
information are embedded in a response message. HTTP uses the services
of TCP on well-known port 80.

HTTP Transaction
Figure 16.4shows the HTTP transaction between the client and
server. Though HTTP uses the services of TCP, HTTP itself is a stateless
protocol. The client initializes the transaction by sending a request
message to the server. The server replies by sending a response to the
client.

Messages: formats of the request and response messages are similar; both
are shown in Figure 16.5. A request message consists of a request line,a
header, and sometimes a body whereas a response message consists of a
status line, a header, and sometimes a body.

Figure 16.4: HTTP

Figure 16.5: Request and response Message
Transaction

The first line in a request message is called as a request line; the

first line in the response message is called as the status line. There is one
common field in both is HTTP version, as shown in Figure 16.6.

Figure 16.6: Request and Status line

Fields of Request Line and Status Line: There is one common field in
both is HTTP version
 Request type: used in request message and are categorized into
methods as shown in Table 16.1
 URL: Uniform Resource Locator.
 HTTP version: current version of HTTP is 1.1.
 Status code: used in response message and consists of 3 digits –
codes in 100 range are informational, codes in 200 range indicate
successful request, codes in 300 range redirect the client to another
URL, codes in 400 range indicate an error at client and codes in
500 range indicate error at server site. Most common codes are
shown in Table 16.2.
 Status phrase: used in the response message. It explains the status
code in text form. Table 16.2 also provides the status phrase for
each code.

Table 16.1: Request methods

 Table 16.2: Status codes and Phrases

Header: used to exchange additional information between the client and

server. It consists of one or more header lines, belongs to one of four types
– general header, request header, response header and entity header.
A request message consists of general header, request header and entity
header whereas a response message consists of general header, response
header and entity header.
 General Header: provides the general information about the
message and present in both request and response message. (see
Table 16.3)
  Request Header: provides client’s configuration and client’s
preferred document format; present only in request message. (see
Table 16.4)
 Response Header: provides server’s configuration; present only in
response message. (see Table 16.5)
 Entity Header: provides information about the body of the
document; usually present in response message but sometimes
present in request message (in which PUT or POST method used).
(see Table 16.6)

Body: present in both request and response message and contains the
document to be send or received.

Table 16.3: General headers

Table 16.4: Request headers

Table 16.5: Response headers

 Table 16.6: Entity headers

Example 16.1:
This example retrieves a document. We use the GET method to
retrieve an image with the path /usr/bin/image1. The request line shows
the method (GET), the URL, and the HTTP version (1.1). The header has
two lines that show that the client can accept images in the GIF or JPEG
format. The request does not have a body. The response message contains
the status line and four lines of header. The header lines define the date,
server, MIME version, and length of the document. The body of the
document follows the header (see Figure 16.7).

Figure 16.7: Example 16.1

16.3 FTP

It is standard mechanism provided by TCP/IP for copying a file

from one host to another.

Problems during File transfer:

 Two systems may use different file name conventions.
 Two systems may have different ways to represent text and data.
 Two systems may have different directory structures.
☺ All these problems have been solved by FTP in very simple way.

FTP is different than other client-server programs. It establishes

two connections between hosts. One is used for data transfer and other is
used for control information (exchange of Commands and Responses).
FTP uses the services of TCP, it needs two connections: well-known port
21 is used for the Control connection and port 20 is used for the Data
connection.

Control connection remains connected during the entire interactive

FTP session whereas the Data connection is opened and then closed for
each file transferred. (See Figure 16.8)

Figure 16.8: FTP

Communication over Control Connection: FTP uses same approach like

SMTP to communicate across the control connection. It uses for
commands and responses the 7-bit ASCII (NVT ASCII) character set.
Each Command or Response is only one short line. Each line is terminated
with a two characters (<CRLF> carriage return and line feed) end-of-line
token. (See Figure 16.9)

Communication over Data Connection: File transfer occurs over the

data connection under the control of the commands sent over the control
connection. A file is to be copied from the server to the client under the
supervision of RETR command. A file is to be copied from the client to
the server under the supervision of STOR command. A list of directory or
file names is to be sent from the server to the client under the supervision
of LIST command. (See Figure 16.10)

Figure 16.9: FTP – Control connection

 Figure 16.10: FTP – Data connection
Before sending a file through the data connection; the client must define
the type of file to be transferred, the structure of the data and the
transmission mode.

File Type: FTP can transfer one of the following types across the data
connection.
• ASCII FILE is a default format (7bit ASCII encoding)
• EBCDIC FILE is used by IBM (EBCDIC encoding)
• IMAGE FILE is the default format for transferring binary file, it is
sent as continuous streams of bits without any encoding.

Data Structure: it uses one of the following interpretations about the

structure of the data.
• File structure format: is a continuous stream of bytes.
• A record structure: file is divided into records (text files).
• Page structure: file divided into pages, each page consist of page
number and page header. Pages can be accessed randomly or
sequentially.

Transmission Mode: it uses one of the three transmission modes

• Stream mode: default mode, data delivered from FTP to TCP, as
continuous streams of bytes (segments of appropriate size).
• Block mode: data can be delivered from FTP to TCP in blocks; each
block is preceded by a 3 byte header. First byte called block descriptor
next 2 bytes defines the size of the block in bytes.
• Compressed mode: compression method normally used is run-length
encoding in which consecutive appearances of data unit are replaced
by one occurrence and the number of repetitions (text file spaces-
blank).
• Anonymous FTP: Some sites have set of files available for public
access, to enable anonymous FTP, user does not need to have an
account and password to access files, instead, the user can use
anonymous as the user name and guest as the password.
16.4 ELECTRONIC MAIL

An Electronic - mail is one of the most popular Internet services.

Internet Designers probably never imagined the popularity of this
application program. At the beginning of the internet era, the messages
sent by email were short and consisted of text only. Today email is much
more complex, it allows a message to include text, audio, and video. It
also allows one message to be sent to one or more recipients.

Email Architecture
When both sender and receiver are connected to their mail servers
via a LAN or a WAN, we need two UAs and two pair of MTAs (client and
server), and pair of MAAs (client and server). This is the most common
email architecture used today. (See Figure 16.11)

Figure 16.11: Email architecture

 User Agent (UA): it provides service to the user to make the

process of sending and receiving a message easier.
 Message Transfer Agents (MTA): a client-server program used
to transfer the message across the internet.
 Message Access Agent (MAA): a client-server program that pulls
the stored email messages.

User Agent (UA)

First component of email system is a user agent (UA), there are
two types of UAs; namely Command driven UA and GUI based UA.
Some command driven UA examples are mail(Linux), eml(UNIX), etc.
Graphical User Interface UAs are more sophisticated and easier to use,
some GUI based UA examples are Outlook Express (Microsoft), Eudora
mail (open source), Netscape mail (Netscape), etc.
 Services provided by UAs are composing messages, reading
messages, replying messages, forwarding messages and handling
mailboxes

Sending Mail: for sending mail, the user, through the UA, creates mail
that looks very similar to postal mail. It has an envelope (sender and
receiver address) and a message; where message contains header (defines
sender, receiver, and subject of the message) and body (actual information
to be read by recipient).

Receiving Mail: When a user receives mail, UA informs to the user with a
notice and if the user is ready to read the mail. A list is displayed in which
each line contains a summary of the information about a particular
message in the mailbox.

Email Address: In the Internet, an email address consists of two parts: a

local part and a domain name, separated by @ sign (see Figure 16.12).

Figure 16.12: Email address

MIME (Multipurpose Internet Mail / Message Extensions)

As we know an electronic mail has a simple structure and it can
send messages only in NVT 7-bit ASCII format; hence it cannot be used
for languages that are not supported by 7-bitASCII characters (such as
French, German, Hebrew, Russian, Chinese, and Japanese). Also, it cannot
be used to send binary files or video or audio data. Solution to this
problem is Multipurpose Internet Mail Extensions (MIME), which is a
supplementary protocol that allows non-ASCII data to be sent through e-
mail. MIME transforms non-ASCII data at the sender site to NVT ASCII
data and delivers them to the client MTA to be sent through the Internet.
The message at the receiving site is transformed back to the original data.
MIME as a set of software functions that transforms non-ASCII data
(stream of bits) to ASCII data and vice versa, as shown in Figure 16.13.
 Figure 16.13: MIME

Message Transfer Agent: SMTP (Simple Mail Transfer Protocol)

Actual mail transfer is done through MTA (message transfer
agents). To send mail, a system must have the client MTA, and to receive
mail, a system must have a server MTA. The protocol that defines the
MTA client and server in the Internet is called as Simple Mail Transfer
Protocol (SMTP). Figure 16.14 shows the actual place of SMTP in today’s
email system.

Figure 16.14: MTA – SMTP

SMTP uses Commands and Responses to transfer the mail

messages between SMTP client and SMTP server. Every command or
response is terminated by a two-character (carriage return and line feed)
which is end-of-line token.

SMTP Commands: commands are sent from the client to the server.
SMTP defines 14 different commands. Out of these, first five are
mandatory; every implementation must support these five commands.
Next three are often used and highly recommended. Last six are seldom
used. (See Table 16.7)
 Table 16.7: SMTP Commands

Table 16.8: SMTP Response Codes

Responses: Responses are sent from the SMTP server to the SMTP client.
A response is a 3 digit code that may be followed by additional textual
information. Table 16.8 shows some of the response codes.
Mail Transfer: Transfer of a mail message occurs in three phases:
connection establishment, mail transfer, and connection termination.

Message Access Agent: POP3and IMAP4

In current scenario of the mail system, first and second stages of mail
delivery use SMTP. However, SMTP is not involved in the third stage
because SMTP is a push protocol; it pushes the message from the client to
the server. Whereas, the third stage needs a pull protocol; the client must
pull messages from the server. This third stage uses a Message Access
Agent.

Presently two MAA protocols are available: POP3 (Post Office

Protocol - version 3) and IMAP4 (Internet Mail Access Protocol - version
4). Figure 16.15 shows the place of these two protocols in today’s email
system.

Figure 16.15: POP3 and IMAP4

Post Office Protocol (POP3)

It is a simple and limited in functionality protocol. POP3 client
software is installed on the recipient computer; the POP3 server software
is installed on the recipient’s mail server. Mail access starts with the client
when the user needs to download e-mail from the mailbox on the mail
server. The client opens a connection with the server on TCP port 110 and
then sends its user name and password to access the mailbox. The user can
then list and retrieve the mail messages, one by one. Figure 16.16 shows
how mails are downloaded using POP3.

POP3 has two modes: the delete mode and the keep mode. In the
delete mode, after each retrieval, the mail is deleted from the mailbox. In
the keep mode, the mail remains in the mailbox after retrieval. The delete
mode is normally used when the user is working at his permanent
computer and can save and organize the received mails after reading or
replying. The keep mode is normally used when the user accesses his mail
away from her primary computer. The mail is read but kept in the system
for later retrieval and organizing.
 Figure 16.16: POP3 exchange of Commands and Responses

Internet Mail Access Protocol (IMAP4)

Other MAA protocol isIMAP4 is similar to POP3, but it has more
features; IMAP4 is more powerful and morecomplex.POP3 is deficient in
several ways. It does not allow the user to organize his mail on the server;
the user cannot have different folders on the server. In addition, POP3
does not allow the user to partially check the contents of the mail before
downloading. IMAP4 uses TCP’s port number 143.
IMAP4 provides the following some extra functions:
 A user can check the e-mail header prior to downloading.
 A user can search the contents of the e-mail for a specific string of
characters prior to downloading.
 A user can partially download e-mail. This is especially useful if
bandwidth is limited and the e-mail contains multimedia with high
bandwidth requirements.
 A user can create, delete, or rename mailboxes on the mail server.
 A user can create a hierarchy of mailboxes in a folder for e-mail
storage.
Domain Name System (DNS)
Every host connected to an internet has a unique IP address. IP
address of that host is used by other computers to find and connect to that
host. But people prefer usually host names instead of IP address of the
host. Therefore we need a mechanism or system that can translate or map
the host name to IP address or IP address to host name. In the internet such
mechanism or system is provided by one of the application layer protocol
called as Domain Name System (DNS).

Now we discuss how actually DNS works to map host name to IP

address. In Figure 16.23, a user wants to use a FTP client to access the
corresponding FTP server running on a remote host. The user knows only
the FTP server name, such as ftp.gnu.org. However, the TCP/IP suite
present on user’s FTP client needs the IP address of the FTP server to
make the connection. For mapping the FTP server’s name to IP address
following are steps.
1. The user passes the FTP server name to the FTP client.
2. FTP client passes the FTP server to the DNS client.
3. We know that each computer, after being booted, knows the
address of one DNS server. The DNS client sends a message to a
DNS server with a query that gives the FTP server name using the
known IP address of the DNS server.
4. Once query message received by the DNS server, responds to the
DNS client with the response message having DNS record (IP
address) of the desired FTP server.
5. The DNS client passes the IP address to the FTP client.
6. Now the FTP client uses the received IP address to access the FTP
server.

Figure 16.23: Working of DNS

Name Space
Internet is divided into over 200 top level domains. Each
domain is divided into sub-domains, which are further partitioned. All
domains can be represented by a tree. The leaves of the tree represent
domains that have no sub-domains (but contain machines). A leaf domain
may contain a single host or represent a company and contain thousands of
hosts. Top level domains could be generic and country domains as shown
in the Figure 16.24.
 Figure 16.24: Domain Name space

The namespace needs to be made hierarchical to be able to scale.

The idea is to name objects based on
• Location (within country, set of organizations, set of companies, etc).
• Unit within that location (company within set of company, etc).
• Object within unit (name of person in company).

A domain name is the sequence of labels from a node to the

root, separated by dots (“.”s), read from left to right. The name space has a
maximum depth of 127 levels. Domain names are limited to 255
characters in length.

A node’s domain name identifies its position in the name

space. Each domain controls how it allocates the domains under it i.e.
Japan makes a domains ac.jp and co.jp that may be different than edu and
com. To create a new domain, permission is required from the domain that
will include it; once created, it can create sub-domains without having to
ask permission from the higher up domains.

Fully Qualified Domain Name (FQDN):If a label is terminated by a null

string, it is called as fully qualified domain name (FQDN).An FQDN is a
domain name that contains the full name of a host. A DNS server can only
match an FQDN to an address. Note that the name must end with a null
label, but because null means nothing, the label ends with a dot (.).

Partially Qualified Domain Name (PQDN):If a label is not terminated

by a nullstring, it is called a partially qualified domain name (PQDN).A
PQDN starts from anode, but it does not reach the root. Example of FQDN
and PQDN are shown in the figure 16.25.
 Figure 16.25: FQDN and PQDN

Distribution of Name Space

Storing the information comprised in the domain name space on
one single computer is very inefficient and also not reliable because it is a
huge amount of information. It is inefficient because responding to
requests from all over the world places a heavy load on the system. It is
not reliable because any failure makes the data inaccessible.

Hierarchy of Name Servers

The solution to this problem is to distribute the information among
many computers called DNS servers. One way to do this is to divide the
whole space into many domains based on the first level. In other words,
we let the root stand alone and create as many domains (sub trees) as there
are first-level nodes. Because a domain created this way could be very
large, DNS allows domains to be divided further into smaller domains
(sub domains). Each server can be responsible (authoritative) for either a
large or small domain. In other words, we have a hierarchy of servers in
the same way that we have a hierarchy of names (see Figure 16.26).

Figure 16.26: Hierarchy of Name Servers

Zone
Since the complete domain name hierarchy cannot be stored on a
single server, it is divided among many servers. What a server is
responsible for or has authority over is called a zone. We can define a
zone as a contiguous part of the entire tree. If a server accepts
responsibility for a domain and does not divide the domain into smaller
domains, the “domain” and the “zone” refer to the same thing. The server
makes a database called a zone file and keeps all the information for every
node under that domain.

However, if a server divides its domain into sub domains and

delegates part of its authority to other servers, “domain” and “zone” refer
to different things. The information about the nodes in the sub domains is
stored in the servers at the lower levels, with the original server keeping
some sort of reference to these lower-level servers (see Figure 16.27).

Figure 16.27: Zones and Domains

Root Server
A root server is a server whose zone consists of the whole tree. A
root server usually does not store any information about domains but
delegates its authority to other servers, keeping references to those servers.
There are several root servers, each covering the whole domain name
space. The root servers are distributed all around the world.

Primary and Secondary Servers

DNS defines two types of servers: primary and secondary. A
primary server is a server that stores a file about the zone for which it is an
authority. It is responsible for creating, maintaining, and updating the zone
file. It stores the zone file on a local disk.

A secondary server is a server that transfers the complete

information about a zone from another server (primary or secondary) and
stores the file on its local disk. The secondary server neither creates nor
updates the zone files. If updating is required, it must be done by the
primary server, which sends the updated version to the secondary. The
primary and secondary servers are both authoritative for the zones they
serve.
Resolution
Mapping of Domain name to IP address or IP address to Domain
name is called resolution.

Resolver
A host that needs to map a name to an address or an address to a
name calls a DNS client called as resolver. The resolve accesses the
closest DNS server with a mapping request.

Mapping Names to Addresses

When, the resolver gives a domain name to the server and asks for
the corresponding address. In this case, the server checks the generic
domains or the country domains to find the mapping.

Mapping Addresses to Names

When, the resolver gives an IP address to the server and asks for
the corresponding domain name; this type of query is called as PTR query.
To answer queries of this kind, DNS uses the inverse domain.

Recursive Resolution
The client (resolver) can ask for a recursive answer from a name
server. This means that the resolver expects the server to supply the final
answer. If the server is the authority for the domain name, it checks its
database and responds. If the server is not the authority, it sends the
request to another server (the parent usually) and waits for the response. If
the parent is the authority, it responds; otherwise, it sends the query to yet
another server. When the query is finally resolved, the response travels
back until it finally reaches the requesting client (see Figure 16.28).

Figure 16.28: Recursive Resolution

Iterative Resolution
If the client does not ask for a recursive answer, the mapping can
be done iteratively. If the server is an authority for the name, it sends the
answer. If it is not, it returns (to the client) the IP address of the server that
it thinks can resolve the query. The client is responsible for repeating the
query to this second server. If the newly addressed server can resolve the
problem, it answers the query with the IP address; otherwise, it returns the
IP address of a new server to the client. Now the client must repeat the
query to the third server. This process is called iterative because the client
repeats the same query to multiple servers (see Figure 16.29).

Figure 16.29: Iterative Resolution

Caching
Each time a server receives a query for a name that is not in its
domain, it needs to search its database for a server IP address. Reduction
of this search time would increase efficiency. DNS handles this with a
mechanism called caching. When a server asks for a mapping from
another server and receives the response, it stores this information in its
cache memory before sending it to the client. If the same or another client
asks for the same mapping, it can check its cache memory and resolve the
problem. However, to inform the client that the response is coming from
the cache memory and not from an authoritative source, the server marks
the response as unauthoritative.

Caching speeds up resolution, but it can also be problematic. If a

server caches a mapping for a long time, it may send an outdated mapping
to the client. To solve this, two techniques are used. First, the authoritative
server always adds information to the mapping called time-to-live (TTL).
It defines the time in seconds that the receiving server can cache the
information. After that time, the mapping is invalid and any query must be
sent again to the authoritative server. Second, DNS requires that each
server keep a TTL counter for each mapping it caches. The cache memory
must be searched periodically and those mappings with an expired TTL
must bepurged.

DNS Messages
DNS has two types of messages: query and response. Both of them
have the same format. The query message consists of a header and
question records whereas the response message consists of a header,
question records, answer records, authoritative records, and additional
records (see Figure 16.30).

Figure 16.30: DNS – Query and Response message

Header: Both query and response messages have the same header format
with some fields set to zero for the query messages. The header is 12 bytes
and its format is shown in Figure 16.31.

Figure 16.31: Header format

The header fields are as follows:

 Identification: This is a 16-bit field used by the client to match the
response with the query. The client uses a different identification
number each time it sends a query. The server duplicates this number
in the corresponding response.
 Flags: This is a 16-bit field consisting of the subfields that defines the
type of message, type of answer requested, the type of desired
resolution, and so on.
 Number of question records: This is a 16-bit field containing the
number of queries in the question section of the message.
 Number of answer records: This is a 16-bit field containing the
number of answer records in the answer section of the response
message. Its value is zero in the query message.
 Number of authoritative records: This is a 16-bit field containing
the number of authoritative records in the authoritative section of a
response message. Its value is zero in the query message.
 Number of additional records: This is a 16-bit field containing the
number of additional records in the additional section of a response
message. Its value is zero in the query message.

Question Section: This is a section consisting of one or more question

records. It is present on both query and response messages.
Answer Section: This is a section consisting of one or more resource
records. It is present only on response messages. This section includes the
answer from the server to the client (resolver). Authoritative Section:
This is a section consisting of one or more resource records. It is present
only on response messages. This section gives information (domain name)
about one or more authoritative servers for the query.

Additional Information Section: This is a section consisting of one or

more resource records. It is present only on response messages. This
section provides additional information that may help there solver. For
example, a server may give the domain name of an authoritative server to
the resolver in the authoritative section, and include the IP address of the
same authoritative server in the additional information section.

Resource Record
Each domain name is associated with a record called as resource record.
The DNS server database consists of resource records. Resource records
are also what is returned by the server to the client. Figure 16.32 shows the
format of are source record.

Figure 16.32: DNS – Resource Record

A resource record has five parts namely Domain name, Time to

Live (TTL), Class, Type and Value.
 Domain Name: The Domain name tells the domain to which this
record applies. Normally many records exist for each domain and each
copy of the database holds information about multiple domains. This
field is the primary search key used to satisfy queries. The order of the
records in the database is not important.
 Time to Live (TTL):Time to live field gives an indication of how
stable the record is. Information that is highly stable is assigned a large
value, such as 86400 (number of seconds in a day). Information that is
highly volatile is assigned a small value, such as 60 seconds (1
minute).
 Class: Class field is always IN for Internet information.
 Type: Type field tells what kind of record this is (see the Table 16.11).
 Value: Value field gives the resource dada value (IP address for type
‘A’ record).

Resource Record types: there are eight types of DNS records as shown in
the Table 16.11.
• Start of Authority (SOA): SOA record provides the name of the
primary source of information about the name server’s zone, the e-
mail address of its administrator, a unique serial number and various
flags and timeouts.

Table 16.11: Resource Record types

• Address (A): Address record is the most important record. It holds a

32 bit IP address for some host; Every Internet host must have at least
one IP address; some hosts have two or more IP addresses (being
connected to multiple networks, having one type A resource record
per network connection); DNS can be made to cycle through those
(for first request to return first record, for second request to return the
second A type record).
• Mail Exchange (MX): MX record specifies the name of the host
prepared to accept e-mail for the specified domain; it is used because
not every machine is prepared to accept e-mail. If someone wants to
send e-mail to [email protected], the sending host needs to find a
mail server at microsoft.com that is willing to accept e-mail. MX
record can provide this information.
• Name Server (NS):The NS record specifies Name Servers i.e., every
DNS database normally has an NS record for each of the top-level
domains.
• Canonical Name (CNAME): The CNAME records allow aliases to
be created. In example: cs.mit.edu 86400 IN CNAME 1cs.mit.edu
creates an alias for 1cs.mit.edu (real domain name).
• Pointer (PTR): The PTR record used to associate a name with an IP
address to allow lookups of the IP address and return the name of the
corresponding machine. This is called reverse lookup.