How to make an HTTP GET request manually with netcat?

Netcat,also known as "nc", is a powerful Unix-networking utility that enables users to interact with network services through a command-line interface (CLI). It uses both TCP and UDP network protocols for communication and is designed to be a reliable back-end tool to instantly provide network connectivity to other applications and users. Although Netcat is more often known for its use in file transfers and port scanning, it can also act as an HTTP client and send and accept raw HTTP requests.

Pre-Requisites

• Basic Understanding of HTTP Requests and Response and Knowledge of HTTP Response Code.
• Working Internet/Network connection or connectivity to the target server.
• Basic Understanding of Command-line interface including Basic commands for using Netcat.
• Availability of Netcat on the host System (Comes Pre-Installed in Linux/Unix Based System).

Basic Anatomy of HTTP Requests

HTTP communications mainly consist of an HTTP request and an HTTP response. The client makes An HTTP request and is processed by the server. The requests contain all of the details we require from the server, including the resource.

An HTTP request is a message sent via an internet browser to a web server. It includes numerous components:

• HTTP method: The HTTP approach specifies the type of request being made, including GET, POST, PUT, DELETE, and many others. It shows the intended movement to be performed on resources.
• URL (Uniform Resource Locator): The URL specifies the region of the aid at the server. It includes a scheme (e.g. HTTP or HTTPS), domain name, path etc.
• Headers: Provides additional information about the request like content type, authentication, cookies etc.
• Request body: Contains optional data sent with request mainly in POST, PUT, and PATCH requests to send data.

Example of Simple HTTP GET request includes:

GET /users/login.html HTTP/1.1
Host: example.com    
User-Agent: Mozilla/5.0 (Ubuntu; Linux x86_64;) Firefox/78.0
Accept: text/html, application/xhtml+xml,application/xml
Accept-Language: en-US, en;q=0.5
Accept-Encoding: gzip, deflate
Connection: close

Here in this example,

• GET/users/login.html is the request line which includes GET HTTP method.
• /users/login.html being the path of GET request to the resource on the server.
• Host: example.com is the header explaining the domain name of the web server where request is being sent.

Now heading back to the main topic, crafting a Netcat Command to Manually Make an HTTP GET Request with Netcat.

Getting Started with Netcat

As explained earlier, Netcat is a versatile and powerful network utility tool that uses TCP and UDP network protocols to read and write on a network. It can be used in various different scenarios, like attacking and defending web servers, and it is also used in debugging websites and web servers.

The most common uses for Netcat are given as follows:

Connecting to the server: Netcat is used to connect the server with specific port of a particular service which is running on the server.

We can connect to a server by using the following syntax on Netcat:

nc [Target IP Address] [Target Port]

For example, let's connect to a FTP server on some IP address like 192.168.1.6 with some port like 21, so the following syntax will be:

nc 192.168.1.6 21

Communication: Netcat can also be used to communicate or chat with and between two users. To chat, we need to first establish a connection between the said users. To do this, we need two devices. One can play the role of initiator to initiate and start the conversation, and one will be a listener. Once the connection is established, the communication will be a two-way communication, meaning the communication can be done from both ends.

Verbose Mode: In Netcat, we can access Verbose using the [-v] parameter. When using it, netcat generates extended information useful for debugging and troubleshooting network issues.Verbose mode generates additional information about the communication process, such as the IP address and port number of the remote host, the status of the communication, etc. Which can be helpful for diagnosing connection issues and identifying potential security threats.

File Transfer: Netcat can be used for transferring files across various devices.

A visual representation of communication between users in Netcat can be found below the following image:

Steps to a Manual GET Request with Netcat

Let's dive into crafting the netcat command for an HTTP GET request.

Step 1: Opening your terminal window:

On Linux, search for Terminal.

On Windows, search for Command Prompt or Windows PowerShell.

Step 2: Accessing Netcat Utility:

If Netcat is installed, you can directly run it by using nc, or you can use nc -h command to also know various built-in commands that come with the netcat.

If it's not installed on your system, you can install it using your system's package manager. For example, on Ubuntu, you can install it by using the following command:

sudo apt-get install netcat

Step 3: Connecting to the server:

Use the following command for your desired web server, followed by its port number (usually port 80 for HTTP):

nc example.com 80

Replace example .com with the actual name of the website you want to access.

Remember to replace example.com with the actual hostname.

Step 4: Sending the HTTP GET Request:

After connecting to the server, type the following HTTP GET request command, replacing the / with the path you want to access:

GET  /  HTTP/1.1
Host: example.com

• Replace the / and example.com with the actual path and website you want to access.
• Press ENTER twice to send the request.

Step 5: Reading the Response:

After sending the response, the web server will respond with the requested content printed on the terminal.

This response includes HTTP headers, status code, and the desired content of the requested file.

Step 6: Closing the Connection:

Once, you have received the response, you can close the connection by typing Ctrl + C on your keyboard.

Example in Action

Let's create the scenario of retrieving the homepage of google website:

Here, in this image GET / HTTP/1.1 represents the GET Request of / path for the targeted host: "google.com"

Conclusion

Manually making HTTP GET requests with Netcat is a valuable skill, which provides foundational insights to HTTP fundamentals and Web Communication. This process provides foundational understanding that is invaluable knowledge to navigate the internet.

Continued efforts towards this sector are advisable and some of the advantages are listed:

• APIs Testing: Sending custom request from Netcat to understand API functionality and Behaviour.
• Debugging Web Applications: Crafting targeted requests from netcat to diagnose issues.
• Learning Web Protocols: To gain knowledge of HTTP fundamentals and gain hands-on experience.