We all use the internet daily, but have you ever stopped to consider the intricacies happening behind the scenes when you type a URL into your browser and hit enter? I have, and I am eager to share my findings with you. In this blog post, I will give a brief overview of the processes involved in delivering a web page to your browser, covering a wide range of topics such as DNS request, TCP/IP, Load-balancer, Firewall, SSL, HTTP/HTTPS, Web and Application servers and Database. So, please make yourself comfortable and prepare yourself for a journey through the inner workings of the internet as we discover what happens when you type in https://www.google.com. Are you ready to uncover the secrets of the internet? Let's begin.

We'll begin by defining some terms that will appear that you will encounter in this blog post:

Server

In computing, a server is a piece of computer hardware or software (computer program) that provides functionality for other programs or devices, called “clients”. This architecture is called the client–server model. — Wikipedia

The Client-Server Model

The client is typically the ‘thing’ requesting information. Typically clients are the web user’s internet-connected devices, such as a computer, laptop or phone connected to Wi-Fi or mobile network and web-accessing software available on those devices or browsers like Google Chrome, Firefox, and Safari. In our example, the client is in fact the browser and the device we are using to type in www.google.com

The server is the ‘thing’ sending or serving information to the client. They store webpages, sites, or apps. In our example, it would be the google server.

When a client device visits a website, it requests a copy of the webpage from the server which is then downloaded onto the client to be displayed in the user’s web browser.

Now that we have that sorted out, let's dive in.

When you type "https://www.google.com" into your browser and press enter, it is similar to making a phone call to a friend's house. Just as you need to know your friend's phone number to make the call, your computer needs to know the IP address (think of it as a number used to identify a computer on the internet) of the server that Google's website is hosted on. The first thing your device does is break down the URL you entered (https://www.google.com) into pieces! But wait, what's a URL?...

What’s a URL?

URL stands for Uniform Resource Locator and is used to access a website. There are several parts in a URL: the protocol, hostname, port, path and more. For this blog post, we will look at the protocol and hostname. In the case of https://www.google.com: HTTPS is the protocol (more on that later) and www.google.com is the hostname, which can be a domain like in this case or also an IP address. The hostname is the part of the URL that's used to identify the website. Now that we have our hostname, we should go ahead and make a request to the server hosting the website, not so fast. You see, the thing is computers on a network are identified by their IP address, and our browser doesn't know the IP address of google's server yet! So our browser has no way to access the server hence it needs a way to get the IP address of google's server, and that is where the DNS comes in. You might ask, why does google not just tell us the IP address of their server so we can directly access it without the need for a DNS, well that's because it's easier for we humans to remember names like "google.com" as opposed to a random sequence of numbers like this "196.104.234.255".

DNS

Think of the Domain Name System (DNS) like the phonebook of the internet! The DNS converts human-friendly domain names like "google.com" into IP addresses that computers can understand. So, when you type in the URL, your computer sends a request to the DNS server to find the IP address of the Google server.

This process called DNS resolution or DNS lookup has 4 steps (and servers) involved in loading a webpage:

DNS recursor - It’s a server designed to receive queries from clients through applications such as web browsers. If this server already has the IP address stored in the cache(memory) it will simply return it. Otherwise, it’ll be responsible for making additional requests, by sending a query to the root DNS server.

The root server (root nameserver) - the root server doesn’t have information about the hostname but it has information on where to find it so it will redirect the requests to the appropriate TLD server.

The TLD ( Top Level Domain) server - it's the next step in the search for a specific IP address, and it hosts the last portion of a hostname. In google.com, the TLD server is “com”. The request is then sent to the authoritative nameserver.

The Authoritative nameserver - it's the last stop in the nameserver query. If this server has access to the requested record, it will return the IP address for the requested hostname back to the DNS recursor that made the initial request — otherwise, it will send an error saying the record couldn’t be found.

Diagram illustrating all the steps involved in DNS resolution. Source:https://www.catchpoint.com/blog/domain-name-to-ip-address

Internet Protocol Suite

Once your computer has the IP address, it establishes a connection to the Google server using the Transmission Control Protocol/Internet Protocol (TCP/IP). This is the set of rules and protocols that govern how data is sent over the internet or local network, such as texts, videos, mp3 etc. It is like the operator of the phone call, ensuring that data is broken up into small packets and sent to the correct destination. Once the TCP connection is established, it is time to begin transferring data! But not so fast, as websites like Google receives a large number of requests at any given time, and it would be overwhelming for one computer to handle all of them. Therefore, the initial connection is usually made to a load balancer. So does that mean the IP address sent to the browser is that of the load balancer? Yes, you're right!

Load Balancer

A load balancer, like HAProxy, is a piece of software or hardware that distributes incoming traffic among multiple servers. This is done to ensure that no single server is overwhelmed with too much traffic and that the website remains available and responsive to users. It is similar to a traffic cop directing cars to different lanes of the highway to prevent any one server from becoming overloaded. But can anyone access it? Let’s ask the firewall…

Firewall

A firewall is a software or hardware device that blocks unauthorized access to or from private networks. It acts as a barrier between a secured internal network and the most vulnerable network, e.g. the internet. It's similar to a bouncer at the front door of your friend's house. The firewall checks each packet to make sure it is safe and allowed to pass through, blocking anything that looks suspicious. The decision to whether to allow the traffic or not is based on a custom set of security rules, such as restrictions on the IP address of the request, the number and frequency of the requests made, or the type of content being requested. A load balancer acts as a firewall. However, you can install separate firewalls for extra security and protection and avoid SPOF (single point of failure).

SSL/TLS

Once the packets have passed through the firewall, they reach the Secure Sockets Layer (SSL)/Transport Layer Security (TLS) encryption. SSL is the predecessor of TLS (Transport Layer Security) and it's used to encrypt the data being sent. This is what gives you the "https" in the URL and the little lock icon in your browser. It is like a password-protected room inside your friend's house, ensuring that all data passed between the browser and the server is secure and private. This is done using an SSL certificate, a private key and a public key. The load balancer sends its SSL certificate alongside the public key to the browser. The browser verifies that the certificate is valid and it's from a trusted Certificate Authority(CA) provider. It then uses the public key to encrypt(scramble) subsequent requests so that if an attacker intercepts the packets in transit, he/she won't be able to see the original content. The data can only be decrypted (unscrambled) with the private key. This is based on a mathematical principle and we don't dive into that here. SSL is particularly useful when exchanging credit card information or other private sensitive information as it makes it impossible for hackers to steal your information. How do you know if a website is using an SSL certificate? Easy, just look at the URL and if you see that padlock and https:// instead of HTTP, then your privacy is safe. Usually, a website with an SSL certificate will always redirect you a the more secure HTTPS version when you try to access it using the HTTP protocol.

Back to the load balancer, so now the load balancer has established a secure connection to the client, the next thing it does is route the request to the appropriate web server. Now, depending on the configuration of the load balancer, it can either send this request to the appropriate web server using HTTP (without encryption) or HTTPS(with encryption) by restarting the process of establishing a secure connection with the web server this time acting as the client to the web server. Usually, if both computers are on the same network, the connection can be sent without encryption because it's much "safer" to transmit now.

Web Server

A web server is computer software and the underlying hardware that accepts requests via HTTP/HTTPS and serves static content, like simple HTML pages, images or plain text files, in other words, content that doesn’t change. It's like the receptionist at your friend's house. It's responsible for interpreting the packets of data and returning the appropriate HTML, CSS, and JavaScript files to the browser. However, most sites don’t only have static content. Quite the opposite most web apps depend on dynamic content, content that changes. That means, for example, it’s possible to interact with the website, save the information on it, log in with a username and a password and so on. To achieve this, we need an application server!

Application Server

An application server is a software program responsible for operating applications, communicating with database servers, managing user information, and more. It's like the personal assistant of your friend. It processes the incoming requests, interacts with the database, and sends back the information in a format that the web server can use. So what's a database?

Database

A database is a collection of data, and the database server is the program that interacts with the database and retrieves, adds, and modifies data in it. It's like the memory of your friend. It's where all the important information is stored. This can include user accounts, product information, blog posts, and more. The application server retrieves the necessary information from the database and sends it back to the webserver to be displayed on the page.

So in summary, the application server is responsible for interacting with the application code and generating web pages, interacting with the database if there is a need to. It then sends this generated page to the web server, the web server usually has access to the static files(HTML, CSS, JavaScript) for the page and it retrieves these files and sends them to the client. The web server can perform things like link caching(storing files in memory for later use) to improve the performance of the application. It's worth noting that the web server can also act as a load balancer. Modern web servers like Apache and Nginx provide these functionalities.

The image below illustrates how the web server, application server and database work together.

Image illustrating how the web server, application server and database work together. Source: https://imgur.com/a/Jk4lHdn

Below is a visual summary of all the steps we have discussed so far. Enjoy!

A diagram was created to summarise the entire process of what happens when you type a URL in the browser and press enter.

Conclusion

And there you have it! All of the steps that happen behind the scenes when you type a simple URL into your browser. From the DNS request to the load balancer to the database, it's a complex dance of technology that all happens in the blink of an eye. The next time you type a URL into your browser, take a moment to appreciate all the hard work that goes into making the internet run smoothly. This is a high-level overview of all the steps involved, and if this post has helped in demystifying what happens under the hood just a little bit, then it has achieved its aim!

Happy coding!