What actually happens when half the internet decides to visit the same place at 8:00 p.m.? Well, from a user’s perspective, nothing should happen. You click, the page opens, and you get what you came for.
But behind the scenes, the website may be dealing with an extremely abnormal situation, and if the server is not ready to handle all the traffic, it leads to one thing. CRASH. This is a problem that many organizations face, and it has only one solution: to beef up the server.
Sports events, concerts, new game releases, and other time-sensitive events can bring a lot of traffic to one website. These large spikes happen maybe once or twice a year, when the events are scheduled, so in other words, companies don’t really need a beefy server since most of the time they have low traffic.
But a single powerful server can only handle so much. Eventually, requests begin waiting, pages become slow, and if the pressure continues, the server may give up responding altogether.
That’s where load balancing comes in, and we’re going to learn more about it.
Think of it as Traffic Control for Website Requests
To understand traffic load better, imagine a supermarket. There is a building with capacity to fit 200 shoppers, so what if 2,000 customers show up at the same time?
Having one enormous checkout till would not solve the problem quite well. The sensible solution is to open multiple tilts and distribute customers among them. That’s exactly what a load balancer performs for websites.
Instead of allowing every visitor to contact one application server directly, the load balancer sits in front of a group of servers and redirects traffic to the server that has the lowest load.
Amazon Web Services describes Elastic Load Balancing in almost exactly these terms: incoming traffic is automatically distributed across multiple targets such as servers, containers, and IP addresses, potentially across several availability zones.
The visitor normally has no idea this happened.
They asked for live results.
One server may have answered their request while another visitor received the same page from a completely different machine.
To the public, it still looks like one website. Behind the curtain, the website may resemble a small army.
This often happens to horse racing websites, since big events like the Breeders’ Cup or the Kentucky Derby bring hundreds of thousands of people to news pages or betting websites. And nobody wants to wait 2 minutes for a website to load just to see the latest Breeders’ Cup odds.
Bettors want instant load times, which is why most betting companies like TwinSpires usually have load balancing set up before they even start their business.
The Requests Do Not Necessarily Get Divided Equally
The simplest approach is called round robin.
Server A receives a request.
Then Server B.
Then Server C.
Then back to Server A.
NGINX, one of the widely used technologies for handling web traffic, supports round robin along with methods such as least connections, where new traffic can be directed toward the server currently handling the fewest active connections.
That difference becomes important during major events because not every request creates the same amount of work.
Loading a simple page may be easy.
Searching a giant database, processing a purchase, or producing personalized information may keep a server busy for much longer.
A smarter load-balancing strategy can therefore consider things such as server capacity, connection count, geography, or the type of request rather than distributing everything with mathematical politeness.
Health Checks Stop One Broken Server Ruining Everybody’s Evening
Load balancing becomes especially valuable when something fails.
Suppose a website has six application servers and one suddenly crashes.
Without a load balancer, some visitors could continue being sent straight toward the dead machine and receive errors.
So, what’s the solution? Well, modern load balancers perform live health checks. They periodically test whether each backend is responding as it should. That’s what AWS and Google Cloud both use health checking for, so traffic can be directed only towards backends that bring back healthy signals.
If the machine begins failing those checks, the load balancer can temporarily remove it from circulation until the dust settles and services go back to functioning normally. And if it recovers on its own and returns healthy signals, it can be returned to service, and traffic will be forwarded to the server again.
Load Balancing Works Especially Well With Autoscaling
Of course, redistributing traffic does not help forever if every available server is already overloaded.
That is where autoscaling enters the picture.
Remember, cloud infrastructure can monitor demand and start additional computing resources only when traffic increases, just to maintain a normal level of functioning. Those new instances are registered behind the load balancer, which gives it more places to send incoming requests and disperse them evenly across servers.
AWS, for example, allows Elastic Load Balancing to work with Auto Scaling groups so new instances are automatically registered as capacity grows and removed when they are no longer needed. Load-balancer metrics such as request volume can also be used to trigger scaling decisions.
Think of it as a restaurant where additional kitchens can appear when the dinner rush arrives.
That would be extremely useful in the restaurant industry.
On the internet, it is reasonably normal.
The important distinction is that autoscaling creates more capacity while load balancing distributes work across that capacity. They solve related problems, but they are not the same thing.
Final Thoughts
Load balancing is crucial for organizations that receive surges of traffic. In today’s era, that doesn’t even have to be a big organization or a big sporting event. One viral video with millions of views on social media can bring a large audience to one server at the same time, causing an overload.
So, if you want to build something properly, think about load balancing before you launch your website, SAAS, or eCommerce store.
