What Is Load Balancing: Types, Algorithms and a Linux Example

Applies to: Nginx 1.26.3 and HAProxy 3.0.11 on Debian 13 (tested); the concepts apply to any web or network service

Load balancing means spreading incoming requests or connections across two or more servers, so that no single server is overloaded and the service keeps working when one of them fails. A load balancer sits in front of the servers (the "backends"), receives all traffic on one address, and passes each request to a healthy backend chosen by an algorithm such as round robin or least connections.

This article explains the types, the common algorithms and health checks, then shows a small working example with Nginx and HAProxy on Linux, with output from our lab.

Why use a load balancer?

• Capacity: two or more servers handle more traffic than one, and you can add servers as traffic grows.
• Availability: when a backend stops answering, the load balancer sends traffic to the others.
• Maintenance without downtime: take one backend out, update it, put it back.

A load balancer is not a backup, and it can become a single point of failure itself. To avoid that, run two load balancers that share a floating IP address, or use a managed load balancer.

Types of load balancing

Layer 4 (network or transport level)

A Layer 4 load balancer works with TCP or UDP connections. It looks at IP addresses and ports only and forwards the whole connection to a backend without reading its content. It suits databases, mail, game servers and any non-HTTP protocol. Nginx does this with its stream module (on Debian, package libnginx-mod-stream, not installed by default), HAProxy with mode tcp.

Layer 7 (application level)

A Layer 7 load balancer understands HTTP. It can route by domain name, URL path or cookie, terminate HTTPS, add headers such as X-Forwarded-For and retry a failed request on another server. Nginx upstream with proxy_pass and HAProxy mode http work at this level.

Other forms you will meet

• DNS round robin: several A records for one name. It spreads clients, but DNS does not check whether a server is up, and resolvers cache answers, so a dead server keeps receiving visitors.
• Windows Server Network Load Balancing (NLB): a Windows feature that joins servers into one cluster with a shared cluster IP. According to Microsoft, an NLB cluster supports up to 32 hosts, all on the same subnet, and Microsoft points to its Software Load Balancer (SLB) instead for SDN deployments and non-Windows workloads.
• Hardware appliances and cloud load balancers: the same ideas, sold as a box or a managed service.

Load balancing algorithms

• Round robin: each backend in turn. This is the Nginx default ("weighted round-robin", according to nginx.org) and HAProxy balance roundrobin.
• Weighted: a stronger server gets a larger share, for example weight=3 in Nginx (the default weight is 1).
• Least connections: the next request goes to the backend with the fewest active connections. Nginx least_conn, HAProxy balance leastconn. HAProxy's manual recommends it for long sessions (such as SQL or LDAP) rather than short HTTP requests.
• IP hash or source: the client IP decides the backend, so a visitor keeps landing on the same server. Nginx ip_hash uses the first three octets of an IPv4 address (the whole IPv6 address) as the key; HAProxy calls it balance source.
• Consistent hash: Nginx hash $request_uri consistent; maps a key (here the URL) to a server, and adding or removing a server remaps only a few keys. Useful in front of caches.

If your application stores login sessions or uploads on the local disk, plain round robin can send a visitor to a server that does not have their session. Either keep sessions and files in shared storage (a database, Redis or a shared file system) or use IP hash.

Health checks

• Passive checks (open source Nginx): Nginx notices failures on real requests. With the defaults max_fails=1 and fail_timeout=10s, one failed attempt marks a server unavailable for 10 seconds.
• Active checks (HAProxy check): the load balancer tests every backend on a timer, even without traffic. HAProxy's defaults are a check every 2000 ms (inter), down after 3 failures (fall), up again after 2 successes (rise). In Nginx, periodic active health checks are part of the commercial subscription, according to nginx.org.

Example: an Nginx load balancer on Linux

1. Install Nginx on the load balancer server (Debian or Ubuntu: apt install nginx).
2. Create /etc/nginx/conf.d/lb.conf, with your backends' private IP addresses in place of the examples:

   upstream app_backend {
       server 10.0.0.11:80;
       server 10.0.0.12:80;
   }
   server {
       listen 80;
       location / {
           proxy_pass http://app_backend;
           proxy_set_header Host $host;
           proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
       }
   }

3. Test the configuration and reload:

   nginx -t && systemctl reload nginx

4. Send a few requests and watch them alternate. In our lab (one Debian 13 container, with two small Nginx sites on 127.0.0.1:8081 and 127.0.0.1:8082 as the backends, each answering with its name), six requests gave:

   backend 1
   backend 2
   backend 1
   backend 2
   backend 1
   backend 2

Lab results for the other settings:

• weight=3 on backend 1: 8 requests went 6 to backend 1 and 2 to backend 2.
• ip_hash; in the upstream block: all 6 requests from the same client went to backend 1.
• Backend 2 stopped: all 6 requests still returned HTTP 200 from backend 1, because Nginx retried the failed request on the next server (proxy_next_upstream error timeout is the default). The error log recorded the failure once:

  [error] 3447#3447: *76 connect() failed (111: Connection refused) while connecting to upstream, client: 127.0.0.1, server: , request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8082/", host: "127.0.0.1:8080"

• Backend 2 started again: after the 10-second fail_timeout, requests alternated again.
• Both backends stopped: the client got HTTP 502 (Bad Gateway).

Nginx only retries when nothing has been sent to the client yet, and it does not retry POST or PATCH requests that already reached a backend unless you add non_idempotent.

The same setup in HAProxy, with active health checks

Install HAProxy (apt install haproxy) and add these sections to the end of /etc/haproxy/haproxy.cfg. On Debian 13 the packaged file already has global and defaults sections with mode http and timeouts.

frontend web
    bind :80
    default_backend app

backend app
    balance roundrobin
    option httpchk GET /
    server app1 10.0.0.11:80 check inter 2s fall 3 rise 2
    server app2 10.0.0.12:80 check inter 2s fall 3 rise 2

Check the file with haproxy -c -V -f /etc/haproxy/haproxy.cfg (HAProxy 3.0 prints Configuration file is valid with -V; without it, a valid file prints nothing). When we stopped backend 2 in the lab, HAProxy logged:

Server app/app2 is DOWN, reason: Layer4 connection problem, info: "Connection refused", check duration: 0ms. 1 active and 0 backup servers left. 0 sessions active, 0 requeued, 0 remaining in queue.

and, once it was back:

Server app/app2 is UP, reason: Layer7 check passed, code: 200, check duration: 0ms. 2 active and 0 backup servers online. 0 sessions requeued, 0 total in queue.

Before you go live

• Every backend runs the same code and configuration, and talks to the same database.
• Sessions and uploaded files are shared, or you use IP hash.
• The application reads the real visitor IP from X-Forwarded-For, or every visitor appears to come from the load balancer.
• HTTPS certificates live on the load balancer if it terminates TLS.
• Backends accept web traffic only from the load balancer (firewall), so nobody bypasses it.

Related articles: Hosting multiple websites with containers and HAProxy, Nginx as a reverse proxy for Apache, Nginx as a reverse proxy with SSL, Apache as a reverse proxy (mod_proxy).

Ucartz KVM VPS and dedicated servers come with full root access for a setup like this, and our engineers can build it for you if you prefer.

Official documentation: Nginx upstream module, HAProxy configuration manual, Microsoft: Network Load Balancing.