Traffic guidance: Load balancing decryption of online world

Large multi-site Internet systems, including content distribution network (CDN), and cloud service providers, with some ways to balance the traffic of the visit. This article tells us that common flow balance design, including their technical means and pros and cons.

Early cloud computing service provider, you can provide a single customer web server, assign an IP address, then configure a DNS record to configure a DNS record to point to this IP address with a domain name for man, and then pass the IP address through the boundary gateway protocol (BGP) ) Announced, BGP is a standard way to exchange routing information between different networks.

This is not load balancing, but the traffic distribution can be performed in redundant multiple network paths, and the network technology can be utilized to get the traffic to the unavailable network, thereby increasing availability (also causing asymmetric routing phenomena) .

Simple DNS load balancing

As the traffic from customers becomes large, the boss hopes that the service is highly available. You launch the second web server, it has its own independent public IP address, then you update the DNS record, introduce the user traffic on the two servers (inner heart hopes to provide services). This has not been problematic before one of the servers fails. Suppose you can monitor the failure quickly, update the DNS configuration (manually update or via software) to delete the resolution to the faulty machine.

Unfortunately, because DNS records are cached, approximately half of the requests will fail before the client cache and the cache that they depend on the DNS server fail. DNS records have a few minutes or longer lifecycle (TTL), so this approach will have a serious impact on system availability.

Worse, some clients will completely ignore TTL, so some requests will continue to be booted to your faulty machine. It is not a good way to set up a short TTL, because this means a higher DNS service load, and there is a longer access delay because the client wants to do more DNS queries. If DNS service is not available for some reason, the setting shorter TTL will make the service’s access to a decline, because not so many clients have a cache of your website IP address.

Increase network load balancing

To solve the above problems, you can add a pair of mutually redundant four-layer (L4) network load balancer, configure the same virtual IP address (VIP). The equalizer can be hardware or software like Haproxy. DNS records of domain name points to VIP and no longer assume load balancing.

Four-layer load balancer can balance users and two web servers

The four-layer equalizer leads network traffic to the backend server. This is usually done based on the five-way group of IP packets, and the five-tuple includes: source address, source port, destination address, destination port, protocol, protocol, protocol, such as TCP or UDP). This method is fast and efficient (also maintaining the basic attribute of TCP), and does not require equalizer to maintain the state of each connection. (More information, please read the Maglev papers published by Google, this paper discusses the details of the four-layer software load balancer.)

Four-layer equalizer can do health checkup on backend services, only distribute traffic on a healthy machine. Unlike the use of DNS to do load balancing, when a backend web service is faulty, it can quickly reset the traffic to other machines, although the existing connection of the faulty machine will be reset.

When the capacity of the backend server is not, the four-layer equalizer can be distributed according to weight. It provides powerful capabilities and flexibility for operators, and hardware cost is relatively small.

Expand to multiple sites

The system scale continues to grow. Your customer wants to use the service, even when the data center is faulty. So you build a new data center, and independently deploy a service and four-layer load balancer cluster, still use the same VIP. The setting of DNS is unchanged.

The edge routers of the two sites have announced their address space, including the VIP address. Requests to this VIP may reach any site, depending on how the network between users and systems is connected, and how the routing policies of each network are configured. This is the preference. Most of this mechanism can work well. If a site is wrong, you can stop the VIP address via the BGP, and the customer’s request will move quickly to another site.

Multiple sites use the preference service

This setting has some problems. The biggest problem is that it cannot control which site is to flow, or limit the traffic of a certain site. There is also a clear way to transfer the user’s request to the nearest site (in order to reduce network delay), but the network protocol and routing circuit configuration should be able to route the user to the nearest site in most cases.

Control inbound request in multi-site system

In order to maintain stability, it is necessary to control the flow rate of each site. To achieve this control, you can assign different VIP addresses to each site, and then use simple or weightless DNS polling to do load balancing. Multi-site service, each site uses a primary VIP, and another site is used as a backup. Based on the DNS that can sense geographic locations.

There are two problems now.

First, use DNS balance means there will be a cached record, if you want to redirect traffic, you will be troublesome.

Second, users can make a new DNS query each time, it may be anywhere, may not be the nearest. If your service is running in a wide range of sites distributed, users feel obvious changes, depending on how large network delays between the user and the service of the service.

Let each site configure all the VIP addresses of all other sites and declare go out (so there will be a fault site), which can solve the first problem. There are some tips on the network, such as the backup site declare route, unlike the primary site to use such specific destination addresses, so that the primary site of each VIP will provide services as long as it is available. This is implemented by BGP, so we should see that traffic will be shifted within one or two minutes after the BGP update.

Even if the site is healthy and has a healthy and service capabilities, the user is really accessible, but there is no good solution for this site. Many large Internet services use DNS to return different resolution results to users of different geographies, and have certain results. However, because the structure of the network address is independent, an address segment may also change the location (for example, when a company re-planning the network), and many users may use the same DNS cache server. So this solution has a certain complexity and is easily error.

Increase seven layers load balancing

After a while, your customers start to have more advanced features.

Although four-layer load balancing can efficiently distribute traffic between multiple web servers, they only operate for source addresses, destination addresses, protocols, and ports. The content requested is unknown, so many advanced features are The four-layer load balance is not realized. The seven layers (L7) load balancing the content and structure of the request, so you can do more things.

Seven-layer load balancing can achieve cache, speed limit, error injection, and can sense the cost of the request when load balancing (some requests require more time deal with more time.).

Seven-layer load balancing can also distribute traffic based on request (such as http cookies), which can end the SSL connection, and help defense the application layer’s denial service (DOS) attack. The shortcomings of large-scale L7 load balancing are cost-processing requests require more computing, and each active request takes some system resources. Run the L4 equalizer cluster in front of one or more L7 equalizers, which helps the extension scale.

in conclusion

Load balancing is a complex puzzle. In addition to the strategies described above, there are different load balancing algorithms to achieve high availability technology, client load balancing technology, and recently lifted service networks, etc.

The core load balancing mode continues to develop with the development of cloud computing, and this technology will continue to develop with large Web service providers to make load balancing technologies more controllable and flexible.