Server Load Balancer:Use ALB connection draining for graceful service shutdown

Scenario 1: Removing a backend server

The following figure shows the state transition of ECS01 after it is removed from a server group where connection draining is enabled.

Scenario 2: Health check failure

WebSocket session interruption behavior

Step 1: Enable connection draining

This step describes how to enable connection draining for the server group from the prerequisites. You can also enable this feature when creating a server group.

1. Log on to the ALB console.

2. In the top navigation bar, select the region where the server group is deployed.

3. In the left-side navigation pane, choose ALB > Server Groups.

4. On the Server Groups page, find the target server group and click its ID.

5. On the Details tab, find the Basic Information section and click Modify Basic Information.

6. In the Modify Basic Information dialog box, click Advanced Settings, and then enable Connection Draining.

7. Set Timeout Period to 300 seconds and click Save.

Step 2: Verify the results

Configure the server

1. Log on to the ECS01 instance. For more information, see Connection methods.

2. Run the following commands to create a directory named WebSocket and navigate to it.

   mkdir WebSocket
   cd WebSocket

3. Run the following commands to install the required dependencies.

   pip install tornado
   pip install websocket-client

4. Run the following command to edit the server.py configuration file.

   vim server.py

   1. Press the i key to enter edit mode and add the following code to start a WebSocket service.

      #!/usr/bin/env python3
      # encoding=utf-8
      
      import tornado.websocket
      import tornado.ioloop
      import tornado.web
      from datetime import datetime
      
      
      # WebSocket handler
      class WebSocketHandler(tornado.websocket.WebSocketHandler):
          def open(self):
              current_time = datetime.now()
              formatted_time = current_time.strftime("%Y-%m-%d %H:%M:%S")
              print("Time:", formatted_time, "WebSocket connection opened")
      
          def on_message(self, message):
              current_time = datetime.now()
              formatted_time = current_time.strftime("%Y-%m-%d %H:%M:%S")
              print("Time:", formatted_time, "Received message:", message)
              self.write_message("Server received your message: " + message)
      
          def on_close(self):
              current_time = datetime.now()
              formatted_time = current_time.strftime("%Y-%m-%d %H:%M:%S")
              print("Time:", formatted_time, "WebSocket connection closed")
      
      # Routing
      application = tornado.web.Application([
          (r"/websocket", WebSocketHandler),
      ])
      
      if __name__ == "__main__":
          print("WebSocket Server Start on 8080 ...")
          application.listen(8080)
          tornado.ioloop.IOLoop.current().start()
      

   2. After you modify the configuration, press the Esc key, enter:wq, and press the Enter key to save and close the file.

5. Navigate to the directory where server.py is located and run the following command to start the WebSocket server.

   python3 server.py

   The following response indicates that the WebSocket backend service has started.

   Websocket Server Start on 8080 ...

Add backend server ECS01 to the server group

1. Log on to the ALB console.

2. In the top navigation bar, select the region where the server group is deployed.

3. In the left-side navigation pane, choose ALB > Server Groups.

4. On the Server Groups page, find the target server group and click Modify Backend Server in the Actions column.

5. On the Backend Servers tab, click Add Backend Server. In the Add Backend Server panel, select ECS01 and click Next.

6. In the Ports/Weights step, select the ECS01 instance, set the port to 8080, and then click OK.

Configure the client

1. Log on to the client and open a command-line interface. Run the following commands to create a directory named WebSocket and navigate to it.

   mkdir WebSocket
   cd WebSocket

2. Run the following command to install the required dependency.

   pip install websocket-client

3. Run the following command to edit the client.py file.

   vim client.py

   1. Press the i key to enter edit mode and add the following code for a WebSocket client to access the service.

      #!/usr/bin/env python3
      # encoding=utf-8
      
      import websocket
      import time
      from datetime import datetime
      
      def on_message(ws, message):
          print("Received message from server:", message)
      
      if __name__ == "__main__":
          ws = websocket.WebSocket()
          ws.connect("ws://<domain_name>:80/websocket") # Replace <domain_name> with your actual domain name.
          print("WebSocket connection opened")
          try:
              while True:
                  current_time = datetime.now()
                  formatted_time = current_time.strftime("%Y-%m-%d %H:%M:%S")
                  print("Sending time:", formatted_time)
                  ws.send("Hello, Server!")
                  result = ws.recv()
                  on_message(ws, result)
                  time.sleep(1)
          except Exception:
              print("WebSocket connection closed")
      

   2. After you modify the configuration, press the Esc key, enter:wq, and press the Enter key to save and close the file.

4. Navigate to the directory where client.py is located and run the following command to access the server ECS01.

   python3 client.py

   The following response indicates that the access is successful.

   WebSocket connection opened
   Sending time: 2024-04-28 17:00:53
   Received message from server: Server received your message: Hello, Server!
   Sending time: 2024-04-28 17:00:54
   Received message from server: Server received your message: Hello, Server!

   The server ECS01 returns the following response.

   WebSocket Server Start on 8080 ...
   Time: 2024-04-28 17:00:53 WebSocket connection opened
   Time: 2024-04-28 17:00:53 Received message: Hello, Server!
   Time: 2024-04-28 17:00:54 Received message: Hello, Server!

Remove the backend server

1. Log on to the ALB console.

2. In the top navigation bar, select the region where the server group is deployed.

3. In the left-side navigation pane, choose ALB > Server Groups.

4. Find the target server group and click its ID.

5. Click the Backend Servers tab, find the target backend server ECS01, and then click Remove in the Actions column.

6. In the Remove dialog box, click OK.

Wait for connection draining to complete

The connection draining timeout is set to 300 seconds in this topic. The test results show that ALB terminates the connection approximately 300 seconds after ECS01 is removed.

• The client returns the following response.

  Sending time: 2024-04-28 17:06:23
  Received message from server: Server received your message: Hello, Server!
  Sending time: 2024-04-28 17:06:24
  WebSocket connection closed

• The ECS01 server instance returns the following response.

  Time: 2024-04-28 17:06:22 Received message: Hello, Server!
  Time: 2024-04-28 17:06:23 Received message: Hello, Server!
  Time: 2024-04-28 17:06:23 WebSocket connection closed

HTTP session interruption behavior

In HTTP scenarios, the response received by the client depends on the connection draining timeout, connection request timeout, and the backend server processing time.

• If connection draining timeout < backend server processing time, the connection is interrupted before ECS01 finishes sending the response. In this case, the client receives a 500-level error response.

• If backend server processing time > connection request timeout, the request to ECS01 times out. In this case, the client receives a 504 error response.

This topic uses a connection draining timeout of 15 seconds and a backend server processing time of 30 seconds. In this scenario, the connection to ECS01 is interrupted before the response is sent, and the client receives a 500-level error response.

Step 1: Enable connection draining

This step describes how to enable connection draining for the server group from the prerequisites. You can also enable this feature when creating a server group.

1. Log on to the ALB console.

2. In the top navigation bar, select the region where the server group is deployed.

3. In the left-side navigation pane, choose ALB > Server Groups.

4. On the Server Groups page, find the target server group and click its ID.

5. On the Details tab, find the Basic Information section and click Modify Basic Information.

6. In the Modify Basic Information dialog box, click Advanced Settings, and then enable Connection Draining.

7. Set Timeout Period to 15 seconds and click Save.

Step 2: Set the ALB connection request timeout

1. Log on to the ALB console.

2. In the top navigation bar, select the region where the ALB instance is deployed.

3. On the Instances page, find the target ALB instance and click its ID.

4. Click the Listener tab, find the target HTTP listener, and then click the listener ID.

5. In the Basic Information area, click Modify Listener.

6. In the Modify Listener dialog box, click Modify next to Advanced Settings.

7. Set the Connection Request Timeout to 60 seconds (the default value), and then click Save.

Step 3: Configure domain name resolution

In production environments, we recommend that you use a custom domain name and point it to the ALB instance domain name by creating a CNAME record.

1. In the left-side navigation pane, choose ALB > Instances.

2. On the Instances page, copy the domain name of the ALB instance.

3. Perform the following steps to create a CNAME record:

   Note

   If your domain name is not registered by using Alibaba Cloud Domains, you must add your domain name to Alibaba Cloud DNS before you can configure a DNS record. For more information, see Manage domain names.

   1. Log on to the Alibaba Cloud DNS console.

   2. On the Authoritative DNS Resolution page, find your domain name and click Settings in the Operations column.

   3. On the Settings tab of the domain name details page, click Add Record.

   4. In the Add Record panel, configure the parameters and click OK. The following table describes the parameters.

      Parameter	Description
      Record Type	Select CNAME from the drop-down list.
      Hostname	Enter the prefix of the domain name. In this example, @ is entered. Note If you use a root domain name, enter @.
      Query Source	Select Default.
      Record Value	Enter the CNAME, which is the domain name of the ALB instance.
      TTL	Select a time-to-live (TTL) value for the CNAME record to be cached on the DNS server. In this example, the default value is used.

Step 4: Verify the results

Log in and configure the server

1. Log on to the ECS01 instance. For more information, see Select a method to connect to an ECS instance.

2. Run the following command to create an HTTP folder and go to the HTTP directory.

   mkdir http
   cd http

3. Run the following command to edit the http_server.py configuration file.

   vim http_server.py

   1. Press the i key to enter edit mode. Then, add the following content to start an HTTP server.

      #!/usr/bin/env python3
      # encoding=utf-8
      
      from http.server import SimpleHTTPRequestHandler, HTTPServer
      from datetime import datetime
      import time
      
      class DelayedHTTPRequestHandler(SimpleHTTPRequestHandler):
          def do_GET(self):
              current_time = datetime.now()
              formatted_time = current_time.strftime("%Y-%m-%d %H:%M:%S")
              print("Time:", formatted_time, "GET request received. Responding after a 30 second delay....")
              time.sleep(30) # Use the time.sleep function to simulate the processing time of the backend server.
              SimpleHTTPRequestHandler.do_GET(self)
      
      PORT = 8080
      server = HTTPServer(("", PORT), DelayedHTTPRequestHandler)
      print(f"Serving HTTP on 0.0.0.0 port {PORT} (http://0.0.0.0:{PORT}/) ...")
      server.serve_forever()
      

   2. After you finish editing, press the Esc key, enter :wq, and press Enter to save and close the file.

4. Go to the directory where http_server.py is located. Run the following command to start the HTTP server.

   python3 http_server.py

   If the following response is returned, the HTTP backend server is started.

   Serving HTTP on 0.0.0.0 port 8080 (http://0.0.0.0:8080/) ...

Add backend server ECS01 to the server group

1. Log on to the ALB console.

2. In the top navigation bar, select the region where the server group is deployed.

3. On the Server Groups page, find the target server group and click Modify Backend Server in the Actions column.

4. On the Backend Servers tab, click Add Backend Server. In the Add Backend Server panel, select ECS01 and click Next.

5. In the Ports/Weights wizard, select the ECS01 instance, set the port to 8080, and then click OK.

Log in and configure the client

1. Log on to the client. Open a command-line window. Run the following command to access the backend server ECS01.

   curl http://<domain_name>:80/ -v

   The following response indicates that the ALB can access the backend service.

   * About to connect() to www.example.com port 80 (#0)
   *   Trying 10.X.X.225...
   * Connected to www.example.com (10.X.X.225) port 80 (#0)
   > GET / HTTP/1.1
   > User-Agent: curl/7.29.0
   > Host: www.example.com
   > Accept: */*

   The server receives the following response.

   Serving HTTP on 0.0.0.0 port 8080 (http://0.0.0.0:8080/) ...
   Time: 2024-02-07 13:57:33 Received a GET request. Responding after a 30-second delay....

Remove backend servers

1. Log on to the ALB console.

2. In the top navigation bar, select the region where the server group is deployed.

3. In the left-side navigation pane, choose ALB > Server Groups.

4. Find the target server group and click the server group ID.

5. Click the Backend Servers tab, find the target backend server ECS01, and then in the Actions column, click Remove.

6. In the Remove dialog box, click OK.

Waiting for connection draining

The test results show that when the graceful connection timeout set for the ALB < the backend server processing time, the client receives a 500 error response.

* About to connect() to www.example.com port 80 (#0)
*   Trying 10.X.X.224...
* Connected to www.example.com (10.XX.XX.224) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: www.example.com
> Accept: */*
> 
< HTTP/1.1 500 Internal Server Error
< Date: Wed, 07 Feb 2024 06:02:24 GMT
< Content-Type: text/html
< Content-Length: 186
< Connection: close
< Via: HTTP/1.1 SLB.87
< 
<html>
<head><title>500 Internal Server Error</title></head>
<body bgcolor="white">
<center><h1>500 Internal Server Error</h1></center>
<hr><center>nginx</center>
</body>
</html>
* Closing connection 0