Run Docker behind the Nginx reverse proxy

This article is about running Docker behind the Nginx reverse proxy. [Editor's Note] Do you want Docker to have a public gateway that can support multiple tenants and the security authentication we expect? Please see that the author recompiled Nginx to support Docker's own protocol as well as supporting Web sockets to achieve this requirement.

A few weeks ago, I wanted to try to see how Docker works in a cloud/shared host like environment. At that time, Docker just released version 1.4, which brought additional security and identity authentication features. In addition, Docker machine can automatically create and run a remote Docker instance.

The shared host environment is usually based on a common gateway, which can be used to manage a large number of incoming and outgoing traffic, including FTP and SSH. The largest part of the environment (no different from icebergs) will be "hidden" in the private network behind these gateways.

Therefore, my question is, is there a way for Docker to have a gateway with similar behavior, including multi tenant support and all the functions you expect?

Facts have proved that there are.

Docker binary files play three roles:

Docker command line ->makes Docker easy to use and very simple

Docker Daemon ->silently doing hard work behind the scenes

Docker init ->early container settings behind the scenes

The command line and Docker Daemon communicate mainly based on the HTTP protocol. I say "mainly" because several APIs "block" connections, especially the container/attach command, which is also called "forward my container's console".

Common articles on the Internet recommend setting up a Nginx reverse proxy and adding basic identity authentication to ensure security.

Unfortunately, this approach has two disadvantages:

The existing Docker client cannot communicate with HTTP basic authentication

When Docker intercepts the connection, the existing Nginx will lose the connection completely

Regarding authentication, I recommend using Docker's TLS authentication, because they support out of the box use. At the same time, using Lua magic, we can use them as "public keys" to achieve proper balance.

How should we deal with the second problem (Nginx lost connection)?

If you confirm the interception method, it becomes much simpler: HTTP can be seen as a "half duplex" protocol, that is, traffic can only flow in one direction at the same time: the client sends a request to the server (one-way), and then the server responds to the request (one-way). When the Docker attach is executed, the Docker can use the original "full duplex" TCP connection, and any end can initiate a request at any time. This is why the reverse proxy will be lost: because the HTTP protocol requires a confirmation response after the request (what the author said in this paragraph is somewhat vague, and readers with understanding can communicate with us in the comments).

Interestingly, there is another mainstream protocol that can do this. It turns out that this standard protocol is so popular that it was accepted by Nginx several years ago. I call it WebSocket.

Therefore, in essence, the idea is to tell Nginx how to deal with the Docker's own protocol just like Websocket. This is the patch:

--- a/src/http/ngx_http_upstream.c Tue Nov 04 19:56:23 2014 +0900
+++ b/src/http/ngx_http_upstream.c Sat Nov 15 16:21:58 2014 +0100
@@ -89,6 +89,8 @@
ngx_table_elt_t *h, ngx_uint_t offset);
static ngx_int_t ngx_http_upstream_process_content_length(ngx_http_request_t *r,
ngx_table_elt_t *h, ngx_uint_t offset);
+static ngx_int_t ngx_http_upstream_process_content_type(ngx_http_request_t *r,
+ ngx_table_elt_t *h, ngx_uint_t offset);
static ngx_int_t ngx_http_upstream_process_last_modified(ngx_http_request_t *r,
ngx_table_elt_t *h, ngx_uint_t offset);
static ngx_int_t ngx_http_upstream_process_set_cookie(ngx_http_request_t *r,
@@ -175,7 +177,7 @@
ngx_http_upstream_copy_header_line, 0, 0 },

{ ngx_string("Content-Type"),
- ngx_http_upstream_process_header_line,
+ ngx_http_upstream_process_content_type,
offsetof(ngx_http_upstream_headers_in_t, content_type),
ngx_http_upstream_copy_content_type, 0, 1 },

@@ -2716,6 +2718,7 @@
u->write_event_handler = ngx_http_upstream_upgraded_write_upstream;
r->read_event_handler = ngx_http_upstream_upgraded_read_downstream;
r->write_event_handler = ngx_http_upstream_upgraded_write_downstream;
+ u->headers_in.chunked = 0;

if (clcf->tcp_nodelay) {
tcp_nodelay = 1;
@@ -3849,6 +3852,25 @@

static ngx_int_t
+ngx_http_upstream_process_content_type(ngx_http_request_t *r, ngx_table_elt_t *h,
+ ngx_uint_t offset)
+{
+ ngx_int_t ret = ngx_http_upstream_process_header_line(r, h, offset);
+ if (ret != NGX_OK) {
+ return ret;
+ }
+
+ // is docker header ?
+ if (ngx_strstrn(h->value.data,
+ "application/vnd.docker.raw-stream", 34 - 1) != NULL) {
+ r->upstream->upgrade = 1;
+ }
+
+ return NGX_OK;
+}
+
+
+static ngx_int_t
ngx_http_upstream_process_last_modified(ngx_http_request_t *r,
ngx_table_elt_t *h, ngx_uint_t offset)
{

The only remaining step is to configure the reverse proxy. By the way, this is my nginx test configuration file nginx. conf:

worker_processes 1;

events {
worker_connections 1024;
}

http {
include mime.types;
default_type application/octet-stream;

sendfile on;

keepalive_timeout 65;

server {
listen 9000;

location / {
proxy_buffering off;
proxy_pass http://localhost:8080;
}
}
}

You just need to use the following command to run Docker on port 8080 or add your parameters to/etc/default/locker:

docker -d -H tcp://localhost:8080

We're done!

Final question

Although hacking this, I noticed that all Nginx need to switch to the appropriate HTTP header for the Websocket protocol:

Request

Connection: Upgrade Upgrade: websocket

Response

HTTP/1.1 101 Upgraded Connection: Upgrade Upgrade: websocket

Therefore, I think another method is to inject appropriate headers into the Docker protocol.

Related Articles

Explore More Special Offers

  1. Short Message Service(SMS) & Mail Service

    50,000 email package starts as low as USD 1.99, 120 short messages start at only USD 1.00

phone Contact Us