Nginx Using Docker - s3/7
日本語記事準備中-シリーズ前後記事リンク追加予定
first post: Cheap Home LAN Playground Using Docker
In this third post of the series, I would like to setup a web server, Nginx, which I will use as a reverse proxy to enable access to different services by their DNS names.
Port 80 http access to service running on docker container
As we have seen in the previous post, it is very easy to run Jupyter Notebook service using Docker. I used port 8888 as it is the default port Jupyter Notebook uses, but as you can see below docker-compose.yml file, I could just use port 80 on my host machine to connect to port 8888 on the docker container.
services:
jupyter:
container_name: jupyter
image: jupyter/base-notebook:notebook-6.5.1
user: root
ports:
- "80:8888"
command: "start-notebook.sh --ServerApp.password='' --ServerApp.token='' --ip=0.0.0.0 --no-browser"
environment:
- "CHOWN_EXTRA=/home/jovyan"
- "CHOWN_EXTRA_OPTS=-R"
volumes:
- type: volume
source: jupyter_volume
target: /home/jovyan
volume:
nocopy: true
volumes:
jupyter_volume: {}
In this case, you can omit :8888 when accessing on your web browser as it automatically chooses to use port 80 for http access.
Mapping host port 80 to container port 8888 is ok and it gives you easier access using web browser. However, if you try to run more web services, like another one of Jupyter Notebook or anything, and you again try to map host port 80 to access those docker containers, you cannot do that. The port 80 on the host machine is already occupied and bound to port 8888 of the Jupyter Notebook container.
It is so easy to run multiple docker containers, yet you can only map port 80 on host machine to a single docker container. How can you try out different services without bringing up and down docker compose here and there?
Reverse proxy
You can setup a web server to receive request traffics that passes on the right traffic to right container.
Let's say I want to have my first jupyter container continue running, another Jupyter Notebook jupyter2 for something else, and maybe blog container using ghost. I can place a web server configured to pass on the http traffic to the right container based on the destination hostname in the http request the web server received.
Let me first configure the web server by creating a directory $HOME/mylan/rp as well as $HOME/mylan/rp/conf.d.
mkdir -p $HOME/mylan/rp/conf.d
cd $HOME/mylan/rp
Here is docker-compose.yml file.
services:
rp:
container_name: rp
image: nginx:1.23.2
ports:
- "80:80"
volumes:
- ./conf.d:/etc/nginx/conf.d
As specified in the yml file, I want to put configuration files here and load them on /etc/nginx/conf.d where all .conf files in the directory gets loaded by nginx.
I will create jupyter.conf at $HOME/mylan/rp/conf.d to have server handle http request for jupyter.mylan.local.
Note that the proxy is pointing to port 8888, so make sure the first Jupyter Notebook is also accessible on port 8888.
server {
listen 80;
server_name jupyter.mylan.local;
# docker resolver
resolver 127.0.0.11 valid=30s;
location / {
set $upstream 192.168.1.56:8888;
proxy_pass http://$upstream;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
}
}
Now I am ready to run this web server. Run this service using docker compose up -d. Once the service is up, I can access http://jupyter.mylan.local on web browser to access Jupyter Notebook.
❯ curl -v http://jupyter.mylan.local/
* Trying 192.168.1.56:80...
* Connected to jupyter.mylan.local (192.168.1.56) port 80 (#0)
> GET / HTTP/1.1
> Host: jupyter.mylan.local
> User-Agent: curl/7.79.1
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 301 Moved Permanently
< Server: nginx/1.23.2
Another Jupyter Notebook
Let me then add another configuration file under $HOME/mylan/rp/conf.d. I will just name it jupyter2.conf.
$ cat conf.d/jupyter2.conf
server {
listen 80;
server_name jupyter2.mylan.local;
# docker resolver
resolver 127.0.0.11 valid=30s;
location / {
set $upstream 192.168.1.56:8889;
proxy_pass http://$upstream;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
}
}
I can go ahead and restart rp by running docker compose restart. It works fine. Nginx loads the configuration. I can run nginx -T on the nginx server to confirm the configuration loaded, and that all is fine. And of course I have not yet added the records on DNS server, so the access will fail. Even if I have the DNS record added, since the second Jupyter Notebook is not yet running, access will still fail.
$ docker exec rp nginx -T
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
# configuration file /etc/nginx/nginx.conf:
user nginx;
worker_processes auto;
--- rest is omitted for brevity ---
# access will fail even if the name resolution is already there
❯ curl -v http://jupyter2.mylan.local/
* Trying 192.168.1.56:80...
* Connected to jupyter2.mylan.local (192.168.1.56) port 80 (#0)
> GET / HTTP/1.1
> Host: jupyter2.mylan.local
> User-Agent: curl/7.79.1
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 502 Bad Gateway
Now, let me go back to $HOME/mylan/jupyter/docker-compose.yml file to add the second service and start the second Jupyter Notebook. Here I will just make things simple and not mount any volume for data persistence. Whatever files or data generated inside the second Jupyter Notebook container will be gone when the container is shutdown.
services:
jupyter:
container_name: jupyter
image: jupyter/base-notebook:notebook-6.5.1
user: root
ports:
- "8888:8888"
command: "start-notebook.sh --ServerApp.password='' --ServerApp.token='' --ip=0.0.0.0 --no-browser"
environment:
- "CHOWN_EXTRA=/home/jovyan"
- "CHOWN_EXTRA_OPTS=-R"
volumes:
- type: volume
source: jupyter_volume
target: /home/jovyan
volume:
nocopy: true
jupyter2:
container_name: jupyter2
image: jupyter/base-notebook:notebook-6.5.1
ports:
- "8889:8888"
command: "start-notebook.sh --ServerApp.password='' --ServerApp.token='' --ip=0.0.0.0 --no-browser"
volumes:
jupyter_volume: {}
Now the first one can be accessed through 8888 on the host machine, and the second one through 8889.
$ docker compose up -d
[+] Running 2/2
⠿ Container jupyter2 Started 0.7s
⠿ Container jupyter Running 0.0s
$ docker compose ps
NAME COMMAND SERVICE STATUS PORTS
jupyter "tini -g -- start-no…" jupyter running (healthy) 0.0.0.0:8888->8888/tcp, :::8888->8888/tcp
jupyter2 "tini -g -- start-no…" jupyter2 running (healthy) 0.0.0.0:8889->8888/tcp, :::8889->8888/tcp
DNS records for the second Jupyter Notebook and Ghost
Let me also update DNS records, both jupyter2 and blog, in $HOME/mylan/dns/config/a-records.conf and then restart DNS container (docker compose restart at $HOME/mylan/dns directory).
$ cat dns/config/a-records.conf
# A Record
#local-data: "somecomputer.local. A 192.168.1.1"
local-data: "jupyter.mylan.local. A 192.168.1.56"
local-data: "jupyter2.mylan.local. A 192.168.1.56"
local-data: "blog.mylan.local. A 192.168.1.56"
# PTR Record
#local-data-ptr: "192.168.1.1 somecomputer.local."
local-data-ptr: "192.168.1.56 jupyter.mylan.local."
I can access the second one at http://jupyter2.mylan.local.
❯ curl -v http://jupyter2.mylan.local/
* Trying 192.168.1.56:80...
* Connected to jupyter2.mylan.local (192.168.1.55) port 80 (#0)
> GET / HTTP/1.1
> Host: jupyter2.mylan.local
> User-Agent: curl/7.79.1
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 302 Found
By the way, if you go back to $HOME/mylan/rp and run docker compose logs, you can see the access log nginx is logging.
Ghost blog
I will continue on to create the blog service by preparing a directory $HOME/mylan/ghost and necessary file.
Here is the docker-compose.yml file which is mostly the copy from the sample provided at the Docker Hub Ghost page.
Three things I have changed are:
- host port - 2368
- url - http://blog.mylan.local
- NODE_ENV: development line uncommented
services:
ghost:
image: ghost:4-alpine
restart: always
ports:
- 2368:2368
environment:
# see https://ghost.org/docs/config/#configuration-options
database__client: mysql
database__connection__host: db
database__connection__user: root
database__connection__password: example
database__connection__database: ghost
# this url value is just an example, and is likely wrong for your environment!
url: http://blog.mylan.local
# contrary to the default mentioned in the linked documentation, this image defaults to NODE_ENV=production (so development mode needs to be explicitly specified if desired)
NODE_ENV: development
db:
image: mysql:8.0
restart: always
environment:
MYSQL_ROOT_PASSWORD: example
Let me go ahead and start these.
$ docker compose up -d
[+] Running 22/22
⠿ db Pulled 17.6s
--- omitted ---
⠿ ghost Pulled 31.7s
--- omitted ---
[+] Running 3/3
⠿ Network ghost_default Created 0.2s
⠿ Container ghost-db-1 Started 1.6s
⠿ Container ghost-ghost-1 Started 1.5s
$ docker compose ps
NAME COMMAND SERVICE STATUS PORTS
ghost-db-1 "docker-entrypoint.s…" db running 3306/tcp, 33060/tcp
ghost-ghost-1 "docker-entrypoint.s…" ghost running 0.0.0.0:2368->2368/tcp, :::2368->2368/tcp
Let me add another server config file at $HOME/mylan/rp/conf.d/blog.conf. See the upstream is pointing to port 2368 which is the port ghost docker container is listening on.
$ cat rp/conf.d/blog.conf
server {
listen 80;
server_name blog.mylan.local;
# docker resolver
resolver 127.0.0.11 valid=30s;
location / {
set $upstream 192.168.1.56:2368;
proxy_pass http://$upstream;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
}
}
This is just the curl output as seen in the previous sections, but you will see awesome ghost blog page on your browser.
❯ curl -v http://blog.mylan.local/
* Trying 192.168.1.56:80...
* Connected to blog.mylan.local (192.168.1.56) port 80 (#0)
> GET / HTTP/1.1
> Host: blog.mylan.local
> User-Agent: curl/7.79.1
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
Some notes on ghost docker-compose.yml and docker network
As you have seen in the Jupyter Notebook docker-compose.yml file, you can include multiple services in one file. And here in this docker-compose.yml file, you can see the environment variable database__connection__host is set to db, which is the second service name running using the image mysql:8.0.
The ghost container is running and is configured to use the database named db. Between these two docker containers, ghost and db, they can lookup each other using service name. There is no ports section for db service to make it accessible on the host machine, but they are on the same docker network and can access each other.
services:
ghost:
image: ghost:4-alpine
restart: always
ports:
- 2368:2368
environment:
database__client: mysql
# here the database to use is set to "db"
# which is the mysql:8.0 database container
# defined in this same file
database__connection__host: db
database__connection__user: root
database__connection__password: example
database__connection__database: ghost
url: http://blog.mylan.local
NODE_ENV: development
db:
image: mysql:8.0
restart: always
environment:
MYSQL_ROOT_PASSWORD: example
# no ports section
# ports:
# - 1234:5678
Unlike other examples, the docker container names were not explicitly configured inside docker-compose.yml file, so they are ghost-db-1 and ghost-ghost-1. Of course you can add container_name line for these services to make them ghost and db if you'd like.
$ docker compose ps
NAME COMMAND SERVICE STATUS PORTS
ghost-db-1 "docker-entrypoint.s…" db running 3306/tcp, 33060/tcp
ghost-ghost-1 "docker-entrypoint.s…" ghost running 0.0.0.0:2368->2368/tcp, :::2368->2368/tcp
And a little bit more on docker network ...
As briefly touched upon in the last section of the previous post, docker container inspect {container_name} shows you tons of details about the container. If you look at the network configuration of ghost, db, and rp, you can see that the first two are on ghost_default network and ghost has 192.168.160.3/20 and db has 192.168.160.2/20, while rp is on rp_default network and has 192.168.144.2/20 IP address configured by docker system.
$ docker container inspect ghost-ghost-1 -f '{{ json .NetworkSettings.Networks }}' | python3 -m json.tool
{
"ghost_default": {
"IPAMConfig": null,
"Links": null,
"Aliases": [
"ghost-ghost-1",
"ghost",
"537d3c9e7019"
],
"NetworkID": "19471798da39dafa7200ad3dd763d2cbb3ffd9cb4d85301f973c9a10764294a5",
"EndpointID": "6b85e1569fa020bf9d7a2786281bee71146223f057c6723f4ee27ba602a3e82f",
"Gateway": "192.168.160.1",
"IPAddress": "192.168.160.3",
"IPPrefixLen": 20,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:c0:a8:a0:03",
"DriverOpts": null
}
}
$ docker container inspect ghost-db-1 -f '{{ json .NetworkSettings.Networks }}' | python3 -m json.tool
{
"ghost_default": {
"IPAMConfig": null,
"Links": null,
"Aliases": [
"ghost-db-1",
"db",
"3af25adfee9a"
],
"NetworkID": "19471798da39dafa7200ad3dd763d2cbb3ffd9cb4d85301f973c9a10764294a5",
"EndpointID": "b125f53ef96ecffb49b47d2f20e65238d7dde95eba1817dd9a3973cb733ce2d1",
"Gateway": "192.168.160.1",
"IPAddress": "192.168.160.2",
"IPPrefixLen": 20,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:c0:a8:a0:02",
"DriverOpts": null
}
}
$ docker container inspect rp -f '{{ json .NetworkSettings.Networks }}' | python3 -m json.tool
{
"rp_default": {
"IPAMConfig": null,
"Links": null,
"Aliases": [
"rp",
"rp",
"1bc59a14613e"
],
"NetworkID": "56f10f4877b46d50d728b0353acc7a933a85e7320825173d277f51a249d22bdd",
"EndpointID": "376da992e7a9d19581f8d25ff805820f83cf715abf9862f083a4eba6ceda4100",
"Gateway": "192.168.144.1",
"IPAddress": "192.168.144.2",
"IPPrefixLen": 20,
"IPv6Gateway": "",
"GlobalIPv6Address": "",
"GlobalIPv6PrefixLen": 0,
"MacAddress": "02:42:c0:a8:90:02",
"DriverOpts": null
}
}
The illustration of these containers and networks would look like this.
Discussion