DNS and Jupyter Notebook Using Docker - s2/7
日本語記事準備中-シリーズ前後記事リンク追加予定
first post: Cheap Home LAN Playground Using Docker
This is the second post of the blog series and will cover DNS and Jupyter Notedbook setup.
If you have your own, custom DNS server on your LAN, you can add whatever name, DNS record on your DNS server for your use.
It is easier to show how you can use DNS when you actually have a web service running. I will be running DNS and Jupyter Notebook using docker in this post.
Accessing service with and without DNS
If you have a web server listening on 192.168.1.250:80 which replies client "YAHOO!", and if you have DNS record yahoo.example.net that resolves to 192.168.1.250, you can access http://yahoo.example.net on your browser to access web server saying "YAHOO!".
Jupyter Notebook on Docker
As briefly touched in the first post, I will be running Jupyter Notebook. For example if it's running on a machine with 192.168.1.56, and the Jupyter Notebook docker container exposing port 8888, then I can access it through web browser by typing in http://192.168.1.56:8888. If you decide to name it jupyter.mylan.local and add the record to be resolved to 192.168.1.56 on your local managed DNS server, you can also access it on http://jupyter.mylan.local:8888.
Let me double check that docker compose is available on my machine.
$ docker compose version
Docker Compose version v2.12.0
Let me create a directory $HOME/mylan/jupyter to place docker-compose.yml file which is the configuration file to tell docker compose what to do.
$ mkdir -p $HOME/mylan/jupyter
$ cd $HOME/mylan/jupyter
# and then create a docker-compose.yml file here
Here is the content of the docker-compose.yml file. And here I am using jupyter/base-notebook image.
services:
jupyter:
container_name: jupyter
image: jupyter/base-notebook:notebook-6.5.1
ports:
- "8888:8888"
command: "start-notebook.sh --ServerApp.password='' --ServerApp.token='' --ip=0.0.0.0 --no-browser"
Now I am ready to have docker run the container. docker compose up -d to run, docker compose ps to check the status, and then docker compose down to stop it. Once it is in running state, I can access http://192.168.1.56:8888 on browser from other machines on my LAN to start using Jupyter Notebook.
$ docker compose up -d
[+] Running 13/13
⠿ jupyter Pulled 97.2s
⠿ cf92e523b49e Pull complete 16.1s
⠿ d122d224d8ab Pull complete 36.9s
⠿ da0342913a35 Pull complete 37.0s
⠿ 4f4fb700ef54 Pull complete 37.1s
⠿ c2f2efbb391f Pull complete 37.2s
⠿ 291238b4cf86 Pull complete 37.3s
⠿ 47fcc7e93076 Pull complete 37.4s
⠿ 8c8e1d8f7987 Pull complete 37.5s
⠿ 3c19a726ce99 Pull complete 92.8s
⠿ 442a90c0da0a Pull complete 92.9s
⠿ e511a84b0db8 Pull complete 93.0s
⠿ 44bce5cd969a Pull complete 93.1s
[+] Running 2/2
⠿ Network jupyter_default Created 0.2s
⠿ Container jupyter Started 2.4s
$ docker compose ps
NAME COMMAND SERVICE STATUS PORTS
jupyter "tini -g -- start-no…" jupyter running (healthy) 0.0.0.0:8888->8888/tcp, :::8888->8888/tcp
Use docker compose down to stop the service.
$ docker compose down
[+] Running 2/2
⠿ Container jupyter Removed 0.8s
⠿ Network jupyter_default Removed 0.4s
$ docker compose ps
NAME COMMAND SERVICE STATUS PORTS
Unbound DNS on Docker
Now let me bring up DNS service using docker by creating docker-compose.yml file on dns directory as I previously did with Jupyter Notebook.
$ mkdir -p $HOME/mylan/dns
$ cd $HOME/mylan/dns
# create a docker-compose.yml file here
Here is the docker-compose.yml file. And here I am using mvance/unbound image.
services:
dns:
image: mvance/unbound:1.16.2
container_name: dns
ports:
- '53:53/udp'
- '53:53'
I go ahead and run this (docker compose up -d) and confirm the status (docker compose ps).
$ docker compose ps
NAME COMMAND SERVICE STATUS PORTS
dns "/unbound.sh" dns running (starting) 0.0.0.0:53->53/tcp, 0.0.0.0:53->53/udp, :::53->53/tcp, :::53->53/udp
In the previous section, I accessed Jupyter Notebook using web browser to confirm the service is running. Likewise, I can send name lookup request to DNS server to confirm the service is fine. Here are example commands you can run from a machine on the same LAN.
host www.google.com. 192.168.1.56
nslookup www.google.com. 192.168.1.56
dig www.google.com. @192.168.1.56
By the way, depending on the image version, it could be using general root nameservers, or Cloudflare DNS over TLS, or something else.
Configure DNS server
Now this DNS server running on docker is functional, I want to configure it so that it can resolve jupyter.mylan.local to 192.168.1.56.
Which file on which directory to edit to configure service really varies. You have to go check the documentation for the docker image or the service you are using.
And when you are to edit a configuration file, you do not do that inside the running container. You prepare the configuration file locally on your machine that runs docker, and have docker map the configuration file to the appropriate location of the docker container when it starts running.
Now let me prepare the configuration file. In case of this image mvance/unbound:1.16.2 I am using, there is a sample file so I will start from there.
mkdir config
cd config
# copy a-records.conf configuration file from "dns" docker container
docker cp dns:/opt/unbound/etc/unbound/a-records.conf .
This sample config file a-records.conf looks like this.
$ cat a-records.conf
# A Record
#local-data: "somecomputer.local. A 192.168.1.1"
# PTR Record
#local-data-ptr: "192.168.1.1 somecomputer.local."
I edit this file by adding lines similarly, removing the leading comment out #, and just change the name and IP address.
$ cat a-records.conf
# A Record
#local-data: "somecomputer.local. A 192.168.1.1"
local-data: "jupyter.mylan.local. A 192.168.1.56"
# PTR Record
#local-data-ptr: "192.168.1.1 somecomputer.local."
local-data-ptr: "192.168.1.56 jupyter.mylan.local."
And then I also edit docker-compose.yml file located at $HOME/mylan/dns to have docker use this file inside the container. The custom configuration file created is at ./config/a-records.conf and I want it to be mapped to /opt/unbound/etc/unbound/a-records.conf file inside the container. The trailing :ro means read-only mode.
More information on this file mapping operation, bind, can be found here in the official documentation.
$ cat docker-compose.yml
services:
dns:
image: mvance/unbound:1.16.2
container_name: dns
ports:
- '53:53/udp'
- '53:53'
volumes:
- './config/a-records.conf:/opt/unbound/etc/unbound/a-records.conf:ro'
I restart the container, and docker will use this updated docker-compose.yml file which will have custom config file loaded to run this container.
# run `cd $HOME/mylan/dns` first if you are not there
docker compose restart
# or docker compose down, and then docker compose up -d
Now I am ready to change the DNS settings on my mobile, laptop, or whatever to use 192.168.1.56 as my DNS server and start accessing my Jupyter Notebook on web browser by accessing http://jupyter.mylan.local:8888.
❯ curl jupyter.mylan.local:8888 -v
* Trying 192.168.1.56:8888...
* Connected to jupyter.mylan.local (192.168.1.56) port 8888 (#0)
Files created on Jupyter Notebook
If you restart the host machine running docker, or even restarting the Jupyter Notebook docker compose to stop and restart the container, you notice that files you created are always all removed. You can create a docker volume as the solution.
Here is the updated docker-compose.yml file. This now creates the volume jupyter_volume and mount it at /home/jovyan which is the working directory of the Jupyter Notebook image I am using.
services:
jupyter:
container_name: jupyter
image: jupyter/base-notebook:notebook-6.5.1
user: root
ports:
- "8888:8888"
command: "start-notebook.sh --ServerApp.password='' --ServerApp.token='' --ip=0.0.0.0 --no-browser"
environment:
- "CHOWN_EXTRA=/home/jovyan"
- "CHOWN_EXTRA_OPTS=-R"
volumes:
- type: volume
source: jupyter_volume
target: /home/jovyan
volume:
nocopy: true
volumes:
jupyter_volume: {}
As you can see, there are many new lines added to the docker-compose.yml file in addition to just adding volume section. In general case, the docker volume created is under directory with root permission on the host machine, and the user of the Jupyter Notebook docker container is non-root, and that leads to permission error whenever you try to create a new file on Jupyter Notebook.
Cleaning up Docker
I have covered major items mentioned in the subject, however, I need to add one more section to cover clean up procedure.
Go to jupyter and dns directory and run docker compose down to stop the containers running using docker. And then run docker system prune --volumes to delete all unused images, volumes, and everything.
$ docker system prune --volumes
WARNING! This will remove:
- all stopped containers
- all networks not used by at least one container
- all volumes not used by at least one container
- all dangling images
- all dangling build cache
Are you sure you want to continue? [y/N] y
Deleted Volumes:
5f9710e79b0e3478c1faf63ba85c8f584cddc264cf82ffe912daf4396b164f12
e80bd0c29be5882802c488061f824402965cc7d0bced938ee59ad4d2a9907551
ee0cd42517800cf647cd7ac371e88544ba67c6cdc7448fdecfea7a52e00c1345
...
f443285371dff464c127e1430dc64e41df86675f8b11f7b1ccc80b57a41f7047
994da20fd1461980079a343d20d0326307d421869b955e5f8874c1391ec701d6
Deleted Images:
untagged: redis@sha256:4bed291aa5efb9f0d77b76ff7d4ab71eee410962965d052552db1fb80576431d
deleted: sha256:3900abf4155226f3f62401054b872ce0c85b5c3b47275cae3d16a39c8646e36b
deleted: sha256:8ba2d28fdd3729dec59c7e11c3c99b5df826f7d4fc63c358ae54833e27a16e92
...
deleted: sha256:3779241fda7b1caf03964626c3503e930f2f19a5ffaba6f4b4ad21fd38df3b6b
deleted: sha256:bacd3af13903e13a43fe87b6944acd1ff21024132aad6e74b4452d984fb1a99a
Deleted build cache objects:
l8z524ebp7tmuvmusjxc4jn2b
rp9mtdw4y51kgjuferu1khdzv
Total reclaimed space: 2.925GB
If you continue on and try out different docker images, different version of those images, and create volumes for containers, those data would pile up and may impact disk space. Or you may want to just get things cleaned up to test out other tutorials or quick start materials from scratch.
In case if you get lost and cannot find which docker-compose.yml file is running which container, you can first run docker ps to find the running container names, and then run docker container inspect container_name_here -f '{{ index .Config.Labels "com.docker.compose.project.working_dir"}}'
$ docker ps --format '{{.Names}}'
dns
jupyter
$ docker container inspect jupyter -f '{{ index .Config.Labels "com.docker.compose.project.working_dir"}}'
/home/{username}/mylan/jupyter
Closing
And this is it for this post. In the next post I will setup a reverse proxy server to further change how user would access a service such as Jupyter Notebook.
Discussion