About docker (for beginners)

This time I will explain about docker.

Advantages of docker

The advantages of docker are
Easy to manage dependencies of packages insatlled with pip etc.

Docker provides virtual environments called containers in which code can be executed.
Therefore, I downloaded the package to my PC using pip, and it did not work because each vesion was not compatible. You can avoid situations like this.
*Other benefits include a uniform environment for team development, high security and scalability. Recently, anaconda has become available for a fee, and I think its value is increasing even more. Docker Desktop has also become a paid service, but it is fine for personal use (even for commercial use). (As of 2023/10/15)

docker basics

Let's try using docker right away!

First, I will explain the world of docker.
dcoker consists of an image and a container.

Image: A template that creates a container. With this, you can create as many containers as you want.
Container: Virtual environment. This is what we want to use

make

・About dockerfile
A docker file is a file that describes settings etc. when creating an image. We will set up the startup environment and install the libraries.

dockerfile

# base on ubuntu image
FROM ubuntu:20.04
USER root

# install package non-interactively
ENV DEBIAN_FRONTEND=noninteractive

# install required libraries
RUN apt-get update && apt-get install -y \
     build-essential \
     libfreetype6-dev\
     libhdf5-serial-dev\
     libzmq3-dev\
     pkg-config\
     python3-pip\
     software-properties-common \
     unzip

# set working directory
RUN apt install -y python3.9
RUN apt install -y python3.9-dev

# COPY local path Container side path
COPY requirements.txt.
RUN python3.9 -m pip install -r requirements.txt

# ENV SITE_DOMAIN = DOMAIN_NAME

WORKDIR /var/my_project
ADD https://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data .

COPY script.py.
ENTRYPOINT [ "python3.9", "script.py" ]

・Creating an image (created from dockerfile)
When executing it, please execute it in the directory where the dockerfile exists.
In the case of the above dockerfile, you will need my_project/script.py (the code you want to execute) and requirements.txt in that directory.

docker build -t image_name:1.0 .
# create image from dockerfile
# docker build -t (add tag) image_name (name of image to be created):1.0 (tag) .(Specify the directory where the dockerfile exists. Here, specify the current directory)

・Creating a container

docker run --name container_name image_name:1.0
# create a container from the image
# docker run --name (assign a name) container_name (name of the container to be created) image_name:1.0 (specify the name and tag of the image that will become the base of the container)

*This image or container is saved in the root file system of the system where Docker is installed and remains even if the directory is deleted.

confirm

・Confirm image

docker images
# Check the created image

REPOSITORY TAG IMAGE ID CREATED SIZE
test_pyhton 1.0 h440e3k5gc6b 36 minutes ago 751MB

・Container confirmation

docker ps -a
# Check the created container
# docker ps (check image) -a (including stopped containers)

CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
34tb10c3k1fb test-pyhton:1.0 "/bin/bash" 3 seconds ago Up 2 seconds test-server-1

to erase

・Erase image
*If there is a container created based on the image, you need to delete the container first.

docker rmi image_id
# erase image
# docker rmi (delete image) image_id or _name:tag (image id or name: tag)

docker rmi $(docker images -a -q)
# Delete all images (be careful)
# windows ver: FOR /f "tokens=*" %i IN ('docker images -a -q') DO docker rmi %i

・Delete container
*If the container is running, you need to stop it first.

docker rm container_id
# delete container
# docker rm (delete container) container_id or _name (id or name of container)

・Delete unused objects

# Delete unnecessary data
docker system prune

stop, restart

A container started with run can be stopped with stop and restarted with start.
·Stop

docker stop container_id
# Stopping the container
# docker stop container_id or _name (container ID or name)

・Restart

docker start container_name
# restart container
# docker start (resume) container_id or _name (container id or name)

*The image is just a template, so there is no concept of stopping or restarting.

use

I will explain how to actually use it.
Here, for beginners like me, we will run a file on docker. We will use docker for this purpose.

First, create a dockerfile.

・Practical commands

docker run --rm --name container_name image_name:1.0
# Create a container from image_name, execute the ENTRYPOINT written in the dockerfile, and delete the container after completion
# docker run (create a container) --rm (delete after processing is complete) --name (assign a name) container_name (name of the container) image_python:1.0 (name and tag of the original image)

# mount
docker run --rm -v /Users/yuto/Downloads/Python/Docker/machine_learning:/var --name container_name image_name:1.0
# Share the absolute path local directory with the docker container, and when the local is changed, the docker side is also changed, and you can execute the changed script file just by running.
#*It is necessary to have the same directory structure as WORKDIR created in the image. WORKDIR sets the execution directory for subsequent commands such as ADD, COPY, CMD, and ENTRYPOINT; if the directory does not exist, it will be created automatically.

"Python script executed!"

By doing the above, you can run the script as many times as you like in the virtual environment.

Now you can run the file in the environment where you installed your favorite package.

I'm sure there's more that can be done, but that's all for now!
Thank you for reading!

Executing background

*note: If you wanna rerun docker image with different config, you have to remove old container from same docker image.

docker run -d --name my_python_app myimage python /path/to/your_script.py

In this command:
my_python_app is the name you assign to your container.
myimage is the name of your Docker image.
/path/to/your_script.py is the path to your Python script within the Docker image.

・Example(execute background with mount)

docker run -itd -v /Users/[username]/Downloads/Programing/Docker/BirdCLEF2024:/home --name kaggle gcr.io/kaggle-images/python python /home/birdcref2024-make-dataset.py

bonus

This is a summary of my personal research.

-What is a tag?
Tags are used for image version management, and you cannot create a combination of image and tag with the same name. If you create an image with the same name and tag, the previous tag will be assigned to the new image, and the previous image cannot be referenced by tag, but only by imageID.