This is the seventh post in the series. Now I have GitLab server to play with, I am going to setup GitLab Runner and GitLab Pages and explain the steps involved in this post.
As a quick introduction, GitLab Runner is something that runs jobs based on the instruction available on GitLab project repository, and GitLab Pages is where GitLab users can use to host websites. What I can do with these is that I can commit/push website content to a project, let GitLab Runner process it and generate website files and upload them to GitLab Pages to host them.
GitLab Runner is an application that runs outside GitLab, and it works with GitLab by periodically checking in to see if there is any job to execute.
I will run this GitLab Runner using docker compose as I have been doing for others so far.
Creating a group and obtaining registration token
GitLab will not let any anonymous Runner to just come and read the project content hosted. GitLab uses token to let GitLab Runner identify itself to come read the project content.
I am going to create a group and then retrieve a token. This way, when I use that token to register GitLab Runner, it is going to be available for the projects under that group.
Let me create a new group named "pages", and then navigate to the Runner menu as described in the link above to find the token string for a GitLab Runner to register and work with this group.
GitLab Runner registration
I will run mere
docker run to run GitLab Runner container to complete the registration. When the container tries to communicate with the GitLab server, it needs to trust the self-signed certificate it's using.
# as always, creating a directory for GitLab Runner mkdir $HOME/mylan/runner cd $HOME/mylan/runner # store the certificate in $HOME/mylan/runner/gitlab.mylan.local.crt openssl s_client -showcerts -connect gitlab.mylan.local:443 -servername gitlab.mylan.local < /dev/null 2>/dev/null | openssl x509 -outform PEM > gitlab.mylan.local.crt
Now here is the
docker run command to run. The output says the registration was successful.
- config will be stored on the host machine's
- the certificate obtained will be on the container machine as
- specifying the GitLab server in
--clone-urlis also needed, otherwise the runner tries to access the project repository using plain http
- registration token obtained previously must be used in
--tls-ca-file=/tmp/gitlab.mylan.local.crtwill make this GitLab Runner container trust the GitLab server using self-signed certificate
$ docker run --rm -v "$(pwd)"/config:/etc/gitlab-runner \ > -v "$(pwd)"/gitlab.mylan.local.crt:/tmp/gitlab.mylan.local.crt \ > gitlab/gitlab-runner register \ > --non-interactive \ > --executor "docker" \ > --docker-image alpine:latest \ > --url "https://gitlab.mylan.local/" \ > --clone-url "https://gitlab.mylan.local/" \ > --registration-token "GR1348941H2DxBfrLkMrFDfFizvfk" \ > --run-untagged="true" \ > --locked="false" \ > --access-level="not_protected" \ > --tls-ca-file=/tmp/gitlab.mylan.local.crt Runtime platform arch=amd64 os=linux pid=7 revision=7178588d version=15.5.1 Running in system-mode. Registering runner... succeeded runner=GR1348941H2DxBfrL Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded! Configuration (with the authentication token) was saved in "/etc/gitlab-runner/config.toml"
I can see on GitLab server GUI that the runner is registered.
Running GitLab Runner using Docker Compose
docker-compose.yml file for the GitLab Runner and run it with
docker compose up -d.
services: runner: image: 'gitlab/gitlab-runner:alpine3.15-v15.3.3' restart: always container_name: mylan-runner volumes: - type: bind source: ./config target: /etc/gitlab-runner - type: bind source: ./gitlab.mylan.local.crt target: /tmp/gitlab.mylan.local.crt - type: bind source: /var/run/docker.sock target: /var/run/docker.sock
I can see that the runner is in online status.
Testing out GitLab Runner
Let me test out this runner following the tutorial available in the official documentation.
Create a test project
Create a new project named "runner test" under the "pages" group.
I will clone the project repository on my first machine and move to the repository directory.
# clone the repository at home dir cd $HOME git clone https://gitlab.mylan.local/pages/runner-test.git cd runner-test
Add .gitlab-ci.yml file
.gitlab-ci.yml file at
$HOME/runner-test/.gitlab-ci.yml and push the change to the GitLab server.
Here is the content.
build-job: stage: build script: - echo "Hello, $GITLAB_USER_LOGIN!" test-job1: stage: test script: - echo "This job tests something" test-job2: stage: test script: - echo "This job tests something, but takes more time than test-job1." - echo "After the echo commands complete, it runs the sleep command for 20 seconds" - echo "which simulates a test that runs 20 seconds longer than test-job1" - sleep 20 deploy-prod: stage: deploy script: - echo "This job deploys something from the $CI_COMMIT_BRANCH branch." environment: production
Once I create the new file
git status shows that there is an untracked file. I do
git commit, and
git push to update the project repository.
❯ git status On branch main Your branch is up to date with 'origin/main'. Untracked files: (use "git add <file>..." to include in what will be committed) .gitlab-ci.yml nothing added to commit but untracked files present (use "git add" to track) ❯ git add .gitlab-ci.yml ❯ git commit -m "Add tutorial cicd file" [main afe0df9] Add tutorial cicd file 1 file changed, 23 insertions(+) create mode 100644 .gitlab-ci.yml ❯ git push Enumerating objects: 4, done. Counting objects: 100% (4/4), done. Delta compression using up to 4 threads Compressing objects: 100% (3/3), done. Writing objects: 100% (3/3), 583 bytes | 583.00 KiB/s, done. Total 3 (delta 0), reused 0 (delta 0), pack-reused 0 To https://gitlab.mylan.local/pages/runner-test.git 3241e74..afe0df9 main -> main
When I go check the GitLab GUI, I can see that the jobs were executed. GitLab Runner is continuously access GitLab to see if there is anything to do, finds the updated repository with
.gitlab-ci.yml file which is job instructions, and executes them.
Here is the capture of "build-job" and I can see it's executing
echo "Hello, $GITLAB_USER_LOGIN!" which is "Hello, ghost!".
You can find numerous things you can do with GitLab CI/CD in the official document. One I would like to cover is GitLab Pages. As mentioned in the beginning of the post, I will update contents to publish in a project repository, have GitLab Runner run programs to generate web contents, and give them to GitLab Pages to publish.
Setting up GitLab Pages
I need to enable GitLab Pages and specify its base URL. I can do so by editing
gitlab.rb configuration file for GitLab. In my setup, the file is located at
As described in the link above, the URL will be in this format:
http(s)://groupname.example.io/projectname. I have created a group named "pages" and also a project named "runner-test", so I would like the Pages URL to be
I picked up part of GitLab Pages configuration section from the
gitlab.rb configuration file below. I changed these two lines,
################################################################################ ## GitLab Pages ##! Docs: https://docs.gitlab.com/ee/administration/pages/ ################################################################################ ##! Define to enable GitLab Pages pages_external_url "http://mylan.local/" gitlab_pages['enable'] = true
Editing the configuration file will not immediately take effect and change the GitLab server. For example in the previous Nginx configuration changes, I restarted the server, or the docker container running Nginx. In case of GitLab, I can just run
gitlab-ctl reconfigure to have GitLab reconfigure and update itself.
# this runs in foreground and you will see # all the GitLab logs generated while it reconfigures itself docker exec gitlab gitlab-ctl reconfigure # or, add "-d" option to let it run in the background docker exec -d gitlab gitlab-ctl reconfigure
Testing out GitLab Pages with plain html
Let me test out GitLab Pages by using the example project available on GitLab.
I will take the similar steps as I did to test out GitLab Runner. I create a new project "plain-html" under the "pages" group, create html and css files under
/public directory, and then create
So here I have the new project, added "/public" directory containing "/public/index.html" and "/public/style.css" files, and also ".gitlab-ci.yml" file.
Here is the
.gitlab-ci.yml file content.
# This file is a template, and might need editing before it works on your project. # Full project: https://gitlab.com/pages/plain-html image: busybox pages: stage: deploy script: - echo "The site will be deployed to $CI_PAGES_URL" artifacts: paths: - public rules: - if: $CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH
Now what happens is that the GitLab Runner finds a new commit with CI/CD instructions file
.gitlab-ci.yml and figures out that there are jobs to execute. I can see on the project CI/CD menu that there is a new job being executed.
Add DNS record
Now, before I access this GitLab Pages URL
https://pages.mylan.local/plain-http, let me quickly add the new DNS record in my local Unbound DNS running on Docker.
On my first machine running Unbound DNS docker container, I'd go edit
$HOME/mylan/dns/config/a-records.conf to add this new line
local-data: "pages.mylan.local. IN A 192.168.1.56" and restart the container.
cd $HOME/mylan/dns # edit config/a-records.conf to add a new record for pages.mylan.local docker compose restart
Add web server
I also need the additional reverse proxy web server just like the one I created enable access to
cd $HOME/mylan/rp/conf.d cp gitlab.conf pages.conf # edit pages.conf accordingly # just changing "server_name" line as below will suffice ### server_name pages.mylan.local; # once the new file is ready, restart the server cd $HOME/mylan/rp docker compose restart rp
Access the Pages site
GitLab Pages is now enabled, web contents were uploaded by GitLab Runner as instructed in the
.gitlab-ci.yml file, DNS record and reverse proxy for
pages.mylan.local is both ready, and all is set! I can now access the URL and see below website.
GitLab Pages and Mkdocs
The first example requires me to prepare all the html, css, and other files to be published on GitLab Pages. In this and the following sections, I will have GitLab Runner execute a static site generator program to prepare html and all the other files that builds up a website.
I am going to use Mkdocs in this section.
Let me start this by creating a new project named "mkdocs" under the existing "pages" group, and start placing required files. I can see in the getting started page in the Mkdocs documentation that the minimal setup is to have
mkdocs.yml file and
docs directory with markdown files, but in this section let me go directly with what I ended up with including the customization and plugins.
Here is the
/requirements.txt file. This is rather for GitLab Runner to use. Mkdocs runs in python, so in CI/CD job, the runner will use python image to install these packages, and then run Mkdocs to generate website contents to upload on GitLab Pages.
# Mkdocs mkdocs # Theme mkdocs-material # Plugins mkdocs-git-revision-date-plugin mkdocs-mermaid2-plugin mkdocs-awesome-pages-plugin
/docs directory and I can add markdown files and subdirectories however I like. Below tree output is what I ended up with. I will just cover one md filie later in the section along with a screenshot.
.pages file is for Mkdocs to use. It just contains one line telling Mkdocs what to label this folder.
You may notice the numbering on directories and files under
/docs. I am doing so so that the markdown_extensions automatically generates TOC and places contents in order.
❯ tree . |-requirements.txt |-.git |-docs | |-index.md | |-20_placeholder | | |-.pages | | |-100.md | |-10_blog | | |-.pages | | |-105_posts.md | | |-100_posts.md |-README.md |-mkdocs.yml
Now I add
.gitlab-ci.yml file. There is an example GitLab project for Mkdocs in the link below, and has the
.gitlab-ci.yml example I can just copy and use. I have removed the python image tag so that the latest version available will be used.
image: python before_script: - pip install -r requirements.txt test: stage: test script: - mkdocs build --strict --verbose --site-dir test artifacts: paths: - test rules: - if: $CI_COMMIT_REF_NAME != $CI_DEFAULT_BRANCH pages: stage: deploy script: - mkdocs build --strict --verbose artifacts: paths: - public rules: - if: $CI_COMMIT_REF_NAME == $CI_DEFAULT_BRANCH
Once the job gets executed, I can access
https://pages.mylan.local/mkdocs/ to see the website generated using Mkdocs.
Here is how the generated website looks like. I can see the TOC in the left pane. The title of each file come from the h1 text, and the folder title comes from what's written in the
.pages file. I can also see that the Mermaid is working fine, which is an external plugin generating diagram from text.
I am not going through every single files, but let me just share
.pages file and the original markdown file from the above capture as example.
So here are the
❯ cat docs/10_blog/.pages title: Blog ❯ cat docs/20_placeholder/.pages title: Another section - placeholder
And the markdown file.
❯ cat docs/10_blog/105_posts.md # Second file This is the second file. # Another h1 Hello there. ## Mermaid > https://mermaid-js.github.io/mermaid/ ```mermaid graph LR user -- http --> jupyter(jupyter.mylan.net:8888) ```
This post basically concludes the series. I would also like to introduce "mdBooks", one similar to MkDocs, and "marp" which can generate slideshow served as a web page, so I might add a couple more short posts after this.
I have been working as a network engineer for seven years. The different roles I have played over these years provided me with an exposure to very rich variety of technologies and products. They were not just routers and switches, but also DDI (DNS, DHCP, IPAM), loadbalancer, GSLB, firewall, Internet proxy, and etc., and playing around with Docker enriched my experience to setup and manage different kind of services.
There were also many occassion at work where I wrote script to automate jobs, study data gathered, and generate reports. Studying git was my original purpose of running my own GitLab server, but it actually facilitated me writing more in private, not just codes but blogs, stories, documents, recipes, or anything. Getting to know of CI/CD, GitLab Pages, and static site generators was also a great addition.
Thus, I thought having your own playground using Docker worth a tons, and I hope this blog series to be of a good entry point to some of the readers. Please share this blog series with your friends, colleagues, who is interested in start trying out Docker and running services at their home LAN.