Running GitLab Runner in a Docker Container

Introduction

GitLab Runner is a powerful tool that integrates with GitLab CI/CD to execute various tasks within your CI/CD pipeline. There are two primary ways to get started with GitLab Runner:

• Using GitLab.com’s SaaS runners or
• Setting up self-managed runners

This guide will focus on the latter, running GitLab Runner in a Docker container. This approach provides flexibility and control over your CI/CD environment.

Prerequisites

Before diving into the setup, ensure you have the following prerequisites:

1. A GitLab account.
2. Docker and Docker Compose (optional) installed on your host machine.

Setting Up Your GitLab Account

1. Sign up for GitLab if you haven’t already.
2. Create a repository in your GitLab account.
3. Inside your repository, create a file named .gitlab-ci.yml to define your pipeline.
4. Write a simple pipeline with the desired jobs (e.g., build and tests).

build-job:
  stage: build
  script:
    - echo "Hello, $GITLAB_USER_LOGIN!"

test-job1:
  stage: test
  script:
    - echo "This job tests something"

test-job2:
  stage: test
  script:
    - echo "This job tests something, but takes more time than test-job1..."
    - sleep 150

Setting runner using Docker Compose

In this section, I will use Docker Compose to manage containers. The following Docker Compose file sets up three essential services:

version: "3.5"
services:
  dind:
   image: docker:20-dind
   privileged: true
   environment:
     DOCKER_TLS_CERTDIR: ""
   command:
     - --storage-driver=overlay2
  runner:
    image: registry.gitlab.com/gitlab-org/gitlab-runner:alpine
    depends_on:
      - dind
    environment:
      - DOCKER_HOST=tcp://dind:2375
    volumes:
      - ./gitlab-runner:/etc/gitlab-runner
  register-runner:
    image: registry.gitlab.com/gitlab-org/gitlab-runner:alpine
    depends_on:
      - dind
    environment:
      - CI_SERVER_URL=${CI_SERVER_URL}
      - REGISTRATION_TOKEN=${REGISTRATION_TOKEN}
    command:
      - register
      - --non-interactive
      - --locked=false
      - --name=${RUNNER_NAME}
      - --executor=docker
      - --docker-image=alpine:latest
    volumes:
      - ./gitlab-runner:/etc/gitlab-runner

1. Dind (Docker in Docker):

• This container runs the Docker daemon within another Docker container.
• The daemon listens on a local Unix socket (unix://var/run/docker.sock) and a TCP socket (tcp://0.0.0.0:2375) for incoming requests.

2. Runner:

• This container executes the GitLab Runner jobs using the “docker” executor.
• The runner depends on the Dind container.
• It is configured to use the Docker daemon hosted within the Dind container.
• A volume is mounted to /etc/gitlab-runner to provide access to the GitLab Runner configuration.

3. Register-runner:

• This container registers the GitLab Runner and exits after registration.
• It also depends on the Dind container.
• Environment variables like CI_SERVER_URL, REGISTRATION_TOKEN, and RUNNER_NAME are set to identify and configure the runner. Environment variables are defined in .env file like:

CI_SERVER_URL=https://gitlab.com/
REGISTRATION_TOKEN=xxxxxxxxxx-xxxxxxxxx
RUNNER_NAME=myrunner

• The runner is set to use the “docker” executor, and it specifies the Docker image (e.g., alpine:latest) to run jobs.
• Similar to the runner container, a volume is mounted to /etc/gitlab-runner.

Execution Steps

1. Clone the project from the repo: https://github.com/thesausau/runner.git

git clone https://github.com/thesausau/runner.git

2. Change the registration token in the .env file. Registration token can be achieved from GitLab UI.

CI_SERVER_URL=https://gitlab.com/
REGISTRATION_TOKEN=US1128901zWhzz99QS-TbwsPxXnX7
RUNNER_NAME=myrunner

3. Run docker-compose up -d to start services defined in compose file.

docker-compose up -d
docker ps
CONTAINER ID   IMAGE                                                 COMMAND                  CREATED      STATUS        PORTS           NAMES
402b87561e1a   registry.gitlab.com/gitlab-org/gitlab-runner:alpine   "/usr/bin/dumb-init …"   6 days ago   Up 23 hours                   gitlab_runner_1
31e9e8e0f52f   docker:20-dind                                        "dockerd-entrypoint.…"   6 days ago   Up 23 hours   2375-2376/tcp   gitlab_dind_1

4. Go to GitLab UI view active runner.

Runner “myrunner” in active state

5. Commit pipeline.

git commit -m "Added pipeline"

6. Jobs completed

Execution Flow

1. The Dind container runs a Docker daemon, listening on the specified sockets.
2. The register-runner container registers a GitLab Runner and outputs a config.toml file at /etc/gitlab-runner on the host machine.
3. After registration, the register-runner container terminates.
4. The runner container, configured with the same base image as the register-runner, uses the config.toml file to run the GitLab Runner.
5. The runner container itself does not run jobs but relies on the “docker” executor.
6. To execute jobs, the runner container references the external Docker daemon hosted within the Dind container, using the DOCKER_HOST environment variable.
7. For each job in your pipeline, an Alpine container is launched within the Dind container.
8. Once a job is completed, the relevant container is removed.

Conclusion

By setting up GitLab Runner in a Docker container, you gain control over your CI/CD environment and ensure a consistent and isolated execution of your CI/CD jobs. This approach simplifies the management of runners and enhances flexibility within your GitLab CI/CD pipeline.

Useful Links

• https://www.docker.com/blog/how-to-deploy-on-remote-docker-hosts-with-docker-compose/
• https://docs.gitlab.com/runner/install/docker.html