Gitlab Runner cannot run `docker` commands in `docker` executor - gitlab

Question

This has been driving me nuts as I think I'm exactly following the documentation by GitLab for setting up DIND using socket in GitLab Runner so I can run docker commands in Gitlab CI job. But it keeps giving the following error -
Running with gitlab-runner 14.0.0 (3b6f852e)
on Gitlab-HiddenLayer-Group-Runner GosSpAyH
Preparing the "kubernetes" executor
00:00
Using Kubernetes namespace: gitlab
Using Kubernetes executor with image docker:19.03.12 ...
Using attach strategy to execute scripts...
Preparing environment
00:07
Waiting for pod gitlab/runner-gosspayh-project-27874308-concurrent-0qkp2h to be running, status is Pending
Waiting for pod gitlab/runner-gosspayh-project-27874308-concurrent-0qkp2h to be running, status is Pending
ContainersNotReady: "containers with unready status: [build helper]"
ContainersNotReady: "containers with unready status: [build helper]"
Running on runner-gosspayh-project-27874308-concurrent-0qkp2h via gitlab-runner-gitlab-runner-6984874897-l9z5z...
Getting source from Git repository
00:02
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/GosSpAyH/0/hiddenlayer/hl-tech-blog/.git/
Created fresh repository.
Checking out c48b6257 as master...
Skipping Git submodules setup
Executing "step_script" stage of the job script
00:00
$ docker info
Client:
Debug Mode: false
Server:
ERROR: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
errors pretty printing info
Cleaning up file based variables
00:01
ERROR: Job failed: command terminated with exit code 1
Here is my toml configuration in values.yaml for GitLab Runner installation in my private Kubernetes cluster.
config: |
[[runners]]
url = "https://gitlab.com/"
executor = "docker"
privileged = true
[runners.docker]
tls_verify = false
image = "docker:19.03.12"
privileged = true
disable_cache = false
volumes = ["/var/run/docker.sock:/var/run/docker.sock", "/cache"]
[runners.cache]
Insecure = false
and my .gitlab-ci.yml is the following -
image: docker:19.03.12
variables:
DOCKER_DRIVER: overlay2
before_script:
- docker info
- echo "$CI_REGISTRY_USER | $CI_REGISTRY_PASSWORD | $CI_REGISTRY"
- docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
build:
stage: build
# Default branch leaves tag empty (= latest tag)
# All other branches are tagged with the escaped branch name (commit ref slug)
script:
- |
if [[ "$CI_COMMIT_BRANCH" == "$CI_DEFAULT_BRANCH" ]]; then
tag=""
echo "Running on default branch '$CI_DEFAULT_BRANCH': tag = 'latest'"
else
tag=":$CI_COMMIT_REF_SLUG"
echo "Running on branch '$CI_COMMIT_BRANCH': tag = $tag"
fi
- docker build --pull -t "$CI_REGISTRY_IMAGE${tag}" -f deploy/Dockerfile .
- docker push "$CI_REGISTRY_IMAGE${tag}"
Note: I'm intentionally leaving the docker-dind service from the .gitlab-ci.yaml file because the documentation says it is not needed.
Additional Information:
Kubernetes Version: 1.20
Gitlab Runner Version: 14.0.0
Running docker commands in CIs is a pretty common workflow and I'm starting to think if it's this difficult to setup, I may as well go back to old ways with using Jenkins.

Answer 1

See if upgrading to GitLab 14.3 (September 2021) would help:
Support for Kubernetes 1.20
In GitLab 14.3, we added support to Kubernetes version 1.20.
GitLab users can benefit from having recent cluster versions in many features, such as the GitLab Kubernetes Agent, Auto DevOps and Cluster Management Project.
You can find the list of supported versions and related timelines in our documentation.
See Documentation and Epic.