I'm currently trying to setup a gitlab ci pipeline. I've chosen to go with the Docker-in-Docker setup.
I got my ci pipeline to build and push the docker image to the registry of gitlab but I cannot seem deploy it using the following configuration:
.gitlab-ci.yml
image: docker:stable
services:
- docker:dind
stages:
- build
- deploy
variables:
DOCKER_HOST: tcp://docker:2375
DOCKER_DRIVER: overlay2
TEST_IMAGE: registry.gitlab.com/user/repo.nl:$CI_COMMIT_REF_SLUG
RELEASE_IMAGE: registry.gitlab.com/user/repo.nl:release
before_script:
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
- docker info
build:
stage: build
tags:
- build
script:
- docker build --pull -t $TEST_IMAGE .
- docker push $TEST_IMAGE
only:
- branches
deploy:
stage: deploy
tags:
- deploy
script:
- docker pull $TEST_IMAGE
- docker tag $TEST_IMAGE $RELEASE_IMAGE
- docker push $RELEASE_IMAGE
- docker run -d --name "review-$CI_COMMIT_REF_SLUG" -p "80:80" $RELEASE_IMAGE
only:
- master
when: manual
When I run the deploy action I actually get the following feedback in my log, but when I go check the server there is no container running.
$ docker run -d --name "review-$CI_COMMIT_REF_SLUG" -p "80:80" $RELEASE_IMAGE
7bd109a8855e985cc751be2eaa284e78ac63a956b08ed8b03d906300a695a375
Job succeeded
I have no clue as to what I am forgetting here. Am I right to expect this method to be correct for deploying containers? What am I missing / doing wrong?
tldr: Want to deploy images into production using gitlab ci and docker-in-docker setup, job succeeds but there is no container. Goal is to have a running container on host after deployment.
Found out that I needed to include the docker socket in the gitlab-runner configuration as well, and not only have it available in the container.
By adding --docker-volumes '/var/run/docker.sock:/var/run/docker.sock' and removing DOCKER_HOST=tcp://docker:2375 I was able to connect to docker on my host system and spawn sibling containers.
Related
Here is my scenario:
I create a docker image from an SQL dump with the following commands, executed from command prompt:
docker pull mariadb:10.4.26
docker run --name test_smdb -e MYSQL_ROOT_PASSWORD=<some_password> -p 3306:3306 -d mariadb:10.4.26
docker exec -it test_smdb mariadb --user root -p<some_password>
MariaDB [(none)]> CREATE DATABASE smdb_dev;
docker exec -i test_smdb mariadb -uroot -p<some_password> smdb_dev --force < C:\smdb-dev.sql
But my task now is create a pipeline, that creates this docker image and puts it into the Azure Container Registry
I found this link - Build and push Docker images to Azure Container Registry:
https://learn.microsoft.com/en-us/azure/devops/pipelines/ecosystems/containers/acr-template?view=azure-devops
And i see that the result should be a yaml file like this:
- stage: Build
displayName: Build and push stage
jobs:
- job: Build
displayName: Build job
pool:
vmImage: $(vmImageName)
steps:
- task: Docker#2
displayName: Build and push an image to container registry
inputs:
command: buildAndPush
repository: $(imageRepository)
dockerfile: $(dockerfilePath)
containerRegistry: $(dockerRegistryServiceConnection)
tags: |
$(tag)
But can someone show me how to combine the two things - the docker image creation and the putting it into the Azure Container Registry?
You would need to make a dockerfile and put this in the repository.
The commands you specified at the top of your question should be your input
It could look something like this (just threw something together. probably wont work as is):
# syntax=docker/dockerfile:1
FROM mariadb:10.4.26
WORKDIR /app
COPY . .
run --name test_smdb -e MYSQL_ROOT_PASSWORD=<some_password> -p 3306:3306 -d mariadb:10.4.26
run MariaDB [(none)]> CREATE DATABASE smdb_dev;
EXPOSE {mariadb port #}
Hello I have a problem with GitLab CI/CD. I'm trying to deploy container to Kubernetes on GKE however I'm getting an error:
This job failed because the necessary resources were not successfully created.
I created a service account with kube-admin rights and created cluster via GUI of GitLab so its fully itegrated. But when I run the job it still doesn't work..
by the way I use kubectl get pods in gitlab-ci file just to test if kubernetes is repsonding.
stages:
- build
- deploy
docker-build:
# Use the official docker image.
image: docker:latest
stage: build
services:
- docker:dind
before_script:
- docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
# Default branch leaves tag empty (= latest tag)
# All other branches are tagged with the escaped branch name (commit ref slug)
script:
- docker build --pull -t "$CI_REGISTRY_IMAGE${tag}" .
- docker push "$CI_REGISTRY_IMAGE${tag}"
deploy-prod:
stage: deploy
image: bitnami/kubectl:latest
script:
- kubectl get pods
environment:
name: production
kubernetes:
namespace: test1
Any Ideas?
Thank you
namespace should be removed.
GitLab creates own namespace for every project
I have set up a simple .gitlab-ci file which should be able to run docker service:
docker:
image: google/cloud-sdk:latest
variables:
DOCKER_DRIVER: overlay2
DOCKER_HOST: tcp://127.0.0.1:2375
services:
- docker:dind
tags:
- docker
script:
- docker pull buster-slim
However it fails as:
https://gitlab.com/knyttl/runnerdemo/-/jobs/932204050
2020-12-25T19:31:04.558361767Z time="2020-12-25T19:31:04.558195638Z" level=info msg="API listen on [::]:2375"
2020-12-25T19:31:04.558522591Z time="2020-12-25T19:31:04.558447616Z" level=info msg="API listen on /var/run/docker.sock"
The service apparently correctly starts, but then it doesn't work:
Cannot connect to the Docker daemon at tcp://127.0.0.1:2375. Is the docker daemon running?
The problem comes from the fact that the docker daemon is not part of the image google/cloud-sdk/ that you specify for this job. You should create your own image with the google/cloud-sdk as a base image. You can also install & start docker in the before_script section of the job. See the doc on DockerHub for the image you use:
Installing additional components
By default, all gcloud components are installed on the default images (google/cloud-sdk:latest and google/cloud-sdk:VERSION).
The google/cloud-sdk:slim and google/cloud-sdk:alpine images do not contain additional components pre-installed.
You can extend these images by following the instructions below:
Debian-based images
cd debian_slim/
docker build --build-arg CLOUD_SDK_VERSION=159.0.0
--build-arg INSTALL_COMPONENTS="google-cloud-sdk-datastore-emulator" \
-t my-cloud-sdk-docker:slim .`
The image which can use DIND is docker:dind, not necessarily google/cloud-sdk:latest, so your .gitlab-ci.yml wuold look like:
docker:
image: docker:dind
variables:
DOCKER_DRIVER: overlay2
services:
- docker:dind
tags:
- docker
script:
- docker pull buster-slim
# ...
# I don't know what needs to be built...
You can check this tutorial for a step by step recipe.
In fact, the only reason why this was not working was:
DOCKER_HOST: tcp://docker:2375
The dind service CAN run within cloud-sdk image, but it needs to be linked as a host.
I am using gitlab.com ci/cd to push images into gitlab registry. I had 2 repositories already and pushing images into registries as registry.gitlab.com/group1/project1 and registry.gitlab.com/group1/project2.
Now I have another 2 repos up and running. I want to run CI/CD for the new projects. I also want to push images into different registries and names them as follows : registry.gitlab.com/group2/project1 and registry.gitlab.com/group2/project2
And I am getting following error:
denied: requested access to the resource is denied
Does free gitlab provide only 2 registries? Do I need to pay to create more registries ?
The example .gitlab-ci.yml below is to lint, build, test, and tag container images for your GitLab project into the matching image registry. Before you can interact with your image registry, you need to login, see the docker login lines, and the GitLab CI Runner uses a generated token for this, but only matches the current project. And also only when you have enabled the Container Registry in the Setting -> General section
---
image: docker:latest
services:
- docker:dind
stages:
- verify
- build
- test
- release
variables:
CI_DOCKER_IMAGE: ${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA}
CI_DOCKER_TAG: ${CI_REGISTRY_IMAGE}:${CI_COMMIT_TAG}
CI_DOCKER_BRANCH: ${CI_REGISTRY_IMAGE}:${CI_COMMIT_REF_NAME}
CI_DOCKER_LATEST: ${CI_REGISTRY_IMAGE}:latest
Docker lint:
stage: verify
image: projectatomic/dockerfile-lint
script:
- dockerfile_lint -p -f Dockerfile
Docker build:
stage: build
before_script:
- docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
script:
- docker build --pull -t ${CI_DOCKER_IMAGE} .
- docker push ${CI_DOCKER_IMAGE}
Docker test:
stage: test
before_script:
- docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
script:
- docker pull ${CI_DOCKER_IMAGE}
- docker run ${CI_DOCKER_IMAGE} /path/to/test.sh
Release branch:
stage: release
before_script:
- docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
script:
- docker pull ${CI_DOCKER_IMAGE}
- docker tag ${CI_DOCKER_IMAGE} ${CI_DOCKER_BRANCH}
- docker push ${CI_DOCKER_BRANCH}
only:
- branches
except:
- master
Release tag:
stage: release
before_script:
- docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
script:
- docker pull ${CI_DOCKER_IMAGE}
- docker tag ${CI_DOCKER_IMAGE} ${CI_DOCKER_TAG}
- docker push ${CI_DOCKER_TAG}
only:
- tags
Release latest:
stage: release
before_script:
- docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
script:
- docker pull ${CI_DOCKER_IMAGE}
- docker tag ${CI_DOCKER_IMAGE} ${CI_DOCKER_LATEST}
- docker push ${CI_DOCKER_LATEST}
only:
- master
If you want to push to an image registry other than the default registry that comes with your project, then you should provide the credentials to the CI/CD -> Variables section. Also, .gitlab-ci.yml should be modified to have the correct registry, username and accesstoken variables in the YAML file.
I am using gitlab's pipeline for CI and CD to build images for my projects.
In every job there are configurations to be set like image and stage but I can't wrap my head around what services are. Can someone explain its functionality? Thanks
Here's a code snippet I use that I found
build-run:
image: docker:latest
stage: build
services:
- docker:dind
script:
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
- docker build -t "$CI_REGISTRY_IMAGE/my-project:$CI_COMMIT_SHA" .
- docker push "$CI_REGISTRY_IMAGE/my-project:$CI_COMMIT_SHA"
cache:
untracked: true
environment: build
The documentation says:
The services keyword defines just another Docker image that is run during your job and is linked to the Docker image that the image keyword defines. This allows you to access the service image during build time.