This is my .gitlab-ci.yml file:
image: docker
stages:
- build
- test
- deploy
build-prod:
stage: build
only:
- master
tags:
- docker
script:
- docker network create -d overlay reprox
environment: master
test-prod:
stage: test
only:
- master
tags:
- runner
script:
- echo "yolo"
environment: master
deploy-prod:
stage: deploy
only:
- master
tags:
- docker
script:
- docker stack deploy -c ./site1/docker-compose.yml site1
- docker stack deploy -c ./site2/docker-compose.yml site2
- docker stack deploy -c ./site3/docker-compose.yml site3
- docker stack deploy -c ./reverse-proxy/docker-compose.yml proxy
environment: master
I have the dummy echo job so it will clone to the worker but when the repo is cloned to any node it always clones to /home/gitlab-runner/builds/random/0/docker/
I don't need variables, I just want it to clone the repo in the /home/gitlab-runner directory for every gitlab-runner.
Related
workflow:
rules:
- if: $CI_COMMIT_BRANCH != "main" && $CI_PIPELINE_SOURCE != "merge_request_event"
when: never
- when: always
variables:
IMAGE_NAME: $CI_REGISTRY_IMAGE
IMAGE_TAG: 1.1
DEV_SERVER_HOST: 3.239.229.167
stages:
test
build
deploy
run_unit_test:
image: node:17-alpine3.14
stage: test
tags:
- docker
- ec2
- remote
before_script:
- cd app
- npm install
script:
- npm test
artifacts:
when: always
paths:
- app/junit.xml
reports:
junit: app/junit.xml
build_image:
stage: build
tags:
- ec2
- shell
- remote
script:
- docker build -t $IMAGE_NAME:$IMAGE_TAG .
push_image:
stage: build
needs:
- build_image
tags:
- ec2
- shell
- remote
before_script:
docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
script:
docker push $IMAGE_NAME:$IMAGE_TAG
deploy_to_dev:
stage: deploy
tags:
- ec2
- shell
- remote
script:
- ssh -o StrictHostKeyChecking=no -i $SSH_PRIVATE_KEY ubuntu#$DEV_SERVER_HOST "
docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY &&
docker run -d -p 3000:3000 $IMAGE_NAME:$IMAGE_TAG"
I dont know why im getting this error
while deploying to dev server gitlab yaml parser is throwing is error " This GitLab CI configuration is invalid: variable definition must be either a string or a hash"
variables:
IMAGE_NAME: $CI_REGISTRY_IMAGE
IMAGE_TAG: "1.1"
DEV_SERVER_HOST: "3.239.229.167"
try adding quotes to variable values
some references - https://gitlab.com/gitlab-org/gitlab-foss/-/issues/22648
I am new to GitLab and was trying to build a sample CICD pipeline. Following is my code:
variables:
REPO_NAME: devsecopscollab/my_test_repo
IMAGE_TAG: demo-app-1.0
include:
- template: SAST.gitlab-ci.yml
- template: Dependency-Scanning.gitlab-ci.yml
stages:
- Test
- Build
- Deploy
job1_runTests:
stage: Test
image: python:3.10.8
before_script:
- apt-get update && apt-get install make
script:
- make test
sast:
stage: Test
artifacts:
name: sast
paths:
- gl-sast-report.json
reports:
sast: gl-sast-report.json
when: always
dependency_scanning:
stage: Test
variables:
CI_DEBUG_TRACE: "true"
artifacts:
name: dependency_scanning
paths:
- gl-dependency-scanning-report.json
reports:
dependency_scanning: gl-dependency-scanning-report.json
when: always
job2_buildImage:
stage: Build
image: docker:20.10.21
services:
- docker:20.10.21-dind
variables:
DOCKER_TLS_CERTDIR: "/certs"
before_script:
- docker login -u $DOCKERHUB_USER -p $DOCKERHUB_PASS
script:
- docker build -t $REPO_NAME:$IMAGE_TAG .
- docker push $REPO_NAME:$IMAGE_TAG
job3_deploy:
stage: Deploy
before_script:
- chmod 400 $SSH_KEY
script:
- ssh -o StrictHostKeyChecking=no -i $SSH_KEY ubuntu#$PUBLIC_IP "
docker login -u $DOCKERHUB_USER -p $DOCKERHUB_PASS &&
docker ps -aq | xargs --no-run-if-empty docker stop | xargs --no-run-if-empty docker rm &&
docker run -d -p 5000:5000 $REPO_NAME:$IMAGE_TAG"
But my pipeline looks like in this image here (no dependency scanning stage is shown):
Is something wrong with this pipeline? Why is dependency scanning stage not visible?
Tried the above given code snippet and was expecting a dependency scanning stage visible on the pipeline.
Dependency scanning works only in ultimate plan. cf: https://gitlab.com/gitlab-org/gitlab/-/issues/327366
https://i.stack.imgur.com/o7GE9.png
Also check this link https://docs.gitlab.com/ee/user/application_security/dependency_scanning/
to trigger the dependency scanning template is easer just use
include:
- template: SAST.gitlab-ci.yml
- template: Dependency-Scanning.gitlab-ci.yml
I am using gitlab.com ci/cd to push images into gitlab registry. I had 2 repositories already and pushing images into registries as registry.gitlab.com/group1/project1 and registry.gitlab.com/group1/project2.
Now I have another 2 repos up and running. I want to run CI/CD for the new projects. I also want to push images into different registries and names them as follows : registry.gitlab.com/group2/project1 and registry.gitlab.com/group2/project2
And I am getting following error:
denied: requested access to the resource is denied
Does free gitlab provide only 2 registries? Do I need to pay to create more registries ?
The example .gitlab-ci.yml below is to lint, build, test, and tag container images for your GitLab project into the matching image registry. Before you can interact with your image registry, you need to login, see the docker login lines, and the GitLab CI Runner uses a generated token for this, but only matches the current project. And also only when you have enabled the Container Registry in the Setting -> General section
---
image: docker:latest
services:
- docker:dind
stages:
- verify
- build
- test
- release
variables:
CI_DOCKER_IMAGE: ${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA}
CI_DOCKER_TAG: ${CI_REGISTRY_IMAGE}:${CI_COMMIT_TAG}
CI_DOCKER_BRANCH: ${CI_REGISTRY_IMAGE}:${CI_COMMIT_REF_NAME}
CI_DOCKER_LATEST: ${CI_REGISTRY_IMAGE}:latest
Docker lint:
stage: verify
image: projectatomic/dockerfile-lint
script:
- dockerfile_lint -p -f Dockerfile
Docker build:
stage: build
before_script:
- docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
script:
- docker build --pull -t ${CI_DOCKER_IMAGE} .
- docker push ${CI_DOCKER_IMAGE}
Docker test:
stage: test
before_script:
- docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
script:
- docker pull ${CI_DOCKER_IMAGE}
- docker run ${CI_DOCKER_IMAGE} /path/to/test.sh
Release branch:
stage: release
before_script:
- docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
script:
- docker pull ${CI_DOCKER_IMAGE}
- docker tag ${CI_DOCKER_IMAGE} ${CI_DOCKER_BRANCH}
- docker push ${CI_DOCKER_BRANCH}
only:
- branches
except:
- master
Release tag:
stage: release
before_script:
- docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
script:
- docker pull ${CI_DOCKER_IMAGE}
- docker tag ${CI_DOCKER_IMAGE} ${CI_DOCKER_TAG}
- docker push ${CI_DOCKER_TAG}
only:
- tags
Release latest:
stage: release
before_script:
- docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
script:
- docker pull ${CI_DOCKER_IMAGE}
- docker tag ${CI_DOCKER_IMAGE} ${CI_DOCKER_LATEST}
- docker push ${CI_DOCKER_LATEST}
only:
- master
If you want to push to an image registry other than the default registry that comes with your project, then you should provide the credentials to the CI/CD -> Variables section. Also, .gitlab-ci.yml should be modified to have the correct registry, username and accesstoken variables in the YAML file.
I'm currently trying to setup a gitlab ci pipeline. I've chosen to go with the Docker-in-Docker setup.
I got my ci pipeline to build and push the docker image to the registry of gitlab but I cannot seem deploy it using the following configuration:
.gitlab-ci.yml
image: docker:stable
services:
- docker:dind
stages:
- build
- deploy
variables:
DOCKER_HOST: tcp://docker:2375
DOCKER_DRIVER: overlay2
TEST_IMAGE: registry.gitlab.com/user/repo.nl:$CI_COMMIT_REF_SLUG
RELEASE_IMAGE: registry.gitlab.com/user/repo.nl:release
before_script:
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
- docker info
build:
stage: build
tags:
- build
script:
- docker build --pull -t $TEST_IMAGE .
- docker push $TEST_IMAGE
only:
- branches
deploy:
stage: deploy
tags:
- deploy
script:
- docker pull $TEST_IMAGE
- docker tag $TEST_IMAGE $RELEASE_IMAGE
- docker push $RELEASE_IMAGE
- docker run -d --name "review-$CI_COMMIT_REF_SLUG" -p "80:80" $RELEASE_IMAGE
only:
- master
when: manual
When I run the deploy action I actually get the following feedback in my log, but when I go check the server there is no container running.
$ docker run -d --name "review-$CI_COMMIT_REF_SLUG" -p "80:80" $RELEASE_IMAGE
7bd109a8855e985cc751be2eaa284e78ac63a956b08ed8b03d906300a695a375
Job succeeded
I have no clue as to what I am forgetting here. Am I right to expect this method to be correct for deploying containers? What am I missing / doing wrong?
tldr: Want to deploy images into production using gitlab ci and docker-in-docker setup, job succeeds but there is no container. Goal is to have a running container on host after deployment.
Found out that I needed to include the docker socket in the gitlab-runner configuration as well, and not only have it available in the container.
By adding --docker-volumes '/var/run/docker.sock:/var/run/docker.sock' and removing DOCKER_HOST=tcp://docker:2375 I was able to connect to docker on my host system and spawn sibling containers.
I have a problem. I have two merge requests from two different branches to the master branch in a project.
Now I want to start an environment in GitLab for each merge request. I do this with a shell executor and I start docker container with docker run image_name where I mount the folder from the build process inside the container. Looks like this:
stages:
- deploy
deploy_stage:
stage: deploy
script:
- docker run -d --name ContainerName -v ${CI_PROJECT_DIR}:/var/www/html -e VIRTUAL_HOST=example.com php
environment:
name: review/$CI_COMMIT_REF_NAME
url: http://example.com
on_stop: stop_stage
tags:
- shell
except:
- master
stop_stage:
stage: deploy
variables:
GIT_STRATEGY: none
script:
- docker stop ContainerName
- docker rm ContainerName
when: manual
environment:
name: review/$CI_COMMIT_REF_NAME
action: stop
tags:
- shell
Now my problem is that one environment is runng and when a new job runs the checkout/code get overwritten by the new pipeline job and both environments have now the same code but they should be different.
Does anyone have a solution for me how I can configure the gitlab runner to have different checkout folder for each merge request?