I have the following pipeline:
# .gitlab-ci.yml
stages:
- build
- push
build:
stage: build
services:
- docker:dind
image: docker:latest
script:
# Build the Docker image
- docker build -t myfe:$CI_COMMIT_SHA .
push:
stage: push
image: bitnami/azure-cli
script:
# - echo $DOCKERHUB_PASSWORD | docker login -u $DOCKERHUB_USERNAME --password-stdin
- echo $ACR_CLIENT_ID | docker login mycr.azurecr.io --username $ACR_CLIENT_ID --password-stdin
# Push the Docker image to the ACR
- docker push myfe:$CI_COMMIT_SHA
only:
- main
# before_script:
# - echo "$DOCKERHUB_PASSWORD" | docker login -u "$DOCKERHUB_USERNAME" --password-stdin
variables:
DOCKERHUB_USERNAME: $DOCKERHUB_USERNAME
DOCKERHUB_PASSWORD: $DOCKERHUB_PASSWORD
It results in the following error:
Using docker image sha256:373... for bitnami/azure-cli with digest bitnami/azure-cli#sha256:9128... ...
ERROR: 'sh' is misspelled or not recognized by the system.
Examples from AI knowledge base:
https://aka.ms/cli_ref
Read more about the command in reference docs
Any idea what this might mean?
The bitnami/azure-cli has an entrypoint of az, so your script is running as az parameters.
To solve this, you need to override the entrypoint using: entrypoint: [""] in your gitlab-ci.yml.
For more info check: https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#override-the-entrypoint-of-an-image
If you want to user an Azure CLI image for this .gitlab-ci.yml file, you should use the official Microsoft image instead:
image: mcr.microsoft.com/azure-cli
Works like a charm!
Related
Here is my scenario:
I create a docker image from an SQL dump with the following commands, executed from command prompt:
docker pull mariadb:10.4.26
docker run --name test_smdb -e MYSQL_ROOT_PASSWORD=<some_password> -p 3306:3306 -d mariadb:10.4.26
docker exec -it test_smdb mariadb --user root -p<some_password>
MariaDB [(none)]> CREATE DATABASE smdb_dev;
docker exec -i test_smdb mariadb -uroot -p<some_password> smdb_dev --force < C:\smdb-dev.sql
But my task now is create a pipeline, that creates this docker image and puts it into the Azure Container Registry
I found this link - Build and push Docker images to Azure Container Registry:
https://learn.microsoft.com/en-us/azure/devops/pipelines/ecosystems/containers/acr-template?view=azure-devops
And i see that the result should be a yaml file like this:
- stage: Build
displayName: Build and push stage
jobs:
- job: Build
displayName: Build job
pool:
vmImage: $(vmImageName)
steps:
- task: Docker#2
displayName: Build and push an image to container registry
inputs:
command: buildAndPush
repository: $(imageRepository)
dockerfile: $(dockerfilePath)
containerRegistry: $(dockerRegistryServiceConnection)
tags: |
$(tag)
But can someone show me how to combine the two things - the docker image creation and the putting it into the Azure Container Registry?
You would need to make a dockerfile and put this in the repository.
The commands you specified at the top of your question should be your input
It could look something like this (just threw something together. probably wont work as is):
# syntax=docker/dockerfile:1
FROM mariadb:10.4.26
WORKDIR /app
COPY . .
run --name test_smdb -e MYSQL_ROOT_PASSWORD=<some_password> -p 3306:3306 -d mariadb:10.4.26
run MariaDB [(none)]> CREATE DATABASE smdb_dev;
EXPOSE {mariadb port #}
Hello I have a problem with GitLab CI/CD. I'm trying to deploy container to Kubernetes on GKE however I'm getting an error:
This job failed because the necessary resources were not successfully created.
I created a service account with kube-admin rights and created cluster via GUI of GitLab so its fully itegrated. But when I run the job it still doesn't work..
by the way I use kubectl get pods in gitlab-ci file just to test if kubernetes is repsonding.
stages:
- build
- deploy
docker-build:
# Use the official docker image.
image: docker:latest
stage: build
services:
- docker:dind
before_script:
- docker login -u "$CI_REGISTRY_USER" -p "$CI_REGISTRY_PASSWORD" $CI_REGISTRY
# Default branch leaves tag empty (= latest tag)
# All other branches are tagged with the escaped branch name (commit ref slug)
script:
- docker build --pull -t "$CI_REGISTRY_IMAGE${tag}" .
- docker push "$CI_REGISTRY_IMAGE${tag}"
deploy-prod:
stage: deploy
image: bitnami/kubectl:latest
script:
- kubectl get pods
environment:
name: production
kubernetes:
namespace: test1
Any Ideas?
Thank you
namespace should be removed.
GitLab creates own namespace for every project
I am using gitlab.com ci/cd to push images into gitlab registry. I had 2 repositories already and pushing images into registries as registry.gitlab.com/group1/project1 and registry.gitlab.com/group1/project2.
Now I have another 2 repos up and running. I want to run CI/CD for the new projects. I also want to push images into different registries and names them as follows : registry.gitlab.com/group2/project1 and registry.gitlab.com/group2/project2
And I am getting following error:
denied: requested access to the resource is denied
Does free gitlab provide only 2 registries? Do I need to pay to create more registries ?
The example .gitlab-ci.yml below is to lint, build, test, and tag container images for your GitLab project into the matching image registry. Before you can interact with your image registry, you need to login, see the docker login lines, and the GitLab CI Runner uses a generated token for this, but only matches the current project. And also only when you have enabled the Container Registry in the Setting -> General section
---
image: docker:latest
services:
- docker:dind
stages:
- verify
- build
- test
- release
variables:
CI_DOCKER_IMAGE: ${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA}
CI_DOCKER_TAG: ${CI_REGISTRY_IMAGE}:${CI_COMMIT_TAG}
CI_DOCKER_BRANCH: ${CI_REGISTRY_IMAGE}:${CI_COMMIT_REF_NAME}
CI_DOCKER_LATEST: ${CI_REGISTRY_IMAGE}:latest
Docker lint:
stage: verify
image: projectatomic/dockerfile-lint
script:
- dockerfile_lint -p -f Dockerfile
Docker build:
stage: build
before_script:
- docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
script:
- docker build --pull -t ${CI_DOCKER_IMAGE} .
- docker push ${CI_DOCKER_IMAGE}
Docker test:
stage: test
before_script:
- docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
script:
- docker pull ${CI_DOCKER_IMAGE}
- docker run ${CI_DOCKER_IMAGE} /path/to/test.sh
Release branch:
stage: release
before_script:
- docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
script:
- docker pull ${CI_DOCKER_IMAGE}
- docker tag ${CI_DOCKER_IMAGE} ${CI_DOCKER_BRANCH}
- docker push ${CI_DOCKER_BRANCH}
only:
- branches
except:
- master
Release tag:
stage: release
before_script:
- docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
script:
- docker pull ${CI_DOCKER_IMAGE}
- docker tag ${CI_DOCKER_IMAGE} ${CI_DOCKER_TAG}
- docker push ${CI_DOCKER_TAG}
only:
- tags
Release latest:
stage: release
before_script:
- docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
script:
- docker pull ${CI_DOCKER_IMAGE}
- docker tag ${CI_DOCKER_IMAGE} ${CI_DOCKER_LATEST}
- docker push ${CI_DOCKER_LATEST}
only:
- master
If you want to push to an image registry other than the default registry that comes with your project, then you should provide the credentials to the CI/CD -> Variables section. Also, .gitlab-ci.yml should be modified to have the correct registry, username and accesstoken variables in the YAML file.
I want to deploy a Docker stack from Azure pipelines. I have set some variables, and I am calling these variables in the Docker stack file. However, none of my environment variables are read in the docker stack file. My question: Is there any explanations why I can't read the environment variables in the yaml file?
Below is all my variables
And here is my docker stack configuration
version: "3.1"
services:
postgres:
image: "postgres"
volumes:
- /home/db-postgres:/data/db
environment:
POSTGRES_PASSWORD: ${POSTGRESPASSWORD}
POSTGRES_DB: ${POSTGRESDB}
main:
command: "flask run --host=127.0.0.1"
image: "personal-image"
ports:
- 5000:5000
environment:
SECRET_KEY: ${FLASK_SERIALIZER_SECRET}
JWT_SECRET_KEY: ${FLASK_JWT_SECRET}
FLASK_APP: app.py
MAIL_USERNAME: ${MAIL_USERNAME}
MAIL_PASSWORD: ${MAIL_PASSWORD}
APP_ADMINS: ${APP_ADMINS}
SQLALCHEMY_DATABASE_URI: ${SQLALCHEMY_DATABASE_URI}
From the azure pipeline yaml file, I can read the environment variables though...
What I don't understand is that in an other project, I m doing the exact same thing, and everything works fine.
Edit: Here is my azure-pipelines.yml script. The agent is a self hosted EC2 Linux agent:
steps:
- bash: |
echo $(DOCKERREGISTRY_PASSWORD) | docker login --username $(DOCKERREGISTRY_USER) --password-stdin
displayName: log in to Docker Registry
- bash: |
sudo service docker start
sudo docker stack deploy --with-registry-auth --prune --compose-file stack.staging.yml my_cluster_name
displayName: Deploy Docker Containers
- bash: |
sudo docker system prune --volumes -f
displayName: Clean memory
- bash: |
docker logout
sudo service docker stop
displayName: logout of Docker Registry
You can check the Agent Specification of the yaml pipeline in other projects. They might use the different agents.
I created a test pipeline and found the environment variables in docker stack file were not be able to be substituted in mac or ubuntu agents. But it seemed working in windows agents.
If you used mac or ubuntu agents to run your pipeline. You might need to use define the environment variables in the dockerComposeFileArgs field. See below:
- task: DockerCompose#0
displayName: 'Build services'
inputs:
containerregistrytype: 'Container Registry'
dockerRegistryEndpoint: Dockerhost
dockerComposeFileArgs: |
MAIL_USERNAME=$(MAIL_USERNAME)
MAIL_PASSWORD=$(MAIL_PASSWORD)
APP_ADMINS=$(APP_ADMINS)
SQLALCHEMY_DATABASE_URI=$(SQLALCHEMY_DATABASE_URI)
action: 'Build services'
Update:
For bash task, you can try usig env field to map the variables. See below:
- bash: |
sudo docker stack deploy ...
displayName: 'Bash Script'
enabled: false
env:
MAIL_USERNAME: $(MAIL_USERNAME)
MAIL_PASSWORD: $(MAIL_PASSWORD)
APP_ADMINS: $(APP_ADMINS)
I'm currently trying to setup a gitlab ci pipeline. I've chosen to go with the Docker-in-Docker setup.
I got my ci pipeline to build and push the docker image to the registry of gitlab but I cannot seem deploy it using the following configuration:
.gitlab-ci.yml
image: docker:stable
services:
- docker:dind
stages:
- build
- deploy
variables:
DOCKER_HOST: tcp://docker:2375
DOCKER_DRIVER: overlay2
TEST_IMAGE: registry.gitlab.com/user/repo.nl:$CI_COMMIT_REF_SLUG
RELEASE_IMAGE: registry.gitlab.com/user/repo.nl:release
before_script:
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $CI_REGISTRY
- docker info
build:
stage: build
tags:
- build
script:
- docker build --pull -t $TEST_IMAGE .
- docker push $TEST_IMAGE
only:
- branches
deploy:
stage: deploy
tags:
- deploy
script:
- docker pull $TEST_IMAGE
- docker tag $TEST_IMAGE $RELEASE_IMAGE
- docker push $RELEASE_IMAGE
- docker run -d --name "review-$CI_COMMIT_REF_SLUG" -p "80:80" $RELEASE_IMAGE
only:
- master
when: manual
When I run the deploy action I actually get the following feedback in my log, but when I go check the server there is no container running.
$ docker run -d --name "review-$CI_COMMIT_REF_SLUG" -p "80:80" $RELEASE_IMAGE
7bd109a8855e985cc751be2eaa284e78ac63a956b08ed8b03d906300a695a375
Job succeeded
I have no clue as to what I am forgetting here. Am I right to expect this method to be correct for deploying containers? What am I missing / doing wrong?
tldr: Want to deploy images into production using gitlab ci and docker-in-docker setup, job succeeds but there is no container. Goal is to have a running container on host after deployment.
Found out that I needed to include the docker socket in the gitlab-runner configuration as well, and not only have it available in the container.
By adding --docker-volumes '/var/run/docker.sock:/var/run/docker.sock' and removing DOCKER_HOST=tcp://docker:2375 I was able to connect to docker on my host system and spawn sibling containers.