I am trying to use Kaniko with Gitlab in order to get rid of the DinD flow.
So, I have this in my .gitlab-ci.yaml
kaniko:
stage: tagging
variables:
CI_REGISTRY: ${AZURE_REGISTRY_USERNAME_DEV}.azurecr.io
CI_REGISTRY_USER: ${AZURE_REGISTRY_USERNAME_DEV}
CI_REGISTRY_PASSWORD: ${AZURE_REGISTRY_PASS_DEV}
image:
name: gcr.io/kaniko-project/executor:debug
entrypoint: [""]
script:
#
- mkdir -p /kaniko/.docker
- echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(printf "%s:%s" "${CI_REGISTRY_USER}" "${CI_REGISTRY_PASSWORD}" | base64 | tr -d '\n')\"}}}" > /kaniko/.docker/config.json
- >-
/kaniko/executor
--context "${CI_PROJECT_DIR}"
--dockerfile "${CI_PROJECT_DIR}/devops/Dockerfile"
--destination "${CI_REGISTRY}/kanikotest:bla"
--verbosity debug
tags: # select gitlab-runner based on this tag(s)
- docker
only:
refs:
- /^feat.*$/
but I keep getting this error in the logs
error checking push permissions -- make sure you entered the correct tag name, and that you are authenticated correctly, and try again: checking push permission for "mysuperregistry.azurecr.io/kanikotest:bla": creating push check transport for mysuperregistry.azurecr.io failed: GET https://mysuperregistry.azurecr.io/oauth2/token?scope=repository%3Akanikotest%3Apush%2Cpull&service=mysuperregistry.azurecr.io: UNAUTHORIZED: authentication required, visit https://aka.ms/acr/authorization for more information.
I am following this guide.
Fun fact... I have successfully deployed Kaniko inside Minikube by creating a secret with the same creds, and I successfully pushed to the same registry.
The syntax of the auth file seems good (I assume the creds are correct), so your code should work if you just set the DOCKER_CONFIG environment variable as following:
kaniko:
stage: tagging
variables:
CI_REGISTRY: ${AZURE_REGISTRY_USERNAME_DEV}.azurecr.io
CI_REGISTRY_USER: ${AZURE_REGISTRY_USERNAME_DEV}
CI_REGISTRY_PASSWORD: ${AZURE_REGISTRY_PASS_DEV}
DOCKER_CONFIG: "$CI_PROJECT_DIR/kanikotest/.docker"
image:
name: gcr.io/kaniko-project/executor:debug
entrypoint: [""]
script:
- mkdir -p $DOCKER_CONFIG
- echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(printf "%s:%s" "${CI_REGISTRY_USER}" "${CI_REGISTRY_PASSWORD}" | base64 | tr -d '\n')\"}}}" > $DOCKER_CONFIG/config.json
- >-
/kaniko/executor
--context "${CI_PROJECT_DIR}"
--dockerfile "${CI_PROJECT_DIR}/devops/Dockerfile"
--destination "${CI_REGISTRY}/kanikotest:bla"
--digest-file "$CI_PROJECT_DIR/docker-content-digest-kanikotest"
--verbosity info
artifacts:
paths:
- docker-content-digest-kanikotest
Adding an extra directory (kanikotest) inside the DOCKER_CONFIG path will avoid concurrent builds to overwrite the same auth file (not required in your case example but a good practice in general).
The --digest-file option will permit also to save the image SHA for following CI jobs.
Related
I have the following pipeline:
# .gitlab-ci.yml
stages:
- build
- push
build:
stage: build
services:
- docker:dind
image: docker:latest
script:
# Build the Docker image
- docker build -t myfe:$CI_COMMIT_SHA .
push:
stage: push
image: bitnami/azure-cli
script:
# - echo $DOCKERHUB_PASSWORD | docker login -u $DOCKERHUB_USERNAME --password-stdin
- echo $ACR_CLIENT_ID | docker login mycr.azurecr.io --username $ACR_CLIENT_ID --password-stdin
# Push the Docker image to the ACR
- docker push myfe:$CI_COMMIT_SHA
only:
- main
# before_script:
# - echo "$DOCKERHUB_PASSWORD" | docker login -u "$DOCKERHUB_USERNAME" --password-stdin
variables:
DOCKERHUB_USERNAME: $DOCKERHUB_USERNAME
DOCKERHUB_PASSWORD: $DOCKERHUB_PASSWORD
It results in the following error:
Using docker image sha256:373... for bitnami/azure-cli with digest bitnami/azure-cli#sha256:9128... ...
ERROR: 'sh' is misspelled or not recognized by the system.
Examples from AI knowledge base:
https://aka.ms/cli_ref
Read more about the command in reference docs
Any idea what this might mean?
The bitnami/azure-cli has an entrypoint of az, so your script is running as az parameters.
To solve this, you need to override the entrypoint using: entrypoint: [""] in your gitlab-ci.yml.
For more info check: https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#override-the-entrypoint-of-an-image
If you want to user an Azure CLI image for this .gitlab-ci.yml file, you should use the official Microsoft image instead:
image: mcr.microsoft.com/azure-cli
Works like a charm!
I am creating a pipeline in Gitlab and need to make use of the image gcr.io/kaniko-project/executor:debug. My problem is that I cannot use artifacts because of some dns related issues so I need to create a custom Dockerfile that includes git. I have come across a few example Dockerfiles, which I have tested by building the Dockerfile and pushing to AWS ECR then using that image in the Gitlab job but each are outputting the same error in the pipeline:
exec /bin/sh: exec format error
Dockerfile:
FROM gcr.io/kaniko-project/executor:debug AS kaniko
FROM alpine:3.14.2
RUN apk --update add git
# RUN setcap cap_ipc_lock= /usr/sbin/vault
COPY --from=kaniko /kaniko/ /kaniko/
ENV PATH $PATH:/usr/local/bin:/kaniko
ENV DOCKER_CONFIG /kaniko/.docker/
ENV DOCKER_CREDENTIAL_GCR_CONFIG /kaniko/.config/gcloud/docker_credential_gcr_config.json
ENV SSL_CERT_DIR /kaniko/ssl/certs
ENTRYPOINT ["/kaniko/executor"]
Gitlab CI Job:
build dockerfile:
stage: build
image:
name: $ECR_REGISTRY/$ECR_REPO:0.0.8
entrypoint: [""]
before_script:
- git --version
script:
- mkdir -p /kaniko/.docker
- echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(printf "%s:%s" "${REGISTRY_USERNAME}" "${REGISTRY_PASSWORD}" | base64 | tr -d '\n')\"}}}" > /kaniko/.docker/config.json
- >-
/kaniko/executor
--context "${CI_PROJECT_DIR}"
--dockerfile "${CI_PROJECT_DIR}/Dockerfile"
--destination "$ECR_REGISTRY/$ECR_REPO:$nextReleaseVersion"
Can anyone suggest what I am doing wrong?
I have a Gitlab pipeline that is failing when it attempts docker build (using Kaniko)
I am yet to do a successful docker build BUT this particular error has started after I :
Changed the kaniko image from image: gcr.io/kaniko-project/executor:debug to gcr.io/kaniko-project/executor:51734fc3a33e04f113487853d118608ba6ff2b81
Added settings for pushing to insecure registries :
--insecure
--skip-tls-verify
--skip-tls-verify-pull
--insecure-pull
After this change part of the pipeline looks like this :
before_script:
- 'dotnet restore --packages $NUGET_PACKAGES_DIRECTORY'
build_job:
tags:
- xxxx
only:
- develop
stage: build
script:
- dotnet build --configuration Release --no-restore
publish_job:
tags:
- xxxx
only:
- develop
stage: publish
artifacts:
name: "$CI_COMMIT_SHA"
paths:
- ./$PUBLISH_DIR
script:
- dotnet publish ./src --configuration Release --output $(pwd)/$PUBLISH_DIR
docker_build_dev:
tags:
- xxxx
image:
name: gcr.io/kaniko-project/executor:51734fc3a33e04f113487853d118608ba6ff2b81
entrypoint: [""]
only:
- develop
stage: docker
before_script:
- echo "Docker build"
script:
- echo "${CI_PROJECT_DIR}"
- cp ./src/Dockerfile /builds/xxx/xxx/xxx/Dockerfile
- mkdir -p /kaniko/.docker
- echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$(printf "%s:%s" "${CI_REGISTRY_USER}" "${CI_REGISTRY_PASSWORD}" | base64 | tr -d '\n')\"}}}" > /kaniko/.docker/config.json
- >-
/kaniko/executor
--context "${CI_PROJECT_DIR}"
--insecure
--skip-tls-verify
--skip-tls-verify-pull
--insecure-pull
--dockerfile "${CI_PROJECT_DIR}/Dockerfile"
--destination "${CI_REGISTRY_IMAGE}:${CI_COMMIT_TAG}"
Part of the output from the pipeline is as below:
[32;1mSkipping Git submodules setup[0;m
section_end:1652535765:get_sources
[0Ksection_start:1652535765:download_artifacts
[0K[0K[36;1mDownloading artifacts[0;m[0;m
[32;1mDownloading artifacts for publish_job (33475)...[0;m
Downloading artifacts from coordinator... ok [0;m id[0;m=33475 responseStatus[0;m=200 OK token[xxxxxxxxxxx
section_end:1652535769:download_artifacts
[0Ksection_start:1652535769:step_script
[0K[0K[36;1mExecuting "step_script" stage of the job script[0;m[0;m
section_end:1652535769:step_script
[0Ksection_start:1652535769:cleanup_file_variables
[0K[0K[36;1mCleaning up project directory and file based variables[0;m[0;m
section_end:1652539354:cleanup_file_variables
[0K[31;1mERROR: Job failed: execution took longer than 1h0m0s seconds
[0;m
What am I missing?
I was missing something in my GitLab project settings which is enabling Project Registries :
https://domain/group/subgroup/project/edit
Visibility,project features,permissions << Container Registry (Toggle to enable)
I want to use Testcontainers for my JUNIT tests and so I created this:
image: gitlab.registry.example:5005/my-custom-maven-image
variables:
MAVEN_CLI_OPTS: "--batch-mode -s $CI_PROJECT_DIR/.m2/settings.xml"
stages:
- test
test:
stage: test
script:
- mvn $MAVEN_CLI_OPTS clean test
services:
- name: docker:dind
alias: docker
command:
- /bin/sh
- -c
- "DOCKER_AUTH_CONFIG=`echo \"{\\\"auths\\\":{\\\"$CI_REGISTRY\\\":{\\\"username\\\":\\\"$CI_REGISTRY_USER\\\",\\\"password\\\":\\\"$CI_REGISTRY_PASSWORD\\\"}}}\"` && mkdir -p \"/root/.docker\" && echo \"${DOCKER_AUTH_CONFIG}\" > \"/root/.docker/config.json\" && cat /root/.docker/config.json && update-ca-certificates && dockerd-entrypoint.sh || exit"
variables:
# Instruct Testcontainers to use the daemon of DinD.
DOCKER_HOST: "tcp://docker:2375"
# Instruct Docker not to start over TLS.
DOCKER_TLS_CERTDIR: ""
DOCKER_TLS_VERIFY: 0
# Improve performance with overlayfs.
DOCKER_DRIVER: overlay2
This gives me the following output when the runner tries to spawn the dind container:
{"auths":{"gitlab.registry.example:5005":{"username":"gitlab-ci-token","password":""}}}
As you can see the password is empty. Printing the CI_REGISTRY_PASSWORD variable in a before_script shows me [masked] as I would expect.
I am about to create an issue in the gitlab-runner project but I wanted to make sure what I did is not wrong beforehand.
Update: Created an issue in the gitlab-runner project
It looks like a bug indeed: CI_REGISTRY_PASSWORD variable is not present at all in the container running DinD service, where it's properly set in job container.
I reproduced your issue by re-using your example in a simplified way:
test:
stage: test
script:
- echo "Registry $CI_REGISTRY - User $CI_REGISTRY_USER - Password $CI_REGISTRY_PASSWORD"
# - sleep 9999
services:
- name: docker:dind
alias: docker
command:
- /bin/sh
- -c
- echo "Registry $CI_REGISTRY - User $CI_REGISTRY_USER - Password $CI_REGISTRY_PASSWORD" && dockerd-entrypoint.sh || exit
This shows in Gitlab UI:
# Services logs (not always shown)
Registry registry.novadiscovery.net - User gitlab-ci-token - Password
# Script logs
$ echo "Registry $CI_REGISTRY - User $CI_REGISTRY_USER - Password $CI_REGISTRY_PASSWORD"
Registry registry.mycompany.com - User gitlab-ci-token - Password [MASKED]
At first I thought the variable was somehow hidden from Gitlab log UI, but instead of being shown as [masked] it was simply not shown at all. However, when inspecting underlying containers running jobs, we can see variable is indeed absent from DinD service:
# Running docker inspect command on machine running Gitlab Runner
# Inspect DinD service container
# CI_REGISTRY_PASSWORD does not exists
docker inspect runner-zz-qri9h-project-663-concurrent-0-8f92ad27e7b78f1c-docker-0 | jq .[0].Config.Env | grep CI_REGISTRY
"CI_REGISTRY_USER=gitlab-ci-token",
"CI_REGISTRY=registry.mycompany.com",
"CI_REGISTRY_IMAGE=registry.mycompany.com/pierre.beucher/sandbox",
# Inspect job container
# CI_REGISTRY_PASSWORD is set
docker inspect runner-zz-qri9h-project-663-concurrent-0-8f92ad27e7b78f1c-build-2 | jq .[0].Config.Env | grep CI_REGISTRY
"CI_REGISTRY_USER=gitlab-ci-token",
"CI_REGISTRY_PASSWORD=xxx",
"CI_REGISTRY=registry.mycompany.com",
"CI_REGISTRY_IMAGE=registry.mycompany.com/pierre.beucher/sandbox",
By comparing variables between job container and service container, it seems all secret or sensible pre-defined CI variables are missing from the services containers. From the above comparison, the following variables were missing (there may be others):
CI_JOB_TOKEN
CI_BUILD_TOKEN
CI_REGISTRY_PASSWORD
CI_REPOSITORY_URL
CI_DEPENDENCY_PROXY_PASSWORD
CI_JOB_JWT
Tested on Gitlab 13.11.3 and Gitlab Runner 13.2.1
I am using gitlab.com ci/cd to push images into gitlab registry. I had 2 repositories already and pushing images into registries as registry.gitlab.com/group1/project1 and registry.gitlab.com/group1/project2.
Now I have another 2 repos up and running. I want to run CI/CD for the new projects. I also want to push images into different registries and names them as follows : registry.gitlab.com/group2/project1 and registry.gitlab.com/group2/project2
And I am getting following error:
denied: requested access to the resource is denied
Does free gitlab provide only 2 registries? Do I need to pay to create more registries ?
The example .gitlab-ci.yml below is to lint, build, test, and tag container images for your GitLab project into the matching image registry. Before you can interact with your image registry, you need to login, see the docker login lines, and the GitLab CI Runner uses a generated token for this, but only matches the current project. And also only when you have enabled the Container Registry in the Setting -> General section
---
image: docker:latest
services:
- docker:dind
stages:
- verify
- build
- test
- release
variables:
CI_DOCKER_IMAGE: ${CI_REGISTRY_IMAGE}:${CI_COMMIT_SHA}
CI_DOCKER_TAG: ${CI_REGISTRY_IMAGE}:${CI_COMMIT_TAG}
CI_DOCKER_BRANCH: ${CI_REGISTRY_IMAGE}:${CI_COMMIT_REF_NAME}
CI_DOCKER_LATEST: ${CI_REGISTRY_IMAGE}:latest
Docker lint:
stage: verify
image: projectatomic/dockerfile-lint
script:
- dockerfile_lint -p -f Dockerfile
Docker build:
stage: build
before_script:
- docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
script:
- docker build --pull -t ${CI_DOCKER_IMAGE} .
- docker push ${CI_DOCKER_IMAGE}
Docker test:
stage: test
before_script:
- docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
script:
- docker pull ${CI_DOCKER_IMAGE}
- docker run ${CI_DOCKER_IMAGE} /path/to/test.sh
Release branch:
stage: release
before_script:
- docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
script:
- docker pull ${CI_DOCKER_IMAGE}
- docker tag ${CI_DOCKER_IMAGE} ${CI_DOCKER_BRANCH}
- docker push ${CI_DOCKER_BRANCH}
only:
- branches
except:
- master
Release tag:
stage: release
before_script:
- docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
script:
- docker pull ${CI_DOCKER_IMAGE}
- docker tag ${CI_DOCKER_IMAGE} ${CI_DOCKER_TAG}
- docker push ${CI_DOCKER_TAG}
only:
- tags
Release latest:
stage: release
before_script:
- docker login -u gitlab-ci-token -p ${CI_JOB_TOKEN} ${CI_REGISTRY}
script:
- docker pull ${CI_DOCKER_IMAGE}
- docker tag ${CI_DOCKER_IMAGE} ${CI_DOCKER_LATEST}
- docker push ${CI_DOCKER_LATEST}
only:
- master
If you want to push to an image registry other than the default registry that comes with your project, then you should provide the credentials to the CI/CD -> Variables section. Also, .gitlab-ci.yml should be modified to have the correct registry, username and accesstoken variables in the YAML file.