I want to build and add a custom image (with ruby, node.js, bower, grunt, jekyll etc.) and tag it as 'myimage:1.0'. This image needs to be stored in gitlab container registry and then used in .gitlab-ci.yml as image: sachin.1.0.0. So that my build via gitlab ci will have everything preinstalled like node.js, etc.
Tried enough, How can this be done ?
Before you do this, you need to configure a gitlab runner which allows you to use docker build. You can configure this using the instructions here depending on your use case
Next, create a new repo in gitlab, let's call it sachin-image.
Inside the root of the git repo, add a Dockerfile with installation of everything you need.
Now, into this repo, add a .gitlab-ci.yml file like so:
---
before_script:
- docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN <my-docker-gitlab-registry-url>
stages:
- build
build_image:
stage: build
script:
- docker build -t gitlab.example.com/my/dockerimage/repo:latest .
- docker push gitlab.example/my/dockerimage/repo:latest
tags:
- docker_engine
At this point, you now have automated docker builds working in gitlab. In order to use this image in future gitlab builds, all you need to use the following image url:
gitlab.example.com/my/dockerimage/repo:latest
Related
I have a remote server where I serve my project via Nginx. I am using Gitlab CI to automatize my deploy process and I have ran into a problem. When I push my commits to the master branch the gitlab-runner run nicely but the problem is that it removes my React build folder (it is ok, as I have put it into the .gitignore), but because it always removes my build folder my Nginx could not serve any files until the build finish, and a new build folder creaeted. Is is there any solution for this problem? It would be nice if I could cache my build file until the build process finish. I attached my gitlab.ci.yml. Thank's in advance!
image: docker:latest
services:
- docker:dind
stages:
- build
- deploy
variables:
GIT_SSL_NO_VERIFY: "1"
build-step:
stage: build
tags:
- shell
script:
- docker image prune -f
- docker-compose -f docker-compose.yml -f docker-compose.prod.yml build
deploy-step:
stage: deploy
tags:
- shell
script:
- docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d
It should be possible to use git fetch and to disable git clean when your deploy job starts. Here are links for the variables to do this:
https://docs.gitlab.com/ee/ci/yaml/#git-clean-flags
https://docs.gitlab.com/ee/ci/yaml/#git-strategy
It would look something like this:
deploy-step:
variables:
GIT_STRATEGY: fetch
GIT_CLEAN_FLAGS: none
stage: deploy
tags:
- shell
script:
- docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d
This should make GitLab use git fetch instead of git clone, and to not run any git clean ... commands. The build artifacts from your previous run should then not be removed.
There are some problems with this though. If something goes wrong with a build, you might end up in a situation where you will have to manually log into the server where the runner is to fix it. The reason that GitLab uses git clean is to prevent these types of problems.
A more proper solution is to use nginx to have a sort of dubble buffer. You can have two different build folders, change the config in nginx, and then send a signal to nginx to reload the config. nginx will then make sure to gracefully switch to the new version of your application, without any interruptions. Here is a link to someone that has done this:
https://syshero.org/2016-06-09-zero-downtime-deployments-using-nginx/
When pushing a docker image with a modified tag (to contain registry) to the gitlab integrated registry i get an access denied.
Using the gitlab registry is using it per project. Once the registry is enabled for a project there is a hint how to push the images to the registry https://gitlab.mydomain.com/**path/to/project**/container_registry.
The problem got solved when the full path was included in the TAG Name.
When i changed the tagname to [registryUrl]:[registryPort]/path/to/project/[imageNameWithTags] i was able to push to the repository/registry.
Indeed you need to do docker login ... as described on the /container_registry page.
You can also rely on some GitLab Predefined environment variables to make code generic and re-use it in many projects.
Here is the example of doing it in .gitlab-ci.yml:
build-image:
stage: build
image: docker:latest
services:
- name: docker:dind
script:
- docker build -t $CI_REGISTRY_IMAGE .
- docker login -u $CI_REGISTRY_USER -p "$CI_JOB_TOKEN" $CI_REGISTRY
- docker push $CI_REGISTRY_IMAGE
See full example in one of our projects
I am using Gitlab for repository and ci/cd.
Currently trying to set up pipeline that creates a docker image from a build stage.
All examples I have seen have a simple naming of image where e.g. the branch is used(master)
My question is if I want to tag the image based on the current tag in the repository how do I do this?
I am presuming I can use a Gitlab runner variable but do not see one to us
There are a lot of predefined variables in Gitlab CI. I think you are looking for CI_COMMIT_TAG.
So you could use it this way:
docker build -t $CI_REGISTRY_IMAGE:$CI_COMMIT_TAG
So the image would look like registry.example.com/group/project:tag
As shell command is not possible yet with variables inside .gitlab-ci.yml, you may edit a build script that get the current tag and build the image inside that script
Both file at the root of your project :
build.sh :
#!/bin/sh
IMAGE="$CI_REGISTRY/$CI_PROJECT_PATH:$CI_COMMIT_REF_NAME-$(git describe --abbrev=0 --tags)"
docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY
docker build --pull -t $IMAGE .
docker push $IMAGE
.gitlab-ci.yml :
image: docker:latest
services:
- docker:dind
stages:
- release
release:
stage: release
script:
- apk update && apk add git
- ./build.sh
I already did CI, but now I want to deploy to my server. My server is the same machine where I do CI, but I do CI in docker-executor. So I can't have acces to server folders to update production.
There is my script:
image: node:9.11.2
cache:
paths:
- node_modules/
before_script:
- npm install
stages:
- test
- deploy
test:
stage: test
script:
- npm run test
deploy:
stage: deploy
script:
#here I want to go to /home/projectFolder and make git pull, npm i, npm start
# but I can't beause I run CI in docker-environment which hasn't acces to my server's dirictories.
First of all you should consider using gitlab auto cicd ( or use it as a base to customize if you dont want to use kubernetes)
You have multiple way to do so but the simplest way should be to use an alpine image and
- install ssh (if necessary)
- load your private ssh key ( from pipeline secrets)
- run your npm commands through ssh.
The cleanest way would be :
- generating adding a valid Dockerfile to your project
- adding docker image generation for each commit on master (in your pipeline)
- Adding docker rm running image (in your pipeline)
- Adding docker run the newly generated image (in your pipeline) (by sharing your docker volume)
- Make nginx redirect to your container.
I can give more detailed advice depending on what you decide to do.
Hoping i helped.
How should I authenticate if I want to use an image from the Gitlab Registry as a base image of another CI build?
According to https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/blob/master/docs/configuration/advanced-configuration.md#using-a-private-docker-registry I first have to manually login on the runner machine. Somehow it feels strange to login with an existing Gitlab user.
Is there a way to use the CI variable "CI_BUILD_TOKEN" (which is described as "Token used for authenticating with the GitLab Container Registry") for authentication to pull the base image from Gitlab Registry?
EDIT: I found out that I can use images from public projects. But I don't really want to make my docker projects public.
UPDATE: Starting with Gitlab 8.14 you can just use the docker images from the build in docker registry. See https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/blob/master/docs/configuration/advanced-configuration.md#support-for-gitlab-integrated-registry
All of the above answers including the acepted one are deprecated, This is possible in 2021:
https://docs.gitlab.com/ee/ci/docker/using_docker_images.html#access-an-image-from-a-private-container-registry
TL;DR
Set the CI/CD variable DOCKER_AUTH_CONFIG value with appropriate authentication information in following format:
Step 1:
# The use of "-n" - prevents encoding a newline in the password.
echo -n "my_username:my_password" | base64
# Example output to copy
bXlfdXNlcm5hbWU6bXlfcGFzc3dvcmQ=
Step 2 (This JSON is the value to be set for DOCKER_AUTH_CONFIG variable):
{
"auths": {
"registry.example.com:5000": {
"auth": "(Base64 content from above)"
}
}
}
Now it's possible, they have included that option months ago.
Use gitlab-ci-tokenas user and the variable $CI_BUILD_TOKEN as password.
This example works on GitLab 8.13.6. It builds the test image if needed, and in the next stage uses it to perform syntax checks:
build_test:
stage: build_test_image
script:
- docker login -u gitlab-ci-token -p $CI_BUILD_TOKEN $CI_REGISTRY
- docker build -t $CI_REGISTRY_IMAGE:test -f dockerfiles/test/Dockerfile .
- docker push $CI_REGISTRY_IMAGE:test
tags:
- docker_build
environment: test
test_syntax:
image: $CI_REGISTRY_IMAGE:test
stage: test
script:
- flake8 --ignore=E501,E265,E402 .
UPDATE: Re-reading the question, the accepted answer is correct. In my example, the job test_syntax will fail to authenticate to the registry, unless the user logins manually from the runner machine. Although, it can work if the 2 runners are on the same host, but it's not the best solution anyway.
In gitlab-ci-multi-runner 1.8 there's an option to add the Registry credentials as a variable, so you only need to login once to get the encoded credentials. See documentation.
No, this is currently not possible in any elegant way. GitLab should implement explicit credentials for the base images, it will be the most straight-forward and correct solution.
You need to docker login on the GitLab Runner machine. You can't use the gitlab-ci-token since they expire and also project-dependant, so you can't actually use one token for every project. Using your own login is pretty much the only solution available right now (happy to get corrected on this one).
This is absolutely possible as of September 2018. I'll post my naive implementation here.
Context:
You'll need to leverage the docker:dind service, which lets you run the docker command inside of a docker container.
This will require you to use a valid docker login, which you can do using GitLab's builtin variables (gitlab-ci-token, $CI-JOB-TOKEN).
You should then be able to authenticate to your repo's registry (example $REGISTRY value: registry.gitlab.com/$USER/$REPO:$TAG), which will allow you to push or pull docker containers from inside the CI/CD context, as well as from any authenticated docker server.
Implementation:
Create this block at top level to ensure it runs before the following jobs:
before_script:
- docker login -u gitlab-ci-token -p $CI_JOB_TOKEN $REGISTRY
Job to build and save images to your registry:
build_container:
image: docker:latest
stage: build
services:
- docker:dind
script:
- docker build -t $REGISTRY .
- docker push $REGISTRY
Job that uses the custom image:
build_app:
image: $REGISTRY
stage: deploy
script:
- npm run build
Regarding Cross-Repo Jobs:
I accomplish this by creating a "bot" GitLab user and assigning them access to repos/groups as appropriate. Then it's just a matter of replacing gitlab-ci-token and $CI_JOB_TOKEN with appropriate environment variables. This is only necessary if the base image is private.
Its possible you first have to login to gitlab container registry of the image you want to use, kindly see below example. Notice the
before_script: which basically auths you before using the image.
image: docker:latest
services:
- docker:dind
stages:
- build
variables:
CONTAINER_RELEASE_IMAGE: registry.gitlab.com/obonyojimmy/node-mono-clr:latest
before_script:
- docker login -u $CI_REGISTRY_USER -p $CI_BUILD_TOKEN registry.gitlab.com
build-app:
stage: build
image: $CONTAINER_RELEASE_IMAGE
script:
- npm run build
I had a similar situation. My Java application uses Testcontainers lib in tests and this lib runs Docker container from private registry. I spent a lot of time trying to figure this out and I managed to handle this by creating a ~/.docker/config.json file in before_script section. I hope it'll help somebody:
image: openjdk:11-jdk-slim
stages:
- build
before_script:
- mkdir ~/".docker"
- echo "{\"auths\":{\"$REGISTRY_HOST\":{\"auth\":\"$(printf "$REGISTRY_USER:$REGISTRY_PASSWORD" | openssl base64 -A)\"}}}" > ~/".docker/config.json"
build:
stage: build
services:
- docker:dind
script:
- ./gradlew build