I'm using GitLab pipelines and have defined my build defintion in the .gitlab-ci.yml file.
I'm using this to build docker containers.
Simple question. Is there a way I can tag my docker containers with either a semver from gitlab or a timestamp.
The build-in variables don't seem to give me much to work with.
On Windows I've been able to use GitVersion before in powershell that gets the semver tag and puts it into a variable you can use in the rest of the build process.
If I can't do that, is it possible to get a timestamp from the OS and use that in the yml file?
You can use the timestamp in your .gitlab-ci.yml like this (taken from our own usage creating <year>.<month> tags and releases:
job-1:
script:
- export VERSION=$(date +%y.%m)
- docker build -t myregistry/project/image:$VERSION
This results in a image tag like: myregistry/project/image:17.10
You can use date +%s instead of date +%y.%m for unixtimestamp.
Depending on your (git)flow you can also use branch-slugs provided by Gitlab CI env vars
Regarding timestamp, another approach is to use existing variables associated to your current pipeline.
See GitLab 13.10 (March 2021)
Predefined CI/CD variables for job start and pipeline created timestamps
Previously, if you wanted to reference the exact date and time when a job started
or a pipeline was created, you needed to retrieve these timestamps in your scripts.
Now they are readily available as predefined CI/CD variables by using CI_JOB_STARTED_AT and
CI_PIPELINE_CREATED_AT, provided in the ISO 8601 format and UTC time zone.
Thanks to #Winkies for this contribution!
See Documentation and Issue.
Unfortunately this variable can't be used directly as an image tag. As seen in the referenced implementation issue, the output of this variable is similar to 2021-03-18T04:45:29Z. Used directly, your image would look something like myimage:2021-03-18T04:45:29Z which is invalid.
Related
I'd like to access one of the gitlab CI/CD pre-defined variables, but when I try to use it, I just get the literal string of my attempt to access it. Per Gitlab's documentation, the correct syntax is "$CI_COMMIT_BRANCH", but this isn't working. I also tried creating my own variables, but the same thing is happening. Here are some screenshots of my .gitlab-ci.yml file and the results of it running:
You are using the cmd batch shell. As described in the documentation, you must use %VARNAME% instead of $VARNAME in your script: sections.
script:
- echo %CI_COMMIT_BRANCH%
All depends on the pipeline flow and the version of the runner and even it's OS.
I see that you have a Windows build so you should use POSH/Batch commands not *nix shell ones.
The commit branch name. Available in branch pipelines, including pipelines for the default branch. Not available in merge request pipelines or tag pipelines.
https://docs.gitlab.com/ee/ci/variables/predefined_variables.html
If I remember correct if it can't interpolate "a variable" i.e. if it doesn't exist in the scope of the script at execution time, it just treats it as a string and prints to screen everything after echo which is the behavior you are seeing.
I solved this by calling the variable the same way as one would in the CMD:
echo %CI_COMMIT_BRANCH%
I'm using Gitlab CI/CD to deploy my .netcore application to AWS Beanstalk.
How can I update the appSettings.json values in my .netcore application when deploying to different environments using variables defined in my CI/CD pipeline settings?
Azure DevOps has a JSON variable substitution feature which I really liked.
GitHub Actions can also hook into this feature
How can I achieve this with Gitlab CI/CD?
I want to use this method of deployment because
I won't have to store sensitive production config values in my repository. Values will be updated by Masked variables setup in the CI/CD Pipeline.
I don't have to rebuild my artefacts every time I deploy to a new environment.
If this can't be done in Gitlabs, whats the recommended best practice?
thanks
I do something similar here with gitlab and the solution was to build a shell script that replaces some string from variable values before starting the deploy job
something like this
script:
- sed -i 's/STRING_TO_REPLACE/$GITLAB_VARIABLE/g' file.json
Just pay attention to escape json quotes correctly to make this possible
I use the same workaround. I filed an issue about this months ago:
https://gitlab.com/gitlab-org/gitlab-foss/-/issues/78486
and I recently talked about this at the gitlab forum:
https://forum.gitlab.com/t/use-variables-per-branch-in-gitlab/43685
unfortunately there does not seem to be a nice/clean solution at the moment.
so I use the workarround:
1: use environment scopes for variables so the same variable can have different values (for test/prod environments)
2: define the environments and branches in the jobs in gitlab-ci.yml
3: create a bunch of sed one liners to do a search/replace..
call it ugly, call it low level but I searched for a nice/clean/gitlab native solution an did not find it.
Is there any unique number in Gitlab CI which can be used as build number as we use in Jenkins. I came to know about the variable "CI_PIPELINE_IID" but the problem with this variable is, it updates for all branches and there is no such variable exist for each branch.
We solved the same issue using the variable $CI_PIPELINE_IID, which is defined as
The project-level IID (internal ID) of the current pipeline. This ID is unique only within the current project.
And is quite similar to Jenkins' $BUILD_NUMBER
See also https://docs.gitlab.com/ee/ci/variables/predefined_variables.html#variables-reference
It looks like CI_COMMIT_SHA or CI_COMMIT_SHORT_SHA are great candidates for this as they give you a reference to the commit it was built from.
For example, if you want to use it as a docker image tag, use
docker build . -t myapp:$CI_COMMIT_SHA
Note that earlier versions of Gitlab (version 8.x) use CI_BUILD_TAG
More variables on: https://docs.gitlab.com/ee/ci/variables/predefined_variables.html#variables-reference
You can check if any of the other ids "CI variables" might work in your case:
CI_JOB_ID 9.0 all The unique id of the current job that GitLab CI uses internally
CI_PIPELINE_ID 8.10 all The unique id of the current pipeline that GitLab CI uses internally (one I)
CI_PROJECT_ID all all The unique id of the current project that GitLab CI uses internally
CI_RUNNER_ID 8.10 0.5 The unique id of runner being used
Yeah,it's a reasonable request for many situations. Someone has issued it before, please refer to: https://gitlab.com/gitlab-org/gitlab/-/issues/23844. But it seems that it has not added this so far.
I am trying to create a JS utility to version stamp a VSTS build with details about the branch and commit id.
I have been using git-rev-sync which works fine locally. However, when the code is checked out using a VSTS build definition, the repo is detached, and I am no longer able to determine from the git repo itself which branch the current code belongs to.
git-rev-sync reports something along the lines of:
Detatched: 705a3e89206882aceed8c9ea9b2f412cf26b5e3f
Instead of "develop" or "master"
I may look at the vsts node SDK which might be able to pick up VSTS environmental variables like you can with Powershell scripts.
Has anyone done this or solved this problem in a neater way?
The build variables will be added to current process’s environment variables, so you can access Build.SourceBranchName build-in variable from environment variable:
PowerShell:
$env:BUILD_SOURCEBRANCHNAME
NodeJS:
process.env.BUILD_SOURCEBRANCHNAME
Shell script:
$BUILD_SOURCEBRANCHNAME
Batch script:
%BUILD_SOURCEBRANCHNAME%
You also can pass it through argument of task ($(Build.SourceBranchName)), for example, using Replace Tokens task to replace variable value to a file, then you can read the value from the file (replace %{BUILD_SOURCEBRANCHNAME}%).
I can't find out how to access variables in a build-script provided by the gitlab-ci.yml-file.
I have tried to declare variables in two ways:
Private Variables in the Web-Interface of GitLab CI
Variable overrides/apennding in config.toml
I try to access them in my gitlab-ci.yml-files commands like that:
msbuild ci.msbuild [...] /p:Configuration=Release;NuGetOutputDir="$PACKAGE_SOURCE"
where $PACKAGE_SOURCE is the desired variable (PACKAGE_SOURCE) but it does not work (it does not seem to be replaced). Executing the same command manually works just as expected (replacing the variable name with its content)
Is there some other syntax required i am not aware of?
I have tried:
$PACKAGE_SOURCE
$(PACKAGE_SOURCE)
${PACKAGE_SOURCE}
PS: Verifying the runner raises no issues, if this matters.
I presume you are using Windows for your runner? I was having the same issue myself and couldn't even get the following to work:
script:
- echo $MySecret
However, reading the Gitlab documentation it has an entry for the syntax of environment variables in job scripts:
To access environment variables, use the syntax for your Runner’s shell
Which makes sense as most of the examples given are for bash runners. For my windows runner, it uses %variable%.
I changed my script to the following, which worked for me. (Confirmed by watching the build output.)
script:
- echo %MySecret%
If you're using the powershell for your runner, the syntax would be $env:MySecret
In addition to what was said in the answer marked as correct above, you should also check whether your CI variables in the gitlab settings are set as "protected". If so, you may not be able to use them in a branch that's not protected.
"You can protect a project, group or instance CI/CD variable so it is only passed to pipelines running on protected branches or protected tags." -> check it https://docs.gitlab.com/ee/ci/variables/index.html#protect-a-cicd-variable
Be aware that.. in certain cases SOME of the variables Gitlab CI CD offer are not always available..
In my case I wanted to use ${CI_COMMIT_BRANCH} but if you read the doc
https://docs.gitlab.com/ee/ci/variables/predefined_variables.html
The commit branch name. Available in branch pipelines, including pipelines for the default branch. Not available in merge request pipelines or tag pipelines.