I want to run the gitlab-ci pages job only if the project is getting a new tag pushed. I already know that you can do this with:
only:
- tags
but the issue is that, if we ever push a tag for an older version (a backported bugfix or something) then this would overwrite the pages build. We tag using SEMVER if that helps
For example, what I'm trying to avoid is having a bug fix for an earlier version replacing the gitlab pages.
Let's say on the master branch we release version 1.5.0, this will build the pages for that version, and those will be the current documentation pages.
Now let's say we had to do a bugfix for version 1.3.0 to 1.3.1, if we make that bug fix and then push it, since pages builds on tags, it would build the docs for 1.3.1 and those would replace the docs for 1.5.0, which I want to avoid somehow.
GitLab CI allows to use regex patterns with only.
In your case, if you want your pipeline to be run only if a new tag is pushed, you should think of a naming convention for tags and find a regex that will only match these new tags. This should give you some ideas:
job:
only:
- /^(\d+\.)?(\d+\.)?(\*|\d+)$/
except:
- branches
- triggers
You need the except statement to specify that the job should only run when a tag is pushed. If you just added - tags to an only statement with a regex, it would run whenever a tag is pushed + when a branch or trigger matches the regex.
One possible solution would be to incorporate an beginning stage job that compares the latest tag and only sends success if the most recent pushed tag is the latest
stages:
- compare tags
- build
- test
- deploy
job:
stage: compare tags
only:
- <semver tag regex goes here>
except:
- branches
script: 'script-that-will-compare-all-semver-tags-to-CI_COMMIT_TAG-and-fail-if-not-latest'
job:
stage: build
...
job:
stage: test
...
...
the whole job should stop if the semver comparison fails
Related
I have a multi-module build with one "leading" module and one additional one. I have set things up so that the additional module is only built when either it or the build files have changed:
build:sbt:module-main:
extends: .build-sbt
build:sbt:module-a:
extends: .build-sbt
only:
changes:
- module-a/**/*
- project/**/*
- "*.yml"
- "*.sbt
The behaviour I observe is that in a pipeline resulting from pushing a new branch, both modules are always built, regardless of the actual changes. Then when new commits are pushed to the branch, the pipelines triggered off of those will behave according to my rule, i.e. module-a will only be built when there was a change that affects it.
I would expect the same behaviour from the start.
I assume that "change" means "what git thinks has changed between this branch and the branch it was based off of". Is that not what change means in this context?
While searching for another answer I discovered by accident that Gitlab is working as designed in this case.
I would like to achieve this Gitlab CI/CD behaviour:
Two jobs defined in the pipeline:
job_beta triggered when a tag is created with a semver number containing a "-rc*" suffix. For example: v1.0.0-rc1.
job_production triggered when a tag is created with a semver number, without any suffix. For example: v1.0.0.
Is there a way to parse the $CI_COMMIT_TAG variable in the .gitlab-ci.yml file of the project? Or any other ideas about how to achieve this?
To achieve this kind of behaviour, you can rely on the only keyword to control which job will be created. Simply specify a regular expression that will either match a semver with the rc suffix or one without any suffix.
You could do it the following way if your semantic versions are prefixed with v:
build-beta:
image: your-build-image:latest
stage: build
only:
- /^v[0-9]+\.[0-9]+\.[0-9]+-rc[0-9]+$/
script:
- ./build-beta.sh # Do something...
build-prod:
image: your-build-image:latest
stage: build
only:
- /^v[0-9]+\.[0-9]+\.[0-9]+$/
script:
- ./build-prod.sh # Do something...
You can also achieve something similar with the rules keyword by matching $CI_COMMIT_REF_NAME with the appropriate regex in an if condition.
I have a ConcourseCI pipeline that automatically increments my version number using this resource type: https://github.com/concourse/semver-resource
My resource declaration looks like this:
- name: version
type: semver
source:
driver: git
initial_version: 0.0.1
uri: {{version-repo-uri}}
branch: {{version-repo-branch}}
file: {{version-file}}
private_key: {{git-key}}
And my job looks like this:
- name: increment-version
plan:
- get: {{git-repo-name}}
trigger: true
- get: version
params: {bump: patch}
- put: version
params: {file: version/version}
So as you can see, right now I'm always bumping PATCH version. However, I'd like an easy and preferably automated way for the pipeline to increment MAJOR, MINOR, PATCH, or RC version depending on the circumstances.
Is there a git hook or something similar that will know when to bump what? Any implementation best practices when automating semver into a pipeline? Any other Concourse resource-type I should be using instead? Or is this something we should definitely leave for a human to decide? And in that case, how to easily integrate that "manual" step into a CD pipeline?
The closest thing I can think of is having a version file in my project (committed to github) with the intended version, which will set MAJOR, MINOR, PATCH numbers. The pipeline picks ups that file, uses it as a base somehow, and only increments RC numbers, but this feels incredibly error prone.
To be clear, I'm not asking how to version releases in theory, and the meanings of what major, minor, or patch are. I'm asking in practice how to implement those recommendations listed on https://semver.org/.
My solution to this problem is to tag the commits, and a script which reads the Git commit message and steps the version based on the content:
If a version tag exist on the current commit: always step PATCH.
Else:
List all commits until the previous tag.
If any message contains "step: major" then step MAJOR
Else if all messages contain "step: patch" or "step: micro" then step PATCH
Else step MINOR
This means that if the committer does not state anything in the message, then the change is assumed to be a functional change.
Tag and push the commit before the built artifact is distributed. Otherwise the next build could produce an already existing version.
Just modify the logic as needed for your use cases.
I want to trigger a pipeline everytime the current milestone branch changes it works fine with hardcoded milistone number
the problem is that we increase the milestone number, every 2 weeks
and gitlab runner doesn't parse .gitlab-ci.yml wildcards
so things like that not working
job:
only:
- milestone-*
i also tried regex as suggested BY Makoto Emura here at the comments
java:
only:
- /^mileston-.*$/
for now i use it this way and update my .gitlab-ci.yml after creating a new milestone
job:
only:
- milestone-10
I try to look for an environment variable for target branch but didn't find any
Does anyone know a solution?
I tested with this regex and it works :
only:
- /^milestone-.*$/
In your comment, you wrote /^mileston-.*$/ instead of /^milestone-.*$/ (the e is missing at the end of milestone)
In my pipeline, I'd like to have a job run only if the Merge Requests target branch is a certain branch, say master or release.
Is this possible?
I've read through https://docs.gitlab.com/ee/ci/variables/ and unless I missed something, I'm not seeing anything that can help.
Update: 2019-03-21
GitLab has variables for merge request info since version 11.6 (https://docs.gitlab.com/ce/ci/variables/ see the variables start with CI_MERGE_REQUEST_). But, these variables are only available in merge request pipelines.(https://docs.gitlab.com/ce/ci/merge_request_pipelines/index.html)
To configure a CI job for merge requests, we have to set:
only:
- merge_requests
And then we can use CI_MERGE_REQUEST_* variables in those jobs.
The biggest pitfall here is only: merge_request has complete different behavior from normal only/except parameters.
usual only/except parameters:
(https://docs.gitlab.com/ce/ci/yaml/README.html#onlyexcept-basic)
only defines the names of branches and tags for which the job will run.
except defines the names of branches and tags for which the job will not run.
only: merge_request: (https://docs.gitlab.com/ce/ci/merge_request_pipelines/index.html#excluding-certain-jobs)
The behavior of the only: merge_requests parameter is such that only jobs with that parameter are run in the context of a merge request; no other jobs will be run.
I felt hard to reorganize jobs to make them work like before with only: merge_request exists on any job. Thus I'm still using the one-liner in my original answer to get MR info in a CI job.
Original answer:
No.
But GitLab have a plan for this feature in 2019 Q2: https://gitlab.com/gitlab-org/gitlab-ce/issues/23902#final-assumptions
Currently, we can use a workaround to achieve this. The method is as Rekovni's answer described, and it actually works.
There's a simple one-liner, get the target branch of an MR from the current branch:
script: # in any script section of gitlab-ci.yml
- 'CI_TARGET_BRANCH_NAME=$(curl -LsS -H "PRIVATE-TOKEN: $AWESOME_GITLAB_API_TOKEN" "https://my.gitlab-instance.com/api/v4/projects/$CI_PROJECT_ID/merge_requests?source_branch=$CI_COMMIT_REF_NAME" | jq --raw-output ".[0].target_branch")'
Explanation:
CI_TARGET_BRANCH_NAME is a newly defined variable which stores resolved target branch name. Defining a variable is not necessary for various usage.
AWESOME_GITLAB_API_TOKEN is the variable configured in repository's CI/CD variable config. It is a GitLab personal access token(created in User Settings) with api scope.
About curl options: -L makes curl aware of HTTP redirections. -sS makes curl silent(-s) but show(-S) errors. -H specifies authority info accessing GitLab API.
The used API could be founded in https://docs.gitlab.com/ce/api/merge_requests.html#list-project-merge-requests. We use the source_branch attribute to figure out which MR current pipeline is running on. Thus, if a source branch has multiple MR to different target branch, you may want to change the part after | and do your own logic.
About jq(https://stedolan.github.io/jq/), it's a simple CLI util to deal with JSON stuff(what GitLab API returns). You could use node -p or any method you want.
Because of the new env variables in 11.6 $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME and $CI_MERGE_REQUEST_TARGET_BRANCH_NAME jobs can be included or excluded based on the source or target branch.
Using the only and except (complex) expressions, we can build a rule to filter merge requests. For a couple examples:
Merge request where the target branch is master:
only:
refs:
- merge_requests
variables:
- $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == "master"
Merge request except if the source branch is master or release:
only:
- merge_requests
except:
variables:
- $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME == "master"
- $CI_MERGE_REQUEST_SOURCE_BRANCH_NAME == "release"
If you want to use multiple refs (let's say merge_requests and tags) and multiple variables, the refs will be OR'd, the variables will be OR'd, and the result will be AND'd:
If any of the conditions in variables evaluates to truth when using only, a new job is going to be created. If any of the expressions evaluates to truth when except is being used, a job is not going to be created.
If you use multiple keys under only or except, they act as an AND. The logic is:
(any of refs) AND (any of variables) AND (any of changes) AND (if kubernetes is active)
Variable expressions are also quite primitive, only supporting equality and (basic) regex. Because the variables will be OR'd you cannot specify both a source and target branch as of gitlab 11.6, just one or the other.
As of GitLab 11.6, there is CI_MERGE_REQUEST_TARGET_BRANCH_NAME.
If this is what you're really after, there could be an extremely convoluted way (untested) you could achieve this using the merge request API and CI variables.
With a workflow / build step something like:
Create merge request from feature/test to master
Start a build
Using the API (in a script), grab all open merge requests from the current project using CI_PROJECT_ID variable, and filter by source_branch and target_branch.
If there is a merge request open with the source_branch and target_branch being feature/test and master respectively, continue with the build, otherwise just skip the rest of the build.
For using the API, I don't believe you can use the CI_JOB_TOKEN variable to authenticate, so you'll probably need to create your own personal access token and store it as a CI variable to use in the build job.
Hope this helps!
Another example, but using rules:
rules:
# pipeline should run on merge request to master branch
- if: $CI_PIPELINE_SOURCE == 'merge_request_event' && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'master'
when: always
# pipeline should run on merge request to release branch
- if: $CI_PIPELINE_SOURCE == 'merge_request_event' && $CI_MERGE_REQUEST_TARGET_BRANCH_NAME == 'release'
when: always
- when: never
Gitlab CI is agnostic of Merge Requests (for now). Since the pipeline runs on the origin branch you will not be able to retrieve the destination.