I am trying to download artifacts from a successful job.
The download for a regular job works but the download of of a job matrix is not working.
Any idea how I have to structure the URL for that.
lambda_build: [tools]
lambda_build: [helper]
lambda-master works with ..../-/jobs/artifacts/master/download?job=lambda-master
I don't find answer in the gitlab documentation.
Executing a matrix job generates a name unique to that pipeline from each variable used, so I think you can't just use the name you gave the job alone.
For example, when declaring this matrix job:
myJob:
script: echo $FOO
parallel:
matrix:
- FOO: [bar, baz]
Gitlab will generate two jobs:
myJob: [bar]
myJob: [baz]
So you might have to put "build: [$lambda_build]" or something into the query parameter.
Related
I struggle with setting up a condition statement to run a specific task only on main and when there is a CI build.
While trying without success, I came across this statement which is an AND of two exactly the same statements:
condition: and(contains(variables['build.sourceBranch'], 'refs/heads/main'), contains(variables['build.sourceBranch'], 'refs/heads/main'))
When running it on my branch, I see this result:
Evaluating: and(contains(variables['build.sourceBranch'], 'refs/heads/main'), contains(variables['build.sourceBranch'], 'refs/heads/main'))
Expanded: and(contains('refs/heads/task/mybranch', 'refs/heads/main'), contains(variables['build.sourceBranch'], 'refs/heads/main'))
Result: False
Why is the second variable not expanded?
To reach my aim of that condition, I'd like to run something like this, but here the expansion does not work as well:
and(endsWith(variables['build.sourceBranch'], 'refs/heads/main'), endsWith(variables['build.reason'], 'CI'))
What am I doing wrong?
Thanks in advance
I have a gitlab repo and I am trying to create some tags on top of required commits. But my CICD library (included) is having a job (job1) in a stage which runs on every tag/branch/commit. I want to exclude it to run on particular formats of tags. (eg. pre_final-2.3.1).
I tried giving the tag pattern in my .gitlab-ci.yaml in the except section of the job as below.
Eg:
job1:
except:
- ^pre_final-\d+\.\d+\.\d+$
It is still adding the job job1 to the pipeline for the CICD builds running on this tag. I believe this pattern is checking with branch name. But is there a way to mention the ref we provide in the except section is branch or tag?
Ref: https://docs.gitlab.com/ee/ci/yaml/#onlyrefs--exceptrefs
You should rules instead as only/except is not developed further. There you can check for the the name of your tag.
job1:
rules:
- if: '$CI_COMMIT_TAG =~ /^pre_final-\d+\.\d+\.\d+$/'
when: never
- when: always
There is no separate classification as branch or tag while using them in only/except.
I realized that I am giving a wrong regex pattern and thus it is not working for the tag pattern I am trying for
^pre_final-\d+\.\d+\.\d+$ --> /^pre_final-\d+\.\d+\.\d+$/
So my .gitlab-ci.yaml contains as below
job1:
except:
# matches with branch or tags names
- /^pre_final-\d+\.\d+\.\d+$/
Since above usage matches both branch and tag names, if we want to match only branch name or match only tag name, then we can use the below format
job1:
except:
variables:
# to match only tag names
- $CI_COMMIT_TAG == /^pre_final-\d+\.\d+\.\d+$/
# to match only branch names
- $CI_COMMIT_BRANCH = /^pre_final-\d+\.\d+\.\d+$/
We have a .gitlab-ci.yml file containing lines like
a_task:
only:
- /^production\/mybranch.*$/
which are clearly meant to match the target git ref.
But we also have:
another_task:
only:
- master
My question is: does this "master" match a part of the git ref as well (so that a tag my-master-123 would match, too) or is it a symbolic thing?
The reason why am asking is that there is also:
third_task:
only:
- tags
That would have to be symbolic, right?
Which would mean that the syntax does e.g. not support a branch named tags, right?
Update
Looks like there are special keywords, tags being one of them.
So indeed that would mean that refs with those special names (external, pipelines, tags, triggers, ...) would not be supported.
from the docs:
only and except are two keywords that set a job policy to limit when jobs are >created:
only defines the names of branches and tags the job runs for.
except defines the names of branches and tags the job does not run for.
Matching via regular expressions is supported, as in your first case, but not default. only: master tasks will run for all refs named master.
I am using the downloading latest artifact feature.
For me it is not clear, how the job name I need to pass is created: my job name contains e.g. spaces, equal signs and brackets:
build win: [USE_PYTHON=ON]
I know that spaces are replaced by +-signs but what about the others characters?
Changing the job name is not an option because I use the matrix-feature and it creates names like these.
Thanks a lot for your help!
Example ci yaml:
build win:
...
parallel:
matrix:
- USE_PYTHON: ["USE_PYTHON=ON", "USE_PYTHON=OFF"]
You can use ASCII encoding like for space %20.
Find them here
https://www.w3schools.com/tags/ref_urlencode.ASP
The consistency of Variable support & the syntax vary wildly in Azure DevOps YAML.
Case in point:
trigger:
- master
# Variable Group has $(testCategory1) with value
# 'TestCategory=bvttestonly | TestCategory=logintest'
variables:
- group: DYNAMIC_VG
jobs:
- job:
pool: 'MyPool' #Has about 10+ self hosted agents
strategy:
parallel: $[ variables['noOfVMsDynamic']]
variables:
indyx: '$(testCategories$(System.JobPositionInPhase))'
indyx2: $[ variables['indyx'] ]
testCategories: $[ variables[ 'indyx2' ] ]
steps:
- script: |
echo "indyx2 - $(indyx2)"
echo "testCategories $(testCategories)"
displayName: 'Display Test Categories'
The step prints:
"indyx2 - $(testCategories1)"
"testCategories $(testCategories1)"
I need to print the value of $(testCategories1) defined in the Variable Group:
'TestCategory=bvttestonly | TestCategory=logintest'
This may work to you:
variables
indyx: $[ variables[format('{0}{1}', 'testCategories', variables['System.JobPositionInPhase'])] ]
It worked for me, in a slightly different situation which also required some dynamic variable names.
Howto: Dynamically resolve a nested variable in Azure DevOps YAML
That because the value of nested variables (like $(testCategories$(System.JobPositionInPhase))) are not yet supported in the build pipelines at this moment.
That the reason why you always get the value $(testCategories1) rather than the real value of variable testCategories1.
I encountered this issue many times in my past posts and we do not have a perfect solution before Azure Devops supports this feature.
For the convenience of testing, I simplified your yaml like following:
jobs:
- job: ExecCRJob
timeoutInMinutes: 800
pool:
name: MyPrivateAgent
displayName: 'Execute CR'
variables:
testCategories1: 123456
testCategoriesSubscripted: $(testCategories$(System.JobPositionInPhase))
strategy:
parallel: $[variables['noOfVMs']]
steps:
- template: execute-cr.yml
parameters:
testCategories: $(testCategoriesSubscripted)
The execute-cr.yml:
steps:
- script: echo ${{ parameters.testCategories }}
We always get the $(testCategories1)NOT the value of it.
If I change the $(testCategories$(System.JobPositionInPhase)) to $(testCategories1), everything work fine.
Since nested variables are not yet supported, As workaround, we need to expand the nested variables for each value of testCategories, like:
- job: B
condition: and(succeeded(), eq(dependencies.A.outputs['printvar.skipsubsequent'], 'Value1'))
dependsOn: A
steps:
- script: echo hello from B
Check the Expressions Dependencies for some more details.
Hope this helps.
If I'm understanding your issue correctly, the problem is that the pipeline evaluates all variables at the runtime of the job. The solution in this scenario is to split your tasks into separate jobs with dependencies.
Have a look at my answer in this post and let me know if it's what you're after : YAML pipeline - Set variable and use in expression for template
I manage to get dynamic name resolution by using get-item to read the corresponding environment variable, allowing construction of the name of the variable and then getting the value.
In our case we save the name of an autogenerated branch into a variable group and each repository will have its own variable.
$branchVarName = "$(Build.Repository.Name).BranchName".replace(".","_")
$branchName = (get-item -Path Env:$branchVarName).value
write-host "/$(System.TeamProject)/_apis/build/builds?&repositoryId=$(Build.Repository.ID)&repositoryType=TFSGit&branchName=refs/heads/$branchName&api-version=6.0"
Notice in the second line that I reference the variable content using .value because get-item returns a name/value key pair.
This extraction has to be done in script but could be exposed as an output variable if needed in another context.