In my .gitlab-ci.yml file I try and set a variable containing a timestamp in the before_script section.
I would then like to expand that variable and append it to the archive I'm creating for my build.
The file goes roughly like this:
#.gitlab-ci.yml
image: node:14.4.0-buster
before_script:
- export DATETIME=$(date "+%Y%m%d%H%M%S")
stages:
#- test # not relevant for this question
- build
- deploy
build:
stage: build
script:
- npm install
- npm run build
- ls -la build
- tar cvfJ build_${DATETIME}.tar.xz build/
- sha1sum build_${DATETIME}.tar.xz
artifacts:
paths:
- build_${DATETIME}.tar.xz
deploy:
image: node:14.4.0-buster
stage: deploy
script:
- sha1sum build_${DATETIME}.tar.xz
- tar xvfJ build_${DATETIME}.tar.xz
# do the actual deploy
only:
- master
The deploy stage fails at sha1sum. The output is:
$ sha1sum build_${DATETIME}.tar.xz
sha1sum: build_20200702165854.tar.xz: No such file or directory
This shows that the expansion is done correctly, yet something is wrong.
What am I missing?
The before_script is run at the start of each job, and so the export DATETIME=$(date "+%Y%m%d%H%M%S") would be different for both stages.
It would probably be better to use ${CI_COMMIT_SHORT_SHA} instead for example.
Related
I have created Docs folder and it should contain report files.
I'm trying to move content from temporary created Allure folder to Docs folder and then copy everything from Docs to public folder to get access to Pages on which that Allure report will be located. I'm doing that process insted of simple copying files from allure folder to public folder to get history about previous runs. Maybe there is some better way to do it ? I'd like to store old reports for some X time (for example for 2 days, to be able to see on ALlure what was wrong is there are some problems) and then delete old ones, not deleting latest which have not reached "deleting point". So, here is my yml file:
stages:
- testing
- deploy
docker_job:
stage: testing
tags:
- docker
image: atools/chrome-headless:java11-node14-latest
before_script:
- npm ci
- npx playwright install
- npm install allure-commandline --save-dev
script: #||true
- npx playwright test
after_script:
- npx allure generate allure-results
rules:
- when: always
allow_failure: true
artifacts:
when: always
paths:
- ./allure-report
expire_in: 1 day
pages:
stage: deploy
script:
- mkdir public
- mv ./allure-report/* Docs
- cp -R ./Docs/* public
artifacts:
paths:
- public
rules:
- when: always
Everything is going good but it doesn't work - mv ./allure-report/* Docs
- cp -R ./Docs/* public are doing nothing or I just can't see any effect. Help me please to correctly solve that problem.
Maybe there is obvious holes in logic, idk, have tried a lot of variants but they all don't work.
Can it be done by my way at all?
okay, so I have done it with "logging in" with my git.conf email/name, then using auth token I did a push of these artifacts to Docs folder, looks like:
stages:
- testing
- deploy
docker_job:
stage: testing
tags:
- docker
image: atools/chrome-headless:java11-node14-latest
before_script:
- npm ci
- npx playwright install
- npm install allure-commandline --save-dev
script: #||true
- npx playwright test
after_script:
- npx allure generate allure-results
rules:
- when: always
allow_failure: true
artifacts:
when: always
paths:
- ./allure-report
expire_in: 15 mins
pages:
stage: deploy
script:
- cp -r -u ./allure-report/* Docs
- cp -R ./Docs/* public
- git config --global user.email "mail"
- git config --global user.name "name"
- git remote set-url origin https://gitlab-ci-token:${token}#gitlab.com/proj_link
- git checkout main
- git add Docs
- git commit -m "assets"
- git push
artifacts:
paths:
- public
rules:
- when: always
I got a lot of different android flavors for one app to build, so i want to split up the building into different yml files. I currently have my base file .gitlab-ci.yml
image: alvrme/alpine-android:android-29-jdk11
variables:
GIT_SUBMODULE_STRATEGY: recursive
before_script:
- export GRADLE_USER_HOME=`pwd`/.gradle
- chmod +x ./gradlew
cache:
key: "$CI_COMMIT_REF_NAME"
paths:
- .gradle/
stages:
- test
- staging
- production
- firebaseUpload
- slack
include:
- local: '/.gitlab/bur.yml'
- local: '/.gitlab/vil.yml'
- local: '/.gitlab/kom.yml'
I am currently trying to build 3 different flavors. But i dont know why only the last included yml file gets executed. the first 2 are ignored.
/.gitlab/bur.yml
unitTests:
stage: test
script:
- ./gradlew testBurDevDebugUnitTest
/.gitlab/vil.yml
unitTests:
stage: test
script:
- ./gradlew testVilDevDebugUnitTest
/.gitlab/kom.yml
unitTests:
stage: test
script:
- ./gradlew testKomDevDebugUnitTest
What you observe looks like the expected behavior:
Your three files .gitlab/{bur,vil,kom}.yml contain the same job name unitTests.
So, each include overrides the specification of this job.
As a result, you only get 1 unitTests job in the end, with the specification from the last YAML file.
Thus, the simplest fix would be to change this job name, e.g.:
unitTests-kom:
stage: test
script:
- ./gradlew testKomDevDebugUnitTest
I'm trying to create a gitlab job that shows a metric for test code coverage. To do that, I'm creating a .coverage file and placing it in a directory that uploads artifacts. In a subsequent stage the artifacts are downloaded and consumed by a coverage tool to produce a coverage report. I noticed that the artifacts are not deleted when the gitlab runner finishes the job and are bloating my filesystem. How can I remove the artifacts directory after the artifacts are uploaded?
Here's what we currently have
stages:
- test
- build
before_script:
- export GITLAB_ARTIFACT_DIR="$(pwd)"/artifacts
[...]
some-test:
stage: test
script:
- [some script that puts something in ${GITLAB_ARTIFACTS_DIR}
artifacts:
expire_in: 4 days
paths:
- artifacts/
some-other-test:
stage: test
script:
- [some script that puts something in ${GITLAB_ARTIFACTS_DIR}
artifacts:
expire_in: 4 days
paths:
- artifacts/
[...]
coverage:
stage: build
before_script:
script:
- [our coverage script]
coverage: '/TOTAL.*\s+(\d+%)$/'
artifacts:
expire_in: 4 days
paths:
- artifacts/
when: always
[...]
after_script:
- sudo rm -rf "${GITLAB_ARTIFACT_DIR}"
According to https://gitlab.com/gitlab-org/gitlab-runner/issues/4146 after_script does not have access to before_script or scripts environment variables.
A solution could be to use cache and artifact simultaneously.
This config will create a new directory depending of the job id ($CI_JOB_ID) for each job execution :
stages:
- test
remote:
stage: test
script :
- mkdir cache-$CI_JOB_ID
- echo hello> cache-$CI_JOB_ID/foo.txt
cache:
key: build-cache
paths:
- cache-$CI_JOB_ID/
artifacts:
paths:
- cache-$CI_JOB_ID/foo.txt
expire_in: 1 week
At the next run, the previous cache-$CI_JOB_ID will be removed and replace by a new directory (as the $CI_JOB_ID will be different). This will keep only one instance of your cached file until the next job execution.
Note : you need to prefix the directory name with cache- otherwise the .gitlab-ci.yml is invalid.
I'd like to add a build signature by the end of a file name in artifacts. I could be job id or a combination of job id and commit reference.
At the moment I get image.slp but I prefer to get something like image.1.slp or image.1.e8f8c4ed.slp . Here is my gitlab-ci.yml:
build-runner:
stage: build
script:
- ./build.sh
- cp ../output/image.slp .
artifacts:
paths:
- image.slp
You should be be able to accomplish that by means of CI_JOB_ID Environment variable.
Refer to docs for a comprehensive list of available variables that you can use.
Probably something like this could solve your problem:
build-runner:
stage: build
script:
- ./build.sh
- cp ../output/image.slp image.$CI_JOB_ID.slp
artifacts:
paths:
- image.$CI_JOB_ID.slp
I use gitlab-ci to test, compile and deploy a small golang application but the problem is that the stages take longer than necessary because they have to fetch all of the dependencies every time.
How can I keep the golang dependencies between two stages (test and build)?
This is part of my current gitlab-ci config:
test:
stage: test
script:
# get dependencies
- go get github.com/foobar/...
- go get github.com/foobar2/...
# ...
- go tool vet -composites=false -shadow=true *.go
- go test -race $(go list ./... | grep -v /vendor/)
compile:
stage: build
script:
# getting the same dependencies again
- go get github.com/foobar/...
- go get github.com/foobar2/...
# ...
- go build -race -ldflags "-extldflags '-static'" -o foobar
artifacts:
paths:
- foobar
As mentioned by Yan Foto, you can only use paths that are within the project workspace. But you can move the $GOPATH to be inside your project, as suggested by extrawurst blog.
test:
image: golang:1.11
cache:
paths:
- .cache
script:
- mkdir -p .cache
- export GOPATH="$CI_PROJECT_DIR/.cache"
- make test
This is a pretty tricky task, as GitLab does not allow caching outside the project directory. A quick and dirty task would be to copy the contents of $GOPATH under some directory inside the project (say _GO), cache it and copy it upon each stage start back to $GOPATH:
after_script:
- cp -R $GOPATH ./_GO || :
before_script:
- cp -R _GO $GOPATH
cache:
untracked: true
key: "$CI_BUILD_REF_NAME"
paths:
- _GO/
WARNING: This is just a (rather ugly) workaround and I haven't tested it myself. It should only exhibit a possible solution.