I'm using self-hosted gitlab version 14.10 with self-registered gitlab runners using docker executors. In this image I'm using gitlab's secret detection job template and it generates artifact gl-secret-detection-report.json and it seems that it uploads it back to some kind of coordinator (I wonder what is this) but unfortunately I still can't see it in my Job artifacts section at this page.
I've checked this question that seemed to be similar, but its not, I've also checked GitLab's documentation and found nothing similar to my issue.
I even can't see Artifacts column in the Job's list at my Pipeline's page:
Okay, I've found the answer here:
To be able to browse the report output files, include the artifacts:paths keyword.
After I've overrided artifacts section for the job it worked fine. There's still no "Artifacts" column in the Job's row at the Pipeline's job list page.
My .gitlab-ci.yml:
variables:
DOCKER_HOST: tcp://docker:2375
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: ""
include:
- project: 'devops/gitlab-templates'
ref: master
file: '/lib/gitlab/ci/templates/Docker/Docker-Build-Risky.DO.yml'
- project: 'devops/gitlab-templates'
ref: master
file: '/lib/gitlab/ci/templates/Security/Trivy-Scan.yml'
- template: Security/Secret-Detection.gitlab-ci.yml
- project: 'devops/gitlab-templates'
ref: master
file: '/lib/gitlab/ci/templates/Security/Dive-Scan.yml'
- project: 'devops/gitlab-templates'
ref: master
file: '/lib/gitlab/ci/templates/Docker/Docker-Retag-n-Push.DO.yml'
- project: 'devops/gitlab-templates'
ref: master
file: '/lib/gitlab/ci/templates/Docker/Docker-Retag-n-Push.AWS.yml'
- project: 'devops/gitlab-templates'
ref: master
file: '/lib/gitlab/ci/templates/Docker/Docker-Retag-n-Push.GCP.yml'
- project: 'devops/gitlab-templates'
ref: master
file: '/lib/gitlab/ci/templates/AWS/Deploy.yml'
secret_detection:
variables:
SECRET_DETECTION_HISTORIC_SCAN: "true"
allow_failure: false
artifacts:
reports:
secret_detection: gl-secret-detection-report.json
# this is the way to make artifacts appear
paths:
- gl-secret-detection-report.json
expire_in: 1 day
integration-tests:
stage: test
needs:
- job: "docker-build"
artifacts: true
...
deploy-to-aws:
environment: production
variables:
...
Related
I'm trying to edit an ADO YAML file down to the bare minimum in order to isolate another issue.
When I run Validate, it comes back with the following error:
No repository found by name templates
Here's the general gist of my YAML:
#resources:
# repositories:
# - repository: templates
# type: git
# name: TemplateProject/TemplateRepo
name: $(VersionName)
trigger:
branches:
include:
- main
batch: true
paths:
exclude: $(ListOfExclusions)
stages:
- template: core/setVersion.yml#templates
- stage: Build
pool: linux
jobs:
- job: BuildDocker
displayName: Build and Push Docker Image
pool: linux
steps:
- task: Docker#2
displayName: Build and push an image to container registry
inputs:
command: buildAndPush
repository: $(RepoName)
dockerfile: $(Build.SourcesDirectory)/Dockerfile
containerRegistry: $(dockerRegistryServiceConnection)
tags: |
$(Tag)
What could be going wrong? The error message makes me think the YAML isn't clean.
It turns out I caused a simple typo when commenting out the resources section of the YAML. I had a template part of the stage that also needed to be commented out, and I neglected to do this.
Once I updated the code to read:
stages:
# - template: core/setVersion.yml#templates
- stage: Build
pool: linux
jobs:
- job: BuildDocker
# etc...
Now my YAML validates with OK.
Is there any way we can pass job id of parent pipeline to child pipeline as variable
package:
stage: package
script:
- zip -r ./service.zip
deploy:
stage: deploy
variables:
trigger:
include:
- project: '<namespace>/<project>'
ref: '<branch>'
file: '<path to yml file>'
strategy: depend
Yes, it's supported by Gitlab. You need to pass the CI_PIPELINE_ID built-in variable.
.trigger_deploy:
stage: deploy
strategy: depend
trigger:
include:
- project: '<namespace>/<project>'
ref: '<branch>'
file: '<path to yml file>'
variables:
PARENT_PIPELINE_ID: $CI_PIPELINE_ID
PARENT_JOB_ID: $CI_JOB_ID
You can file these and more variables in Gitlab predefined variables documentation
I'm trying to set up MobSF SAST within Gitlab-ci and having a few issues.
I've followed the instructions within the Gitlab Docs and within the MobSF Gitlab repo
However, when I add:
To my .gitlab-ci.yml . I get a yml error stating that it could not get access
My .gitlab-ci.yml file looks like:
sast:
stage: Security
tags:
- docker
include:
- project: 'gitlab-org/security-products/analyzers/mobsf'
ref: master
file: '/template/mobsf.gitlab-ci.yml'
I have a docker image on my machine with gitlab-runners as an image. Does anyone have any thoughts about how to implement this so that i can run automated MobSF SAST on both Android and iOS?
So after working through this, It seems that you must have the following included in yoru gitlab-ci.yml file:
variables:
#required for Mobile SAST
SAST_EXPERIMENTAL_FEATURES: "true"
include:
- template: Security/SAST.gitlab-ci.yml
sast:
image: docker:19.03.8
stage: Security
variables:
SEARCH_MAX_DEPTH: 4
artifacts:
reports:
sast: gl-sast-report.json
tags:
- docker
I have two .yml files in my repo. One for build, one for deployment. The main reason why I would like to keep build separate from the deployment is that I also would like to store variables for environments in my repo, e.i. in variables-dev.yml and variables-prod.yml files. So there is no need to create a new build every time (which includes running tests, docker image build etc.).
The file build.yml:
trigger:
paths:
exclude:
- build.yml
- deploy.yml
stages:
- stage: build
jobs:
...
And the deploy.yml, which I want to be triggered only on the completion of the build pipeline. That's why I add the first exclusion of all paths, but add one on pipeline resource.
trigger:
paths:
exclude:
- '*'
resources:
pipelines:
- pipeline: build
source: build
trigger:
branches:
include:
- '*'
stages:
- stage: dev
variables:
- template: variables-dev.yml
jobs:
- deployment: deploy_dev
environment: 'dev'
strategy:
runOnce:
deploy:
steps:
...
- stage: prod
dependsOn: dev
variables:
- template: variables-prod.yml
jobs:
- deployment: deploy_prod
environment: 'prod'
strategy:
runOnce:
deploy:
steps:
...
Unfortunately it does not seem to work. The top trigger blocks lower trigger. And if I remove the top trigger than the deploy pipeline is triggered at the same time with the build one.
you have to start your deploy.yml with trigger: none
trigger: none
resources:
pipelines:
- pipeline: ci-pipeline
source: my-build-pipeline
trigger:
enabled: true
branches:
include:
- master
Set your triggers for the second yml to none, then add this setting in the "Triggers" section of the UI. It will stage your builds as you describe
When I take a mergeRequest in GitLab, there is a compile error with description:
CI build failed for 4a0b9b43 , view Details：.gitlab-ci.yml not
found in this commit.
But I have sync with the destination branch and compile succeed on native, how can I fix this issue?
You must place the .gitlab-ci.yml file to the root directory of a project first. If you want to cross-reference other CI configuration locally or from somewhere (GitLab instance required to install on remote locations), you must use the include tag
How-to
Please check your GL instance version first before you are using include. Compare your instance version with the minimum version required from the documentation. If you need to upgrade, back up everything and run an update. It should take hours before you can go back into business.
To cross-reference CI config files locally, use the include:local. Make sure they are on the same branch. If that is on other branch, use ref.
include:
- local: '/templates/.gitlab-ci-template.yml'
In case you want to cross-reference CI config files somewhere within the instance, use
include:
- project: 'my-group/my-project'
file: '/templates/.gitlab-ci-template.yml'
# You can also specify ref, with the default being the head of the object.
- project: 'my-group/my-project'
ref: master # Git branch
file: '/templates/.gitlab-ci-template.yml'
- project: 'my-group/my-project'
ref: v1.0.0 # Git tag
file: '/templates/.gitlab-ci-template.yml'
- project: 'my-group/my-project'
ref: 787123b47f14b552955ca2786bc9542ae66fee5b # Git SHA
file: '/templates/.gitlab-ci-template.yml'
If you prefer to use CI config templates shipped with the instance, use include:template. Check the GitLab's template collection for more details.
include:
- template: Auto-DevOps.gitlab-ci.yml
If deployed outside the instance borders, like GitLab.com, use include:remote.
include:
- remote: 'https://gitlab.com/awesome-project/raw/master/.gitlab-ci-template.yml'
Examples
The first example was derived from GitLab's CI config file for GitLab EE (gitlab-org/gitlab).
image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.6.3-golang-1.12-git-2.24-lfs-2.9-chrome-73.0-node-12.x-yarn-1.16-postgresql-9.6-graphicsmagick-1.3.33"
stages:
- sync
- prepare
- quick-test
- test
- post-test
- review-prepare
- review
- qa
- post-qa
- notification
- pages
variables:
RAILS_ENV: "test"
NODE_ENV: "test"
SIMPLECOV: "true"
GIT_DEPTH: "20"
GIT_SUBMODULE_STRATEGY: "none"
GET_SOURCES_ATTEMPTS: "3"
KNAPSACK_RSPEC_SUITE_REPORT_PATH: knapsack/report-master.json
FLAKY_RSPEC_SUITE_REPORT_PATH: rspec_flaky/report-suite.json
BUILD_ASSETS_IMAGE: "false"
ES_JAVA_OPTS: "-Xms256m -Xmx256m"
ELASTIC_URL: "http://elastic:changeme#elasticsearch:9200"
after_script:
- date
include:
- local: .gitlab/ci/cache-repo.gitlab-ci.yml
- local: .gitlab/ci/cng.gitlab-ci.yml
- local: .gitlab/ci/docs.gitlab-ci.yml
- local: .gitlab/ci/frontend.gitlab-ci.yml
- local: .gitlab/ci/global.gitlab-ci.yml
- local: .gitlab/ci/memory.gitlab-ci.yml
- local: .gitlab/ci/notifications.gitlab-ci.yml
- local: .gitlab/ci/pages.gitlab-ci.yml
- local: .gitlab/ci/qa.gitlab-ci.yml
- local: .gitlab/ci/reports.gitlab-ci.yml
- local: .gitlab/ci/rails.gitlab-ci.yml
- local: .gitlab/ci/review.gitlab-ci.yml
- local: .gitlab/ci/setup.gitlab-ci.yml
- local: .gitlab/ci/dev-fixtures.gitlab-ci.yml
- local: .gitlab/ci/test-metadata.gitlab-ci.yml
- local: .gitlab/ci/yaml.gitlab-ci.yml
- local: .gitlab/ci/releases.gitlab-ci.yml
You can see more examples on the .gitlab-ci.yml reference in the GitLab documentation.