I'm trying to set up MobSF SAST within Gitlab-ci and having a few issues.
I've followed the instructions within the Gitlab Docs and within the MobSF Gitlab repo
However, when I add:
To my .gitlab-ci.yml . I get a yml error stating that it could not get access
My .gitlab-ci.yml file looks like:
sast:
stage: Security
tags:
- docker
include:
- project: 'gitlab-org/security-products/analyzers/mobsf'
ref: master
file: '/template/mobsf.gitlab-ci.yml'
I have a docker image on my machine with gitlab-runners as an image. Does anyone have any thoughts about how to implement this so that i can run automated MobSF SAST on both Android and iOS?
So after working through this, It seems that you must have the following included in yoru gitlab-ci.yml file:
variables:
#required for Mobile SAST
SAST_EXPERIMENTAL_FEATURES: "true"
include:
- template: Security/SAST.gitlab-ci.yml
sast:
image: docker:19.03.8
stage: Security
variables:
SEARCH_MAX_DEPTH: 4
artifacts:
reports:
sast: gl-sast-report.json
tags:
- docker
Related
I'm trying to edit an ADO YAML file down to the bare minimum in order to isolate another issue.
When I run Validate, it comes back with the following error:
No repository found by name templates
Here's the general gist of my YAML:
#resources:
# repositories:
# - repository: templates
# type: git
# name: TemplateProject/TemplateRepo
name: $(VersionName)
trigger:
branches:
include:
- main
batch: true
paths:
exclude: $(ListOfExclusions)
stages:
- template: core/setVersion.yml#templates
- stage: Build
pool: linux
jobs:
- job: BuildDocker
displayName: Build and Push Docker Image
pool: linux
steps:
- task: Docker#2
displayName: Build and push an image to container registry
inputs:
command: buildAndPush
repository: $(RepoName)
dockerfile: $(Build.SourcesDirectory)/Dockerfile
containerRegistry: $(dockerRegistryServiceConnection)
tags: |
$(Tag)
What could be going wrong? The error message makes me think the YAML isn't clean.
It turns out I caused a simple typo when commenting out the resources section of the YAML. I had a template part of the stage that also needed to be commented out, and I neglected to do this.
Once I updated the code to read:
stages:
# - template: core/setVersion.yml#templates
- stage: Build
pool: linux
jobs:
- job: BuildDocker
# etc...
Now my YAML validates with OK.
I'm using self-hosted gitlab version 14.10 with self-registered gitlab runners using docker executors. In this image I'm using gitlab's secret detection job template and it generates artifact gl-secret-detection-report.json and it seems that it uploads it back to some kind of coordinator (I wonder what is this) but unfortunately I still can't see it in my Job artifacts section at this page.
I've checked this question that seemed to be similar, but its not, I've also checked GitLab's documentation and found nothing similar to my issue.
I even can't see Artifacts column in the Job's list at my Pipeline's page:
Okay, I've found the answer here:
To be able to browse the report output files, include the artifacts:paths keyword.
After I've overrided artifacts section for the job it worked fine. There's still no "Artifacts" column in the Job's row at the Pipeline's job list page.
My .gitlab-ci.yml:
variables:
DOCKER_HOST: tcp://docker:2375
DOCKER_DRIVER: overlay2
DOCKER_TLS_CERTDIR: ""
include:
- project: 'devops/gitlab-templates'
ref: master
file: '/lib/gitlab/ci/templates/Docker/Docker-Build-Risky.DO.yml'
- project: 'devops/gitlab-templates'
ref: master
file: '/lib/gitlab/ci/templates/Security/Trivy-Scan.yml'
- template: Security/Secret-Detection.gitlab-ci.yml
- project: 'devops/gitlab-templates'
ref: master
file: '/lib/gitlab/ci/templates/Security/Dive-Scan.yml'
- project: 'devops/gitlab-templates'
ref: master
file: '/lib/gitlab/ci/templates/Docker/Docker-Retag-n-Push.DO.yml'
- project: 'devops/gitlab-templates'
ref: master
file: '/lib/gitlab/ci/templates/Docker/Docker-Retag-n-Push.AWS.yml'
- project: 'devops/gitlab-templates'
ref: master
file: '/lib/gitlab/ci/templates/Docker/Docker-Retag-n-Push.GCP.yml'
- project: 'devops/gitlab-templates'
ref: master
file: '/lib/gitlab/ci/templates/AWS/Deploy.yml'
secret_detection:
variables:
SECRET_DETECTION_HISTORIC_SCAN: "true"
allow_failure: false
artifacts:
reports:
secret_detection: gl-secret-detection-report.json
# this is the way to make artifacts appear
paths:
- gl-secret-detection-report.json
expire_in: 1 day
integration-tests:
stage: test
needs:
- job: "docker-build"
artifacts: true
...
deploy-to-aws:
environment: production
variables:
...
I got a lot of different android flavors for one app to build, so i want to split up the building into different yml files. I currently have my base file .gitlab-ci.yml
image: alvrme/alpine-android:android-29-jdk11
variables:
GIT_SUBMODULE_STRATEGY: recursive
before_script:
- export GRADLE_USER_HOME=`pwd`/.gradle
- chmod +x ./gradlew
cache:
key: "$CI_COMMIT_REF_NAME"
paths:
- .gradle/
stages:
- test
- staging
- production
- firebaseUpload
- slack
include:
- local: '/.gitlab/bur.yml'
- local: '/.gitlab/vil.yml'
- local: '/.gitlab/kom.yml'
I am currently trying to build 3 different flavors. But i dont know why only the last included yml file gets executed. the first 2 are ignored.
/.gitlab/bur.yml
unitTests:
stage: test
script:
- ./gradlew testBurDevDebugUnitTest
/.gitlab/vil.yml
unitTests:
stage: test
script:
- ./gradlew testVilDevDebugUnitTest
/.gitlab/kom.yml
unitTests:
stage: test
script:
- ./gradlew testKomDevDebugUnitTest
What you observe looks like the expected behavior:
Your three files .gitlab/{bur,vil,kom}.yml contain the same job name unitTests.
So, each include overrides the specification of this job.
As a result, you only get 1 unitTests job in the end, with the specification from the last YAML file.
Thus, the simplest fix would be to change this job name, e.g.:
unitTests-kom:
stage: test
script:
- ./gradlew testKomDevDebugUnitTest
I’d like to use the artifacts created by the Security/SAST.gitlab-ci.yml template in my final pipeline stage (reporting).
How can I modify the Security/SAST.gitlab-ci.yml template to store the artifacts somewhere in my project dir? I tried to define the following for this template, but this is not working:
artifacts:
paths:
- binaries/
I’d be happy for every kind of support.
Thank you
Solution
Your parameters need to be updated. Since SAST.gitlab-ci.yml cannot be updated directly, you need to either override one of the blocks from your gitlab-ci.yml which includes the file, or define and include your custom SAST.gitlab-ci.yml. It seems like you can get away with simply overriding the sast block. Specifically, override the artifacts -> reports -> sast parameter.
Example
sast:
stage: test
artifacts:
reports:
sast: gl-sast-report.json
You also need to ensure the stages and build step is something resembling
stages:
- build
- test
include:
- template: Security/SAST.gitlab-ci.yml
build:
stage: build
script:
- ...
artifacts:
paths:
- binaries/
References
Gitlab SAST: https://docs.gitlab.com/ee/user/application_security/sast/
My CI has two main steps. Build and deploy. The result of build is that an artifact is uploaded to maven nexus. And currently manual deploy step just takes the latest artifact from nexus and deploys it.
stages:
- build
- deploy
full:
stage: build
image: ubuntu
script:
- // Build and upload to nexus here
deploy:
stage: deploy
script:
- // Take latest artifact from nexus and deploy
when: manual
But to me this doesn't seem to make that much sense to always deploy latest build from every pipeline. I think ideally deploy step of each pipeline should deploy the artifact that was build by the same pipelines build task. Otherwise deploy step of each pipeline will do exactly the same thing regardless when it is started.
So I have two questions.
1) How can I make my deploy step to deploy the version that was build by this run?
2) If I still want to keep the "deploy latest" functionality, then does gitlab support adding a task separate of each pipeline because as I explained this step doesn't make a lot of seance to be in pipeline? I imagine it being in a separate specific place.
Not too familiar with maven and nexus, but assuming you can name the artifact before you push it, you can add one of the built-in environment variables that dictates which pipeline it's from.
ie:
...
Build:
stage: build
script:
- ./buildAsNormal.sh > build$CI_PIPELINE_ID.extension
- ./pushAsNormal.sh
Deploy:
stage: deploy
script:
- ./deployAsNormal #(but specify the build$CI_PIPELINE_ID.extension file)
There are a lot of CI env variables you can use that are extremely useful. The full list of them is here. The difference with $CI_PIPELINE_ID and $CI_JOB_ID is that the pipeline id is constant for all jobs in the pipeline, no matter when they execute. That means the pipeline id will be the same even if you run a manual step a week after the automated steps. The job id is specific to each job.
Regarding your comment, the usage of artifacts: can solve your problem.
You can put the version number in a file and get the file in the next stage :
stages:
- build
- deploy
full:
stage: build
image: ubuntu
script:
- echo "1.0.0" > version
- // Build and upload to nexus here
artifacts:
paths:
- version
expire_in: 1 week
deploy:
stage: deploy
script:
- VERSION = $(cat version)
- // Take the artifact from nexus using VERSION variable and deploy
when: manual
An alternative is to build, push to nexus and use artifact: to pass the result of the build to the Deploy job :
stages:
- build
- deploy
full:
stage: build
image: ubuntu
script:
- // Build and put the result in out/ directory
- // Upload the result from out/ to nexus
artifacts:
paths:
- out/
expire_in: 1 week
deploy:
stage: deploy
script:
- // Take the artifact from in out/ directory and deploy it
when: manual