I am trying to use Infrastructure as Code (IaC) Scanning scans my IaC configuration files for known vulnerabilities. Reference: https://docs.gitlab.com/ee/user/application_security/iac_scanning/. However, I encountered the system problem in the pipeline.
[DEBU] [kics] [2022-11-11T03:55:29Z] [/go/pkg/mod/gitlab.com/gitlab-org/security-products/analyzers/command#v1.9.2/run.go:256] ▶ SAST_EXCLUDED_PATHS=spec,test,tests,tmp39
[INFO] [kics] [2022-11-11T03:55:29Z] [/go/pkg/mod/gitlab.com/gitlab-org/security-products/analyzers/command#v1.9.2/run.go:131] ▶ Detecting project
[INFO] [kics] [2022-11-11T03:55:29Z] [/go/pkg/mod/gitlab.com/gitlab-org/security-products/analyzers/command#v1.9.2/run.go:153] ▶ Analyzer will attempt to analyze all projects in the repository41
[INFO] [kics] [2022-11-11T03:55:29Z] [/go/pkg/mod/gitlab.com/gitlab-org/security-products/analyzers/command#v1.9.2/run.go:165] ▶ Running analyzer42
[DEBU] [kics] [2022-11-11T03:55:29Z] [/go/src/app/analyze.go:48] ▶ custom rulesets not enabled
[INFO] [kics] [2022-11-11T03:55:29Z] [/go/src/app/analyze.go:67] ▶ path /builds/leo/tcf
[ERRO] [kics] [2022-11-11T03:55:38Z] [/go/src/app/analyze.go:87] ▶ Encountered a system problem; status code: 2, error: exit status 2, detail:
[FATA] [kics] [2022-11-11T03:55:38Z] [/go/src/app/main.go:30] ▶ kics scanner failure: exit status 2
Uploading artifacts for failed job00:0048Uploading artifacts...
WARNING: gl-sast-report.json: no matching files. Ensure that the artifact path is relative to the working directory
ERROR: No files to upload
Cleaning up project directory and file based variables00:0154ERROR: Job failed: exit code 1
I can't find relevant information about the errors. How can I solve the errors?
You can use your own config, independent of the GitLab template.
gitlab-ci.yml:
image:
name: checkmarx/kics:latest
entrypoint: [""]
script:
- kics scan -q /app/bin/assets/queries -p ${PWD} --ignore-on-exit all --report-formats glsast -o ${PWD} --output-name kics-results
artifacts:
reports:
sast: gl-sast-kics-results.json
when: always
Related
I'm attempting to setup GitLab dependency scanning for a repository in my self-hosted GitLab server. I have included the job template and the test stage since I have overridden the stage clause. The job starts, but it fails soon after. When I set the variable SECURE_LOG_LEVEL to debug I see the following output.
$ /analyzer run
Using java version 'adoptopenjdk-17.0.2+8'
[INFO] [gemnasium-maven] [2023-01-28T15:21:00Z] [/go/src/app/cmd/gemnasium-maven/main.go:55] ▶ GitLab gemnasium-maven analyzer v3.11.1
[DEBU] [gemnasium-maven] [2023-01-28T15:21:00Z] [/go/src/app/finder/finder.go:64] ▶ inspect directory: .
[DEBU] [gemnasium-maven] [2023-01-28T15:21:00Z] [/go/src/app/finder/finder.go:96] ▶ skip ignored directory: .git
[DEBU] [gemnasium-maven] [2023-01-28T15:21:00Z] [/go/src/app/finder/detect.go:84] ▶ Selecting gradle for maven because this is the first match
[INFO] [gemnasium-maven] [2023-01-28T15:21:00Z] [/go/src/app/finder/finder.go:116] ▶ Detected supported dependency files in '.'. Dependency files detected in this directory will be processed. Dependency files in other directories will be skipped.
[DEBU] [gemnasium-maven] [2023-01-28T15:21:00Z] [/go/src/app/cmd/gemnasium-maven/main.go:234] ▶ Exporting dependencies for /path/to/my/app/build.gradle
[DEBU] [gemnasium-maven] [2023-01-28T15:21:05Z] [/go/src/app/builder/gradle/gradle.go:85] ▶ /path/to/my/app/gradlew --init-script /gemnasium-gradle-plugin-init.gradle gemnasiumDumpDependencies
Downloading https://services.gradle.org/distributions/gradle-7.1.1-bin.zip
..........10%...........20%...........30%..........40%...........50%...........60%..........70%...........80%...........90%...........100%
Welcome to Gradle 7.1.1!
Here are the highlights of this release:
- Faster incremental Java compilation
- Easier source set configuration in the Kotlin DSL
For more details see https://docs.gradle.org/7.1.1/release-notes.html
Starting a Gradle Daemon (subsequent builds will be faster)
FAILURE: Build failed with an exception.
* Where:
Initialization script '/gemnasium-gradle-plugin-init.gradle'
* What went wrong:
Could not compile initialization script '/gemnasium-gradle-plugin-init.gradle'.
> startup failed:
> General error during conversion: Unsupported class file major version 61
java.lang.IllegalArgumentException: Unsupported class file major version 61
at groovyjarjarasm.asm.ClassReader.<init>(ClassReader.java:189)
at groovyjarjarasm.asm.ClassReader.<init>(ClassReader.java:170)
at groovyjarjarasm.asm.ClassReader.<init>(ClassReader.java:156)
at groovyjarjarasm.asm.ClassReader.<init>(ClassReader.java:277)
...
How can I resolve this issue? I am using GitLab server v15.7.5.
I found a StackOverflow thread that is very similar to the issue I'm having.
It appears the version of ASM bundled with GitLab gemnasium-maven analyzer v3.11 does not support class files version 61 in Java v17.0.2+8.
I was able to get this working by downgrading GitLab gemnasium-maven analyzer to v2.31.0 and have filed a support request with GitLab to notify them of the issue.
To downgrade the dependency scanner, add the following block of code to your CICD template
.ds-analyzer:
variables:
DS_MAJOR_VERSION: 2
I have an error when using the template for IaC SAST.
Gitlab is 14.10, installed in my cloud, and accessible from the internet (I changed my real GitLab ULR to owngitlab.com)
The GitLab runner is installed in k8s.
Try to use SAST in my projects, but I get errors. Could somebody please help with this?
Here is part of my .gitlab-ci.yml
include:
- template: Security/SAST-IaC.latest.gitlab-ci.yml
stages:
- test
Pipeline log:
$ /analyzer run
[INFO] [kics] [2022-05-20T12:41:44Z] ▶ GitLab kics analyzer v1.5.2
[INFO] [kics] [2022-05-20T12:41:44Z] ▶ Detecting project
[INFO] [kics] [2022-05-20T12:41:44Z] ▶ Found relevant files in project, analyzing entire repository
[INFO] [kics] [2022-05-20T12:41:44Z] ▶ Running analyzer
[INFO] [kics] [2022-05-20T12:41:44Z] ▶ path /builds/tf/modules
[INFO] [kics] [2022-05-20T12:42:43Z] ▶ Creating report
[WARN] [kics] [2022-05-20T12:42:43Z] ▶ Could not detect git executable
Uploading artifacts for successful job
Uploading artifacts...
gl-sast-report.json: found 1 matching files and directories
ERROR: Uploading artifacts as "sast" to coordinator... error error=couldn't execute POST against https://owngitlab.com/api/v4/jobs/139/artifacts?artifact_format=raw&artifact_type=sast: Post "https://owngitlab.com/api/v4/jobs/139/artifacts?artifact_format=raw&artifact_type=sast": open gl-sast-report.json: permission denied id=139 token=hZ5dXhA7
WARNING: Retrying... context=artifacts-uploader error=invalid argument
ERROR: Uploading artifacts as "sast" to coordinator... error error=couldn't execute POST against https://owngitlab.com/api/v4/jobs/139/artifacts?artifact_format=raw&artifact_type=sast: Post "https://owngitlab.com/api/v4/jobs/139/artifacts?artifact_format=raw&artifact_type=sast": open gl-sast-report.json: permission denied id=139 token=hZ5dXhA7
WARNING: Retrying... context=artifacts-uploader error=invalid argument
ERROR: Uploading artifacts as "sast" to coordinator... error error=couldn't execute POST against https://owngitlab.com/api/v4/jobs/139/artifacts?artifact_format=raw&artifact_type=sast: Post "https://owngitlab.com/api/v4/jobs/139/artifacts?artifact_format=raw&artifact_type=sast": open gl-sast-report.json: permission denied id=139 token=hZ5dXhA7
FATAL: invalid argument
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: command terminated with exit code 1
I'm testing the SAST functionnality provided automatically by Gitlab.
My project (java, spring boot, maven) compiles well locally with mvn clean package.
I'm working with a hosted Gitlab and shared runners.
CI wasn't implemented yet.
I chose the generated .gitlab-ci.yml file
stages:
- test
sast:
stage: test
include:
- template: Security/SAST.gitlab-ci.yml
And then, the job fails with a message like :
$ /analyzer run
[INFO] [Find Security Bugs] [2021-11-01T16:27:54Z] ▶ GitLab Find Security Bugs analyzer v2.28.7
[INFO] [Find Security Bugs] [2021-11-01T16:27:54Z] ▶ Detecting project
[INFO] [Find Security Bugs] [2021-11-01T16:27:54Z] ▶ Found project in /builds/myrepo/myproject
[INFO] [Find Security Bugs] [2021-11-01T16:27:54Z] ▶ Running analyzer
[INFO] [Find Security Bugs] [2021-11-01T16:27:54Z] ▶ Found Mvnw project in /builds/myrepo/myproject directory
[INFO] [Find Security Bugs] [2021-11-01T16:27:54Z] ▶ Found 1 analyzable projects.
[INFO] [Find Security Bugs] [2021-11-01T16:27:54Z] ▶ Building Mvnw project at /builds/myrepo/myproject.
[ERRO] [Find Security Bugs] [2021-11-01T16:29:08Z] ▶ Project couldn't be built: exit status 1
[FATA] [Find Security Bugs] [2021-11-01T16:29:08Z] ▶ exit status 1
How can I know what went wrong ?
The error is in the second to last line of the container:
[ERRO] [Find Security Bugs] [2021-11-01T16:29:08Z] ▶ Project couldn't be built: exit status 1
[FATA] [Find Security Bugs] [2021-11-01T16:29:08Z] ▶ exit status 1
Assuming that your maven job is building fine locally, and it's building fine in a different maven container, I'd suggest that you configure the logging on the SAST job using the logging levels defined in the documentation, and gather some additional information: https://docs.gitlab.com/ee/user/application_security/sast/#logging-level
I am trying to follow the steps in https://spring.io/guides/gs/spring-boot-for-azure/. Under Config and deploy the app to Azure, when I run the "mvn com.microsoft.azure:azure-webapp-maven-plugin:1.8.0:config" command, the execution starts but doesn't complete. It stops where it is supposed to take user inputs but doesn't accept any values. Please find the execution logs below -
PS C:\Users\rthan\OneDrive\Documents\code\rest-service> mvn com.microsoft.azure:azure-webapp-maven-plugin:1.8.0:config
[INFO] Scanning for projects...
[INFO]
[INFO] -------------------< com.rainbowhomes:rest-service >--------------------
[INFO] Building rest-service 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- azure-webapp-maven-plugin:1.8.0:config (default-cli) # rest-service ---
[WARNING] The plugin may not work if you change the os of an existing webapp.
Define value for OS(Default: Linux):
1. linux [*]
2. windows
3. docker
Not sure what caused the issue, but if you want to deploy a Spring Boot app to Azure, you could refer to this doc, I successfully deployed the application via this doc last time, see this post.
I have selenium testing Maven project and i create a job on jenkins to run the POM.xml file by integrating it with GitHub. So when i click on "Build Now" to run the job in Jenkins its directly get the entire code from git and works as expected with SUCCESS result on windows machine. But when i create the same job in Jenkins on Linux machine it gives me errors. Need help on this
Started by user Admin fmcdjen
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] node
Running on Jenkins in /var/jenkins_home/workspace/SeleniumTest
[Pipeline] {
[Pipeline] timeout
Timeout set to expire in 20 min
[Pipeline] {
[Pipeline] stage
[Pipeline] { (Checkout SCM)
[Pipeline] echo
Checking out git repo
[Pipeline] checkout
using credential gitHubID
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url git#github.ford.com:dealer-platform/QA-selenium-automation.git # timeout=10
Fetching upstream changes from git#github.ford.com:dealer-platform/QA-selenium-automation.git
> git --version # timeout=10
using GIT_SSH to set credentials
> git fetch --tags --progress git#github.ford.com:dealer-platform/QA-selenium-automation.git +refs/heads/*:refs/remotes/origin/*
> git rev-parse origin/master^{commit} # timeout=10
Checking out Revision 25693e55034bef998abb437292745132f8b9c1c4 (origin/master)
> git config core.sparsecheckout # timeout=10
> git checkout -f 25693e55034bef998abb437292745132f8b9c1c4
Commit message: "comitted"
> git rev-list --no-walk 25693e55034bef998abb437292745132f8b9c1c4 # timeout=10
[Pipeline] }
[Pipeline] // stage
[Pipeline] stage
[Pipeline] { (Maven Build & Unit Testing)
[Pipeline] echo
Maven build and test
[Pipeline] withMaven
[withMaven] Options: []
[withMaven] Available options:
[withMaven] using JDK installation provided by the build agent
[withMaven] using Maven installation 'Maven_3.6.0'
[Pipeline] {
[Pipeline] sh
+ mvn test -DSuiteXmlFile=testng.xml
----- withMaven Wrapper script -----
Picked up JAVA_TOOL_OPTIONS: -Dmaven.ext.class.path="/var/jenkins_home/workspace/SeleniumTest#tmp/withMavena8cb286b/pipeline-maven-spy.jar" -Dorg.jenkinsci.plugins.pipeline.maven.reportsFolder="/var/jenkins_home/workspace/SeleniumTest#tmp/withMavena8cb286b"
Apache Maven 3.6.0 (97c98ec64a1fdfee7767ce5ffb20918da4f719f3; 2018-10-24T14:41:47-04:00)
Maven home: /var/jenkins_home/tools/hudson.tasks.Maven_MavenInstallation/Maven_3.6.0
Java version: 1.8.0_181, vendor: Oracle Corporation, runtime: /usr/lib/jvm/java-8-openjdk-amd64/jre
Default locale: en, platform encoding: UTF-8
OS name: "linux", version: "3.10.0-957.21.2.el7.x86_64", arch: "amd64", family: "unix"
[INFO] [jenkins-event-spy] Generate /var/jenkins_home/workspace/SeleniumTest#tmp/withMavena8cb286b/maven-spy-20190719-093036-9118622080910645891447.log.tmp ...
[INFO] Scanning for projects...
[WARNING]
[WARNING] Some problems were encountered while building the effective model for FMCDealerFramework:FMCDealerFramework:jar:0.0.1-SNAPSHOT
[WARNING] 'dependencies.dependency.systemPath' for util.Xls_Reader:Xls_Reader:jar should not point at files within the project directory, ${project.basedir}/repository/xls_Reader.jar will be unresolvable by dependent projects # line 42, column 24
[WARNING]
[WARNING] It is highly recommended to fix these problems because they threaten the stability of your build.
[WARNING]
[WARNING] For this reason, future Maven versions might no longer support building such malformed projects.
[WARNING]
[INFO]
[INFO] ---------------< FMCDealerFramework:FMCDealerFramework >----------------
[INFO] Building FMCDealerFramework 0.0.1-SNAPSHOT
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) # FMCDealerFramework ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) # FMCDealerFramework ---
[INFO] Nothing to compile - all classes are up to date
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) # FMCDealerFramework ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /var/jenkins_home/workspace/SeleniumTest/src/test/resources
[INFO]
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) # FMCDealerFramework ---
[INFO] No sources to compile
[INFO]
[INFO] --- maven-surefire-plugin:3.0.0-M3:test (default-test) # FMCDealerFramework ---
[INFO] No tests to run.
[INFO]
[INFO] -------------------------------------------------------
[INFO] T E S T S
[INFO] -------------------------------------------------------
[INFO] Running TestSuite
...
... TestNG 6.14.3 by Cédric Beust (cedric#beust.com)
...
pass
pass
pass
pass
Read test data file mailreport.properties...
No Recepient Email Ids found.
No Recepient Email Ids found.
No Recepient Email Ids found.
[ERROR] Tests run: 12, Failures: 8, Errors: 0, Skipped: 4, Time elapsed: 1.232 s <<< FAILURE! - in TestSuite
[ERROR] setUpBrowser(com.ford.FMCDealer.testcases.TC04_FMC_Quick_Filters) Time elapsed: 0.362 s <<< FAILURE!
java.lang.IllegalStateException: The driver is not executable: /var/jenkins_home/workspace/SeleniumTest/./lib/chromedriver.exe
[ERROR] tearDown(com.ford.FMCDealer.testcases.TC04_FMC_Quick_Filters) Time elapsed: 0.443 s <<< FAILURE!
java.lang.NullPointerException
[ERROR] setUpBrowser(com.ford.FMCDealer.testcases.TC07_FMC_Stemming) Time elapsed: 0.446 s <<< FAILURE!
java.lang.IllegalStateException: The driver is not executable: /var/jenkins_home/workspace/SeleniumTest/./lib/chromedriver.exe
[ERROR] tearDown(com.ford.FMCDealer.testcases.TC07_FMC_Stemming) Time elapsed: 0.448 s <<< FAILURE!
java.lang.NullPointerException
[ERROR] setUpBrowser(com.ford.FMCDealer.testcases.TC05_FMC_Toggle_to_change_view) Time elapsed: 0.449 s <<< FAILURE!
java.lang.IllegalStateException: The driver is not executable: /var/jenkins_home/workspace/SeleniumTest/./lib/chromedriver.exe
[ERROR] tearDown(com.ford.FMCDealer.testcases.TC05_FMC_Toggle_to_change_view) Time elapsed: 0.45 s <<< FAILURE!
java.lang.NullPointerException
[ERROR] setUpBrowser(com.ford.FMCDealer.testcases.TC03_FMC_Sort_Option_Filter) Time elapsed: 0.453 s <<< FAILURE!
java.lang.IllegalStateException: The driver is not executable: /var/jenkins_home/workspace/SeleniumTest/./lib/chromedriver.exe
[ERROR] tearDown(com.ford.FMCDealer.testcases.TC03_FMC_Sort_Option_Filter) Time elapsed: 0.455 s <<< FAILURE!
java.lang.NullPointerException
[INFO]
[INFO] Results:
[INFO]
[ERROR] Failures:
[ERROR] TC03_FMC_Sort_Option_Filter>TestCaseBase.setUpBrowser:104->TestCaseBase.selectBrowser:157->TestCaseBase.setUpChromeWin32:351 » IllegalState
[ERROR] TC03_FMC_Sort_Option_Filter>TestCaseBase.tearDown:296 » NullPointer
[ERROR] TC04_FMC_Quick_Filters>TestCaseBase.setUpBrowser:104->TestCaseBase.selectBrowser:157->TestCaseBase.setUpChromeWin32:351 » IllegalState
[ERROR] TC04_FMC_Quick_Filters>TestCaseBase.tearDown:296 » NullPointer
[ERROR] TC05_FMC_Toggle_to_change_view>TestCaseBase.setUpBrowser:104->TestCaseBase.selectBrowser:157->TestCaseBase.setUpChromeWin32:351 » IllegalState
[ERROR] TC05_FMC_Toggle_to_change_view>TestCaseBase.tearDown:296 » NullPointer
[ERROR] TC07_FMC_Stemming>TestCaseBase.setUpBrowser:104->TestCaseBase.selectBrowser:157->TestCaseBase.setUpChromeWin32:351 » IllegalState
[ERROR] TC07_FMC_Stemming>TestCaseBase.tearDown:296 » NullPointer
[INFO]
[ERROR] Tests run: 12, Failures: 8, Errors: 0, Skipped: 4
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 6.880 s
[INFO] Finished at: 2019-07-19T09:30:44-04:00
[INFO] ------------------------------------------------------------------------
[INFO] [jenkins-event-spy] Generated /var/jenkins_home/workspace/SeleniumTest#tmp/withMavena8cb286b/maven-spy-20190719-093036-9118622080910645891447.log
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:3.0.0-M3:test (default-test) on project FMCDealerFramework: There are test failures.
[ERROR]
[ERROR] Please refer to /var/jenkins_home/workspace/SeleniumTest/target/surefire-reports for the individual test results.
[ERROR] Please refer to dump files (if any exist) [date].dump, [date]-jvmRun[N].dump and [date].dumpstream.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[Pipeline] }
[withMaven] junitPublisher - Archive test results for Maven artifact FMCDealerFramework:FMCDealerFramework:jar:0.0.1-SNAPSHOT generated by maven-surefire-plugin:test (default-test): target/surefire-reports/*.xml
[withMaven] junitPublisher - Jenkins JUnit Attachments Plugin not found, can't publish test attachments.Recording test results
[withMaven] Jenkins Task Scanner Plugin not found, don't display results of source code scanning for 'TODO' and 'FIXME' in pipeline screen.
[withMaven] Publishers: Pipeline Graph Publisher: 2 ms, Junit Publisher: 34 ms
[Pipeline] // withMaven
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // timeout
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
ERROR: script returned exit code 1
Finished: FAILURE
Given you're running your test on a Linux box you need to use Linux chromedriver version
I.e. for Chrome 75: https://chromedriver.storage.googleapis.com/index.html?path=75.0.3770.140/
You should be using https://chromedriver.storage.googleapis.com/75.0.3770.140/chromedriver_linux64.zip
Make sure to make it executable by running chmod command like:
script {
sh 'chmod +x /var/jenkins_home/workspace/SeleniumTest/./lib/chromedriver'
}
However it should be easier to integrate WebDriverManager to automatically download the necessary version of the driver for your browser.
If your Linux machine doesn't have GUI you will also either need to setup Xvfb or add --headless key to your browser initialization routine.
More information: How to use Jenkins pipeline to launch Selenium tests