I am trying to integrate Gitlab SAST into my pipeline and facing the following error for SemGrep analyzer during generation of the report.
The error:
[DEBU] [Semgrep] [2022-10-11T10:02:55Z] [/go/src/buildapp/analyze.go:137] METRICS: Using configs from the Registry (like --config=p/ci) reports pseudonymous rule metrics to semgrep.dev.
To disable Registry rule metrics, use "--metrics=off".
Using configs only from local files (like --config=xyz.yml) does not enable metrics.
More information: https://semgrep.dev/docs/metrics
Scanning across multiple languages:
java | 86 rules × 37 files
js | 13 rules × 1 file
[INFO] [Semgrep] [2022-10-11T10:02:55Z] [/go/pkg/mod/gitlab.com/gitlab-org/security-products/analyzers/command#v1.9.1/run.go:179] Creating report
[DEBU] [Semgrep] [2022-10-11T10:02:55Z] [/go/src/buildapp/convert.go:25] Converting report with the root path: /path/to/my/repo
[FATA] [Semgrep] [2022-10-11T10:02:55Z] [/go/src/buildapp/main.go:27] tool notification error: SemgrepError Error while running rules: 0 bytes read on a total of 2 expected bytes
Any ideas where to look?
The change Gitlab runner to bigger one (cpu, ram) helped.
I am trying to deploy some of my jar libraries through JitPack. So far I am still testing things out, thus version codes are dev-SNAPSHOT or master-SNAPSHOT for the libraries.
For most libraries this seems to work well (at least as far as fetching the artifacts goes); however, one library had a failed build for master-SNAPSHOT.
The corresponding build, master-36ef0715cd-1, reports failure. The last lines of the log read:
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 18.995 s
[INFO] Finished at: 2021-01-23T06:52:54Z
[INFO] ------------------------------------------------------------------------
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8 -Dhttps.protocols=TLSv1.2
Found module: org.traffxml:traff-source-android:0.0.1-SNAPSHOT
Build tool exit code: 0
Looking for artifacts...
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8 -Dhttps.protocols=TLSv1.2
Picked up JAVA_TOOL_OPTIONS: -Dfile.encoding=UTF-8 -Dhttps.protocols=TLSv1.2
Looking for pom.xml in build directory and ~/.m2
Found artifact: org.traffxml:traff-source-android:0.0.1-SNAPSHOT
ERROR: Time-out getting container status
Error building
So, apparently, the jar got built but fetching container status failed after that. This could well be an issue with JitPack’s infrastructure: over the last couple hours, it has taken several retries to get the artifacts to build, and I do not see any difference between this library and the others which built successfully.
How can I retry the failed build, or otherwise fix this (other than by going the crude way of pushing a new commit to my repo)?
Eventually I had to commit more changes, which eventually solved this for the moment.
On the long run, one thing to try:
Go to https://jitpack.io/
In the box, enter your group and artifact ID, and click Look up.
In the list of availale builds, click the Branches tab.
Look for the corresponding -SNAPSHOT build for your branch, and click Get it.
No guarantee this works, but it did trigger a new build after pushing a new commit. Feedback appreciated.
This was originating from Azure DevOps pipilene while running an analysis with SonarQube.
I tried to apply below steps but no luck.
NPM Task
npm install -g increase-memory-limit
Pipeline Variable
SONAR_SCANNER_OPTS = -Xmx4096m
Error Log
INFO: Sensor TypeScript analysis [javascript]
INFO: Using TypeScript at: 'D:\a\45\s\Source\RockyBrands.MODocuments\Presentation Tier\RockyBrands.MODocuments.Web.UI\node_modules'
INFO: Found 2 tsconfig.json file(s): [D:\a\45\s\Source\RockyBrands.MODocuments\Presentation Tier\RockyBrands.MODocuments.Web.UI\obj\Dev\Package\PackageTmp\tsconfig.json, D:\a\45\s\Source\RockyBrands.MODocuments\Presentation Tier\RockyBrands.MODocuments.Web.UI\tsconfig.json]
INFO: 23 source files to be analyzed
INFO: Analyzing 23 files using tsconfig: D:\a\45\s\Source\RockyBrands.MODocuments\Presentation Tier\RockyBrands.MODocuments.Web.UI\tsconfig.json
INFO: 0/23 files analyzed, current file: Source/RockyBrands.MODocuments/Presentation Tier/RockyBrands.MODocuments.Web.UI/app/Home/Payments/batch-details/batch.details.component.ts
INFO: 0/23 files analyzed, current file: Source/RockyBrands.MODocuments/Presentation Tier/RockyBrands.MODocuments.Web.UI/app/Home/Payments/batch-details/batch.details.component.ts
INFO: 0/23 files analyzed, current file: Source/RockyBrands.MODocuments/Presentation Tier/RockyBrands.MODocuments.Web.UI/app/Home/Payments/batch-details/batch.details.component.ts
INFO: 0/23 files analyzed, current file: Source/RockyBrands.MODocuments/Presentation Tier/RockyBrands.MODocuments.Web.UI/app/Home/Payments/batch-details/batch.details.component.ts
INFO: 0/23 files analyzed, current file: Source/RockyBrands.MODocuments/Presentation Tier/RockyBrands.MODocuments.Web.UI/app/Home/Payments/batch-details/batch.details.component.ts
**##[error]ERROR: eslint-bridge Node.js process is unresponsive. This is most likely caused by process running out of memory. Consider setting sonar.javascript.node.maxspace to higher value (e.g. 4096).
ERROR: eslint-bridge Node.js process is unresponsive. This is most likely caused by process running out of memory. Consider setting sonar.javascript.node.maxspace to higher value (e.g. 4096).
##[error]ERROR: Failure during analysis, Node.js command to start eslint-bridge was: {NODE_PATH=D:\a\45\s\Source\RockyBrands.MODocuments\Presentation Tier\RockyBrands.MODocuments.Web.UI\node_modules} node D:\a\45\.sonarqube\out\.sonar\.sonartmp\eslint-bridge-bundle\package\bin\server 49855
java.lang.IllegalStateException: eslint-bridge is unresponsive**
at org.sonar.plugins.javascript.eslint.EslintBridgeServerImpl.request(EslintBridgeServerImpl.java:202)
at org.sonar.plugins.javascript.eslint.EslintBridgeServerImpl.analyzeTypeScript(EslintBridgeServerImpl.java:186)
ERROR: eslint-bridge Node.js process is unresponsive. This is most likely caused by process running out of memory
You could try to added sonar.javascript.node.maxspace=8192 in the sonar-project.properties.
Besides, if you are using the private agent, you need to consider the RAM size of the machine where your private agent is located. You can try to reduce the number of files contained in tsconfig.json files.
And if you are using the hosted agent, you could try to increase the size of the memory using the command. (e.g node --max-old-space-size=16384./node_modules/#angular/cli/bin/ng build --prod). Please reference this similar thread : https://github.com/TrilonIO/aspnetcore-angular-universal/issues/736#issuecomment-517374967. According to codehippie1’s comment, it needs same changes in csproj and package.json.
We are trying to scan our docker images using Anchore Engine Jenkins plugin.
Currently we create our application docker images, push it in our own private local registry and then deploy it in our test environments.
Now, we want to setup docker image scanning in our CI/CD process to check for any vulnerabilities.
We have installed Anchore Engine using the recommended Docker-Compose yaml method given in the Documentation link:
https://anchore.freshdesk.com/support/solutions/articles/36000020729-install-on-docker-swarm
Post installation, we installed the
Anchore Container Image Scanner Plugin in Jenkins.
We configured the plugin as mentioned in the document link:
https://wiki.jenkins.io/display/JENKINS/Anchore+Container+Image+Scanner+Plugin
However, the scanning fails. Error Message as follows:
2018-10-11T07:01:44.647 INFO AnchoreWorker Analysis request accepted, received image digest sha256:7d6fb7e5e7a74a4309cc436f6d11c29a96cbf27a4a8cb45a50cb0a326dc32fe8
2018-10-11T07:01:44.647 INFO AnchoreWorker Waiting for analysis of 10.180.25.2:5000/hello-world:latest, polling status periodically
2018-10-11T07:01:44.647 DEBUG AnchoreWorker anchore-engine get policy evaluation URL: http://10.180.25.2:8228/v1/images/sha256:7d6fb7e5e7a74a4309cc436f6d11c29a96cbf27a4a8cb45a50cb0a326dc32fe8/check?tag=10.180.25.2:5000/hello-world:latest&detail=true
2018-10-11T07:01:44.648 DEBUG AnchoreWorker Attempting anchore-engine get policy evaluation (1/300)
2018-10-11T07:01:44.675 DEBUG AnchoreWorker anchore-engine get policy evaluation failed. URL: http://10.180.25.2:8228/v1/images/sha256:7d6fb7e5e7a74a4309cc436f6d11c29a96cbf27a4a8cb45a50cb0a326dc32fe8/check?tag=10.180.25.2:5000/hello-world:latest&detail=true, status: HTTP/1.1 404 NOT FOUND, error: {
"detail": {},
"httpcode": 404,
"message": "image is not analyzed - analysis_status: not_analyzed"
}
NOTE:
In Image TAG 10.180.25.2:5000/hello-world:latest, 10.180.25.2:5000 is our local private registry and hello-world:latest is latest hello-world image available in docker hub which we pulled and pushed in our registry to try out image scanning using Anchore-Engine.
Unfortunately we are not able to find much resource online to try and resolve the above mentioned issue.
Anyone who might have worked on Anchore-Engine, please may I request to have a look and help us resolve this issue.
Also, any suggestions or alternatives to anchore-engine or detailed steps in case we might have missed anything would be really appreciated.
End of the output is as follows:
2018-10-15T00:48:43.880 WARN AnchoreWorker anchore-engine get policy evaluation failed. HTTP method: GET, URL: http://10.180.25.2:8228/v1/images/sha256:7d6fb7e5e7a74a4309cc436f6d11c29a96cbf27a4a8cb45a50cb0a326dc32fe8/check?tag=10.180.25.2:5000/hello-world:latest&detail=true, status: 404, error: {
"detail": {},
"httpcode": 404,
"message": "image is not analyzed - analysis_status: not_analyzed"
}
2018-10-15T00:48:43.880 WARN AnchoreWorker Exhausted all attempts polling anchore-engine. Analysis is incomplete for sha256:7d6fb7e5e7a74a4309cc436f6d11c29a96cbf27a4a8cb45a50cb0a326dc32fe8
2018-10-15T00:48:43.880 ERROR AnchorePlugin Failing Anchore Container Image Scanner Plugin step due to errors in plugin execution
hudson.AbortException: Timed out waiting for anchore-engine analysis to complete (increasing engineRetries might help). Check above logs for errors from anchore-engine
at com.anchore.jenkins.plugins.anchore.BuildWorker.runGatesEngine(BuildWorker.java:480)
at com.anchore.jenkins.plugins.anchore.BuildWorker.runGates(BuildWorker.java:343)
at com.anchore.jenkins.plugins.anchore.AnchoreBuilder.perform(AnchoreBuilder.java:338)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.Build$BuildExecution.build(Build.java:206)
at hudson.model.Build$BuildExecution.doRun(Build.java:163)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:421)
I also checked status and found below:
docker run anchore/engine-cli:latest anchore-cli --u admin --p admin123 --url http://172.18.0.1:8228/v1 system status
Service analyzer (dockerhostid-anchore-engine, http://anchore-engine:8084): up
Service catalog (dockerhostid-anchore-engine, http://anchore-engine:8082): up
Service policy_engine (dockerhostid-anchore-engine, http://anchore-engine:8087): down (unavailable)
Service simplequeue (dockerhostid-anchore-engine, http://anchore-engine:8083): up
Service apiext (dockerhostid-anchore-engine, http://anchore-engine:8228): up
Service kubernetes_webhook (dockerhostid-anchore-engine, http://anchore-engine:8338): up
Engine DB Version: 0.0.7
Engine Code Version: 0.2.4
It seems service policy engine is down
Service policy_engine (dockerhostid-anchore-engine, http://anchore-engine:8087): down (unavailable)
I also checked the docker logs . I found below error:
[service:policy_engine] 2018-10-15 09:37:46+0000 [-] [bootstrap] [DEBUG] service (policy_engine) starting in: 4
[service:policy_engine] 2018-10-15 09:37:46+0000 [-] [bootstrap] [INFO] Registration complete.
[service:policy_engine] 2018-10-15 09:37:46+0000 [-] [bootstrap] [INFO] Checking feeds client credentials
[service:policy_engine] 2018-10-15 09:37:46+0000 [-] [bootstrap] [DEBUG] Initializing a feeds client
[service:policy_engine] 2018-10-15 09:37:47+0000 [-] [bootstrap] [DEBUG] init values: [None, None, None, (), None, None]
[service:policy_engine] 2018-10-15 09:37:47+0000 [-] [bootstrap] [DEBUG] using values: ['https://ancho.re/v1/service/feeds', 'https://ancho.re/oauth/token', 'https://ancho.re/v1/account/users', 'anon#ancho.re', 3, 60]
[service:policy_engine] 2018-10-15 09:37:47+0000 [-] [urllib3.connectionpool] [DEBUG] Starting new HTTPS connection (1): ancho.re
[service:policy_engine] 2018-10-15 09:37:50+0000 [-] [bootstrap] [ERROR] Preflight checks failed with error: HTTPSConnectionPool(host='ancho.re', port=443): Max retries exceeded with url: /v1/account/users/anon#ancho.re (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7ffa905f0b90>: Failed to establish a new connection: [Errno 113] No route to host',)). Aborting service startup
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/anchore_manager/cli/service.py", line 158, in startup_service
raise Exception("process exited: " + str(rc))
Exception: process exited: 1
[anchore-policy-engine] [anchore_manager.cli.service/startup_service()] [INFO] service process exited at (Mon Oct 15 09:37:50 2018): process exited: 1
[anchore-policy-engine] [anchore_manager.cli.service/startup_service()] [INFO] exiting service thread
Thanks and Regards,
Rohan Shetty
When images are added to anchore-engine, they are queued for analysis which moves them through a simple state machine that starts with ‘not_analyzed’, goes to ‘analyzing’ and finally ends in either ‘analyzed’ or ‘analysis_failed’. Only when an image has reached ‘analyzed’ will a policy evaluation be possible.
The anchore Jenkins plugin will add an image, then poll the engine for image status/evaluation for the configured number of tries (default 300). Once the image goes to ‘analyzed’ (where policy evaluation is possible), the plugin will then receive a policy evaluation result from the engine.
The plugin will fail the build (by default) if the max retries has been performed and the image has not reached ‘analyzed’, if the image does reach ‘analyzed’ but the policy evaluation is producing a ‘fail’ result (meaning the image didn’t pass your configured policy checks). Note that all build failure behavior can be controlled in the plugin (I.e. there are options to allow the plugin to succeed even if the analysis or image eval fails).
You’ll need to look at the end of the output from your build run (instead of just the beginning from your post), and combined with the information above, it should be clear which scenario is causing the plugin to fail the build.
We have resolved the issue.
Root Cause:
We were not able to establish a successful https connection to URL : https://ancho.re from within the anchore-engine docker container.
As a result the service:policy_engine was not able to start.
https://ancho.re is required to download policy feeds and sync-up periodically. Without these policy anchore-engine won't be able to analyse the docker images.
Solution:
1) We passed a HTTPS_PROXY URL as an environment variable in the docker-compose.yaml of anchore-engine.
We used this proxy URL to bypass restrictions in our environment and establish a connection with https://ancho.re url.
2) Restarted the docker containers.
Finally we got all services up and running including Anchore policy-engine.
FYI:
It takes a while to download all the required Feeds depending on your internet speed.
Lastly, Thanks to the Anchore community for quick responses and support over slack.
Hope this helps.
Warm Regards,
Rohan Shetty
My app is actually running fine when I started it and keep an eye on it for few hours.
BUT later time(I'm not sure what exact time of inactivity), It does show "Server timed out" (I crop some logs below)
[ERROR] (server - L:463) <node.mycouchbase.server:11210> (SRV=0x2111c60,IX=4) Server timed out. Some commands have failed
[INFO] (confmon - L:166) Not applying configuration received via CCCP. No changes detected. A.rev=152466, B.rev=152466
[INFO] (cccp - L:110) Re-Issuing CCCP Command on server struct 0x2116980
[ERROR] (cccp - L:133) <NOHOST:NOPORT> Got I/O Error=0x17
[INFO] (cccp - L:110) Re-Issuing CCCP Command on server struct 0x2185e30
All just working fine again when I restart my Node.js app (Expressjs app).
This problem seem regularly happening
Please give me some suggestion what could be actually problems behind.
Thanks