The pre-commit run suddenly fails without changes being made to the pre-commit configuration file. The log shows the following error message when attempting to initialise the environment for Flake8:
...
[INFO] Initializing environment for https://gitlab.com/pycqa/flake8.git.
An unexpected error has occurred: CalledProcessError: command: ('/usr/bin/git', 'fetch', 'origin', '--tags')
return code: 128
expected return code: 0
stdout: (none)
stderr:
fatal: could not read Username for 'https://gitlab.com': No such device or address
...
The pre-commit configuration file includes the following entry:
---
repos:
- repo: https://gitlab.com/pycqa/flake8.git
rev: 3.9.2
hooks:
- id: flake8
exclude: 'tests|env|docs'
The Flake8 project has moved to GitHub and requires updating the repository URL in the pre-commit configuration file:
---
repos:
- repo: https://github.com/pycqa/flake8.git
rev: 3.9.2
hooks:
- id: flake8
exclude: 'tests|env|docs'
Related
I am trying to use Infrastructure as Code (IaC) Scanning scans my IaC configuration files for known vulnerabilities. Reference: https://docs.gitlab.com/ee/user/application_security/iac_scanning/. However, I encountered the system problem in the pipeline.
[DEBU] [kics] [2022-11-11T03:55:29Z] [/go/pkg/mod/gitlab.com/gitlab-org/security-products/analyzers/command#v1.9.2/run.go:256] ▶ SAST_EXCLUDED_PATHS=spec,test,tests,tmp39
[INFO] [kics] [2022-11-11T03:55:29Z] [/go/pkg/mod/gitlab.com/gitlab-org/security-products/analyzers/command#v1.9.2/run.go:131] ▶ Detecting project
[INFO] [kics] [2022-11-11T03:55:29Z] [/go/pkg/mod/gitlab.com/gitlab-org/security-products/analyzers/command#v1.9.2/run.go:153] ▶ Analyzer will attempt to analyze all projects in the repository41
[INFO] [kics] [2022-11-11T03:55:29Z] [/go/pkg/mod/gitlab.com/gitlab-org/security-products/analyzers/command#v1.9.2/run.go:165] ▶ Running analyzer42
[DEBU] [kics] [2022-11-11T03:55:29Z] [/go/src/app/analyze.go:48] ▶ custom rulesets not enabled
[INFO] [kics] [2022-11-11T03:55:29Z] [/go/src/app/analyze.go:67] ▶ path /builds/leo/tcf
[ERRO] [kics] [2022-11-11T03:55:38Z] [/go/src/app/analyze.go:87] ▶ Encountered a system problem; status code: 2, error: exit status 2, detail:
[FATA] [kics] [2022-11-11T03:55:38Z] [/go/src/app/main.go:30] ▶ kics scanner failure: exit status 2
Uploading artifacts for failed job00:0048Uploading artifacts...
WARNING: gl-sast-report.json: no matching files. Ensure that the artifact path is relative to the working directory
ERROR: No files to upload
Cleaning up project directory and file based variables00:0154ERROR: Job failed: exit code 1
I can't find relevant information about the errors. How can I solve the errors?
You can use your own config, independent of the GitLab template.
gitlab-ci.yml:
image:
name: checkmarx/kics:latest
entrypoint: [""]
script:
- kics scan -q /app/bin/assets/queries -p ${PWD} --ignore-on-exit all --report-formats glsast -o ${PWD} --output-name kics-results
artifacts:
reports:
sast: gl-sast-kics-results.json
when: always
Goal: run .py files via. dvc.yaml.
There are stages before it, in dvc.yaml, that don't produce the error.
dvc exp run:
(venv) me#ubuntu-pcs:~/PycharmProjects/project$ dvc exp run
Stage 'inference' didn't change, skipping
Running stage 'load_data':
> load_data.py
/bin/bash: line 1: load_data.py: Permission denied
ERROR: failed to reproduce 'load_data': failed to run: load_data.py, exited with 126
dvc repro:
(venv) me#ubuntu-pcs:~/PycharmProjects/project$ dvc repro
Stage 'predict' didn't change, skipping
Stage 'evaluate' didn't change, skipping
Stage 'inference' didn't change, skipping
Running stage 'load_data':
> load_data.py
/bin/bash: line 1: load_data.py: Permission denied
ERROR: failed to reproduce 'load_data': failed to run: pdl1_lung_model/load_data.py, exited with 126
dvc doctor:
DVC version: 2.10.2 (pip)
---------------------------------
Platform: Python 3.9.12 on Linux-5.15.0-46-generic-x86_64-with-glibc2.35
Supports:
webhdfs (fsspec = 2022.5.0),
http (aiohttp = 3.8.1, aiohttp-retry = 2.5.2),
https (aiohttp = 3.8.1, aiohttp-retry = 2.5.2),
s3 (s3fs = 2022.5.0, boto3 = 1.21.21)
Cache types: hardlink, symlink
Cache directory: ext4 on /dev/nvme0n1p5
Caches: local
Remotes: s3
Workspace directory: ext4 on /dev/nvme0n1p5
Repo: dvc, git
dvc exp run -v:
output.txt
dvc exp run -vv:
output2.txt
Solution 1
.py files weren't running as scripts.
They need to be; if you want to run one .py file per stage in dvc.yaml.
To do so, you want to append Boiler-plate code, at the bottom of each .py file.
if __name__ == "__main__":
# invoke primary function() in .py file, w/ params
Solution 2
chmod 777 ....py
Soution 3
I forgot the python in cmd:
load_data:
cmd: python pdl1_lung_model/load_data.py
I have an error when using the template for IaC SAST.
Gitlab is 14.10, installed in my cloud, and accessible from the internet (I changed my real GitLab ULR to owngitlab.com)
The GitLab runner is installed in k8s.
Try to use SAST in my projects, but I get errors. Could somebody please help with this?
Here is part of my .gitlab-ci.yml
include:
- template: Security/SAST-IaC.latest.gitlab-ci.yml
stages:
- test
Pipeline log:
$ /analyzer run
[INFO] [kics] [2022-05-20T12:41:44Z] ▶ GitLab kics analyzer v1.5.2
[INFO] [kics] [2022-05-20T12:41:44Z] ▶ Detecting project
[INFO] [kics] [2022-05-20T12:41:44Z] ▶ Found relevant files in project, analyzing entire repository
[INFO] [kics] [2022-05-20T12:41:44Z] ▶ Running analyzer
[INFO] [kics] [2022-05-20T12:41:44Z] ▶ path /builds/tf/modules
[INFO] [kics] [2022-05-20T12:42:43Z] ▶ Creating report
[WARN] [kics] [2022-05-20T12:42:43Z] ▶ Could not detect git executable
Uploading artifacts for successful job
Uploading artifacts...
gl-sast-report.json: found 1 matching files and directories
ERROR: Uploading artifacts as "sast" to coordinator... error error=couldn't execute POST against https://owngitlab.com/api/v4/jobs/139/artifacts?artifact_format=raw&artifact_type=sast: Post "https://owngitlab.com/api/v4/jobs/139/artifacts?artifact_format=raw&artifact_type=sast": open gl-sast-report.json: permission denied id=139 token=hZ5dXhA7
WARNING: Retrying... context=artifacts-uploader error=invalid argument
ERROR: Uploading artifacts as "sast" to coordinator... error error=couldn't execute POST against https://owngitlab.com/api/v4/jobs/139/artifacts?artifact_format=raw&artifact_type=sast: Post "https://owngitlab.com/api/v4/jobs/139/artifacts?artifact_format=raw&artifact_type=sast": open gl-sast-report.json: permission denied id=139 token=hZ5dXhA7
WARNING: Retrying... context=artifacts-uploader error=invalid argument
ERROR: Uploading artifacts as "sast" to coordinator... error error=couldn't execute POST against https://owngitlab.com/api/v4/jobs/139/artifacts?artifact_format=raw&artifact_type=sast: Post "https://owngitlab.com/api/v4/jobs/139/artifacts?artifact_format=raw&artifact_type=sast": open gl-sast-report.json: permission denied id=139 token=hZ5dXhA7
FATAL: invalid argument
Cleaning up project directory and file based variables
00:00
ERROR: Job failed: command terminated with exit code 1
I am getting the following error while setting up the development environment for fabric2.3.
go version go1.17.1 linux/amd64
setup environment: Ubuntu 20.04 virtual machine
CLI output after running 'make dist-clean all'
Testing with pkcs11...
Trying configuration [0]
--- FAIL: TestGetBCCSPFromOpts (0.00s)
pkcs11_test.go:67:
Error Trace: pkcs11_test.go:67
Error: Received unexpected error:
pkcs11: library path not provided
Could not initialize BCCSP PKCS11
github.com/hyperledger/fabric/bccsp/factory.GetBCCSPFromOpts
/home/souptik/github.com/souptikmakarov/fabric/bccsp/factory/pkcs11.go:93
github.com/hyperledger/fabric/bccsp/factory.TestGetBCCSPFromOpts
/home/souptik/github.com/souptikmakarov/fabric/bccsp/factory/pkcs11_test.go:57
testing.tRunner
/usr/local/go/src/testing/testing.go:1193
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1371
Test: TestGetBCCSPFromOpts
--- FAIL: TestPKCS11FactoryGet (0.00s)
pkcs11factory_test.go:48:
Error Trace: pkcs11factory_test.go:48
Error: Received unexpected error:
pkcs11: library path not provided
Test: TestPKCS11FactoryGet
--- FAIL: TestPKCS11FactoryGetEmptyKeyStorePath (0.00s)
pkcs11factory_test.go:66:
Error Trace: pkcs11factory_test.go:66
Error: Received unexpected error:
pkcs11: library path not provided
Test: TestPKCS11FactoryGetEmptyKeyStorePath
FAIL
I have even tried installing pkcs11-proxy from https://github.com/SUNET/pkcs11-proxy but that didn't solve the issue
Any help is appreciated in fixing this
I try to use artifacts on my Gitlab and when my runner try to put my artifact file on Giltlab, i have this message :
WARNING: Uploading artifacts to coordinator... failed id=504 responseStatus=405 Method Not Allowed status=405 Method Not Allowed token=HebbSqMD
WARNING: Retrying...
WARNING: Uploading artifacts to coordinator... failed id=504 responseStatus=405 Method Not Allowed status=405 Method Not Allowed token=HebbSqMD
WARNING: Retrying...
WARNING: Uploading artifacts to coordinator... failed id=504 responseStatus=405 Method Not Allowed status=405 Method Not Allowed token=HebbSqMD
FATAL: invalid argument
ERROR: Job failed: exit code 1
My job is :
build:
stage: build
script:
- echo "Building the app" > out.txt
artifacts:
paths:
- out.txt
Gitlab 8.16 or 8.17 : same probleme
Gitlab runner on other host (docker executor) : 1.11.0 o 1.10.4 : same probleme
Simply, verified artifacts enabled in /etc/gitlab/gitlab.rb file:
## Build Artifacts
# gitlab_rails['artifacts_enabled'] = true
# gitlab_rails['artifacts_path'] = "/mnt/storage/artifacts"