I tried executing this command in the Terminal(and my environment is Ubuntu22.04) and it worked well with some Loading Effects, but when I configure this command in my docker container(created by my CI/CD pipeline executor) it does not work properly when the CI/CD pipeline runs. Below is my configuration and execution error:
- nsenter --mount=/host/proc/1/ns/mnt sh -c "cd /home/..USER../Desktop/..PROJECT..&&export https_proxy=..PROXY../&&gh pr create --title "test" --body "test" --base cicd_test"
Error in the CI/CD Console logs which runs properly in the local Terminal with some loading effects
hope run properly in the docker or some other alternative command that may work
Related
Hi people of the internet.
Basically I am unable to run even the simplest job and I keep getting the same error no matter what I put in the .gitlab-ci.yml file. See example below:
Here is the .gitlab-ci.yml file:
stages:
- test
job1:
stage: test
tags:
- testing
script:
- echo "Hello world!"
Here is the output ("?" corresponds to intentionally blacked out information):
Running with gitlab-runner 14.10.0 (c6bb62f6)
on runner_test ????????
Preparing the "shell" executor
00:00
Using Shell executor...
Preparing environment
00:00
Running on LAPTOP-????????...
Getting source from Git repository
00:01
WriteError:
Line |
219 | $HOST="[MASKED]"
| ~~~~~~~~~~~~~~~~~~~~~~
| Cannot overwrite variable Host because it is read-only or constant.
ERROR: Job failed: exit status 1
I know that $HOST is a reserved variable in powershell but I don't see the link between the error and the code. It may have something to do with the configuration of the runner on Windows. Has anyone encountered this error on Gitlab before? Or any suggestions on how to debug?
Here are the steps that I took to install the runner on Gitlab for Windows (see https://docs.gitlab.com/runner/install/windows.html):
Create a folder somewhere in the system: C:\GitLab-Runner.
Download the binary for 64-bit and put it into the folder (see https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-windows-amd64.exe).
Run prompt as an administrator
Run the following command:
cd C:\GitLab-Runner
gitlab-runner.exe register
Enter your GitLab instance URL (see Gitlab > Settings > CI/CD > Runners > Specific runners)
Enter the token to register the runner (see Gitlab > Settings > CI/CD > Runners > Specific runners)
Enter a description for the runner: runner_test for instance
Enter the tags associated with the runner, separated by commas: testing, windows for instance
Provide the runner executor: shell
Install GitLab Runner as a service and start it
cd C:\GitLab-Runner
gitlab-runner.exe install
gitlab-runner.exe start
I also had to install the latest version of pwsh in Windows (see gitlab-runner: prepare environment failed to start process pwsh in windows):
Run prompt as an administrator
Install the newer pwsh.exe:
winget install Microsoft.PowerShell
Restart the runner
cd C:\GitLab-Runner
gitlab-runner.exe restart
This issue was due to my choice of shell for some reason. A Gitlab runner can choose a shell among the following: bash, sh, powershell, pwsh, and cmd (the last one being deprecated now).
As I stated above I had been using pwsh. So, I went after the config.toml file inside of the C:\GitLab-Runner directory to manually make the change from pwsh to powershell.
...
[[runners]]
name = "runner_test"
executor = "shell"
shell = "powershell"
...
I then restarted the runner and got the job to complete properly:
cd C:\GitLab-Runner
gitlab-runner restart
I still get the error (more like a warning now) but it does not prevent the job from finishing anymore. If anyone has a better answer with a proper explanation I would gladly accept it as the answer to this question.
Note that pwsh to powershell are both powershell scripts (see https://docs.gitlab.com/runner/shells/index.html):
powershell Fully Supported PowerShell script. All commands are executed in PowerShell Desktop context. In GitLab Runner 12.0-13.12, this is the default when registering a new runner.
pwsh Fully Supported PowerShell script. All commands are executed in PowerShell Core context. In GitLab Runner 14.0 and later, this is the default when registering a new runner.
What I'm doing
I am using AWS batch to run a docker container for a large compute job. I have configured the ECR/ECS successfully to the best of my knowledge but am having issues running the required commands for reasons that are beyond my level of understanding with docker ( newbie )
What I need to do is pass the below commands into my application and start my application to perform some heavy computing tasks; all commands listed below must be present.
The Issue(s)
The issue arises when I send the submit job to AWS batch; this service pulls the image from the ACR ( amazon container repository ) and spins up a compute environment. The issue comes from when I try to run the command I pass in, below I will go throgh it.
"command": [
"mkdir -p logging",
"chmod 777 logging/",
"docker run -t -i -e my-application", # container name
"-e APIKEY",
"-e BASEURI",
"-e APIUSER",
"-v WORKSPACE /logging:/src/log",
"DOCKERIMAGE",
"python my_app.py",
"-t APP_USER",
"-e APP_ENVIRONMENT",
"-u APP_USERNAME",
"-p APP_PASSWORD",
"-i IN_PATH",
"-o OUT_PATH",
"-b tmp/"
]
The command above generates the following error(s)
container_linux.go:370: starting container process caused: exec: "mkdir -p log": executable file not found in $PATH
I tried to pass in the command to echo the env var $PATH but was unsuccesfull getting a response and resulted in a similar error.
I have ran successfully "ls" and was able to see the directory contents of my application inside.
I am not however able to run any of these commands that I have included in the command [] section. I have tried just running python and such in hopes of getting a more detailed error but was unsuccessful.
Logic in plain English
Create a path called logging if it doesnt exist
set the permissions for logging
run the docker container and pass in the environment variables while doing so
Tell docker to run the python file my_app.py and pass in the expected runtime args
Execute and perform the required logic deligated in the python3 application
Questions
Why can I not create a directory here called "logging" where am I ?
Am I running these properly as defined by AWS batch? or docker
What am I missing or where am I going wrong?
AWS Batch high level doc
AWS Batch link specific to what i'm doing
Assuming that you're following the syntax described in the Container
Properties
section of the AWS docs, you have several problems with the syntax of
your command directive.
First
The command directive can only run a single command. You can't mash together a bunch of commands as you're trying to do in your example. If you need to run multiple commands you would need to embed them as an argument to a shell. For example, something like:
command: ["/bin/sh", "-c", "mkdir -p logging; chmod 777 logging; ..."]
Second
You must properly tokenize your
command lines -- that is, when you type mkdir -p logging at the
command prompt, the shell splits this into three parts (or "tokens"): ['mkdir', '-p', 'logging']. You need to do the same thing when building up the
list of arguments to command.
This is invalid:
command: ["mkdir -p logging"]
That would looking for a command named mkdir -p logging, and of course no such command exists. That would properly be written as:
command: ["mkdir", "-p", "logging"]
Third
I'm not very familiar with the AWS batch environment, but it's unlikely you can run a docker command inside a docker` container as you're trying to do. It's unclear why you're doing this, though: why not just configure your AWS batch job with the appropriate image, environment variables, etc?
Take a look at some of these example job definitions.
During executing Puppet task run --nodes ip command i get reports/logs for most of the servers but only for a single server i get "no output for this node ".Even changes are not reflected in that machine while the same command works for rest of the machines.But at the same time job is not failed,it succeeds.
Note: I have a .sh file inside module/tasks/init.sh which contains puppet agent -t --tags (i pass parameters) command in it.Upon executing puppet task run command,puppet agent -t will be ran on those machines.
What will be the cause and is there anything not allowing to run that .sh file?
Update:
tasks module - modulename/tasks/init.sh
#!/bin/bash
puppet agent -t --noop --tags $PT_class --environment=$PT_env
Command to be executed in master:
puppet task run task_module_name class=modulename env=staging --query 'inventory { trusted.extensions.certname = "fqdn"}'
When in run this commadn it gets executed successfully but tasks module is not executed on agent machine
Note: I face this only on CIS hardening AMI on AWS
I install a Specific Runners,and the status is actived.
my .gitlab-ci.ymi file code:
stages:
- build
build_maven:
stage: build
only:
- master
script:
- echo "hello CI/CD"
tags:
- vue-dev-pub
when I push the master branch,the gitlab-runner is running,but it's pending infinitely。
the job page show:
This job has not started yet
This job is in pending state and is waiting to be picked by a runner
if I excute the runner manually,the job can pass.
the command of gitlab-runner verify shows:
Runtime platform arch=amd64 os=linux pid=24616 revision=d0b76032 version=12.0.2
WARNING: Running in user-mode.
WARNING: The user-mode requires you to manually start builds processing:
WARNING: $ gitlab-runner run
WARNING: Use sudo for system-mode:
WARNING: $ sudo gitlab-runner...
Verifying runner... is alive runner=T4iKvsT3
I am waiting for you respond,thanks!
If you run the runner manually in debug mode gitlab-runner --debug run you may see the actual error message, in my case it was:
WARNING: Failed to process runner builds=0 error=failed to update executor: missing Machine options executor=docker+machine runner=pSUsX4yR
That's because on runner creation, I selected option docker+machine rather than docker.
After amending /etc/gitlab-runner/config.toml to docker and running gitlab-runner restart followed by gitlab-runner verify, pipeline started running again.
I had a similar problem with my (shell) runners on linux. It would work fine on runners installed and registered on one of my computers but not another. (Even as tags matched correctly in runner and job)
After
gitlab-runner register I would get:
New runner. Has not connected yet
After
gitlab-runner verify that error would go away. But I would get This job is in pending state and is waiting to be picked by a runner
After
gitlab-runner restart
It would all work.
gitlab-runner status
gitlab-runner: Service is running!
Maybe you have tagged your runner but your job has no tags. Refer :how to run untagged jobs
https://stackoverflow.com/a/53371027/10570524
The tags section in your .gitlab-ci.yml file specifies this job has to be picked by a runner that has the same tags (reference).
tags:
- vue-dev-pub
So unless there is actually a runner available for your project that has the vue-dev-pub tag it will keep waiting for one to become available.
first, remove the old config in sys
rm /etc/systemd/system/gitlab-runner.servicetemd
now, you need install gitlab-runner with gitlab user:
gitlab-runner install --user=gitlab-runner --working-directory=/home/gitlab-runner
root installations fail
I've failed on running job by gitlab-runner using shell-executor.My setup is one server(centos6), with Gitlab installed, and the same server has the runners installed and configured.
Gitlab v11.8.0-ee
My project has a very simple yaml config file:
dev:
script:
- pwd
when running a job,I get the following output:
[0KRunning with gitlab-runner 11.8.0 (4745a6f3)
[0;m[0K on holiday zs7axHmm
[0;m[0KUsing Shell executor...
[0;msection_start:1551421397:prepare_script
[0KRunning on host-192-168-200-85...
section_end:1551421397:prepare_script
[0Ksection_start:1551421397:get_sources
[0K[32;1mFetching changes...[0;m
HEAD is now at ba9b6da Add new file
[32;1mChecking out ba9b6dac as master...[0;m
[32;1mSkipping Git submodules setup[0;m
section_end:1551421398:get_sources
[0Ksection_start:1551421398:restore_cache
[0Ksection_end:1551421398:restore_cache
[0Ksection_start:1551421398:download_artifacts
[0Ksection_end:1551421398:download_artifacts
[0Ksection_start:1551421398:build_script
[0K[32;1m$ # Auto DevOps variables and functions # collapsed multi-line command[0;m
[32;1m$ setup_docker[0;m
[32;1m$ build[0;m
Logging to GitLab Container Registry with CI credentials...
bash: line 91: docker: command not found
section_end:1551421398:build_script
[0Ksection_start:1551421398:after_script
[0Ksection_end:1551421398:after_script
[0Ksection_start:1551421398:upload_artifacts_on_failure
[0Ksection_end:1551421398:upload_artifacts_on_failure
[0K[31;1mERROR: Job failed: exit status 1
[0;m`
I disabled shared runner for this project.but still job fails with docker error.
because project has gitlab-ci.yml instead of .gitlab-ci.yml ,gitlab-runner skip my config file and use default config that run docker command while I set executor to shell.the file name is very important and because my file name dosen't start with "." ,gitlab-runner ignore my config.