Passing arguments to python script in Azure Devops - python-3.x

I am trying to pass a system variable to the python script from azure devops. This is what I currently have in Yaml file:
- script: pytest test/test_pipeline.py
--$(global_variable)
--junitxml=$(Build.StagingDirectory)/test_pipeline-results.xml
displayName: 'Testing Pipeline'
condition: always()
The variable I need in my script is $(global_variable). The variable contains a value of $(Build.SourcesDirectory). It is the global variable. I am getting an error message as "unrecognised arguments" when I run the job.
Any help to tackle this will be helpful.
Thanks!
EDIT:
Complete log:
`##[section]Starting: Testing Pipeline
==============================================================================
Task : Command line
Description : Run a command line script using Bash on Linux and macOS and cmd.exe on Windows
Version : 2.151.2
Author : Microsoft Corporation
Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
==============================================================================
Generating script.
Script contents:
pytest test/test_pipeline.py --my_param="/home/vsts/work/1/s" --junitxml=/home/vsts/work/1/a/test_pipeline-results.xml
========================== Starting Command Output ===========================
[command]/bin/bash --noprofile --norc /home/vsts/work/_temp/64fb4a65-90de-42d5-bfb3-58cc8aa174e3.sh
ERROR: usage: pytest [options] [file_or_dir] [file_or_dir] [...]
pytest: error: unrecognized arguments: --my_param=/home/vsts/work/1/s
inifile: None
rootdir: /home/vsts/work/1/s
##[error]Bash exited with code '4'.
##[section]Finishing: Testing Pipeline`

I tried to write a simple sample for you refer.
In your .py file, please use add_argument to read in the command line parameter. In this issue, this command line parameter comes from your task specified.
A.py file:
import argparse
def parse_argument():
parse = argparse.ArgumentParser(description="ForTest!")
parse.add_argument("-test_data")
parse.add_argument("-build_dir")
And then, in PythonScript#0 task:
scriptPath: ‘A.py’
argument: -test_data $(myVariable1) -build_dir $(myVariable2)
myVariable1 and myVariable2 are all my customized variable. You can all use environment variable.
Since in python script, add_argument is used to read in the parameter which from command line. So it can receive the value from Argument specified.
In addition, for the issue which in your question, I think you’d better delete the content from your script: --my_param="/home/vsts/work/1/s" and try again. Because --my_param could not the valid argument for pytest.
Hope can help you.

Related

Unable to trigger shell script from azure repo through pipeline

I have deploy.sh file in my azure repository and i need to execute this deploy.sh file from azure pipeline.
Here is the steps that i defined in pipeline. I tried two ways to do it through cmd and bash, both are not picking the right location of deploy.sh file
Here is the error i am getting:
This is the error from CMD, but CMD shows green even though path is not correct
This is the error for bash
Question
How to correct the path and do successful execution of deploy.sh?
You should never hard code the path of the pipeline run. Instead you should use the predefined variables that will automatically pick the build ID and also path. For your case you will need the
$(Pipeline.Workspace)/s/Orchestration/dev/deploy.sh
If you have multiple repositories checkout you should also use the name of the repository like
$(Pipeline.Workspace)/s/dotnetpipeline/Orchestration/dev/deploy.sh
$(Pipeline.Workspace)/s should be also replaced by $(Build.SourcesDirectory)
The predefined variables should follow the $(var) notation on the .YML file
This solved my issue
- task: CmdLine#2
inputs:
script: |
echo Write your commands here
cd $(Build.Repository.Name)/Orchestration/dev/
chmod +x deploy.sh
./deploy.sh
echo deploy.sh execution completed

Gitlab job failure: Cannot overwrite variable Host because it is read-only or constant

Hi people of the internet.
Basically I am unable to run even the simplest job and I keep getting the same error no matter what I put in the .gitlab-ci.yml file. See example below:
Here is the .gitlab-ci.yml file:
stages:
- test
job1:
stage: test
tags:
- testing
script:
- echo "Hello world!"
Here is the output ("?" corresponds to intentionally blacked out information):
Running with gitlab-runner 14.10.0 (c6bb62f6)
on runner_test ????????
Preparing the "shell" executor
00:00
Using Shell executor...
Preparing environment
00:00
Running on LAPTOP-????????...
Getting source from Git repository
00:01
WriteError:
Line |
219 | $HOST="[MASKED]"
| ~~~~~~~~~~~~~~~~~~~~~~
| Cannot overwrite variable Host because it is read-only or constant.
ERROR: Job failed: exit status 1
I know that $HOST is a reserved variable in powershell but I don't see the link between the error and the code. It may have something to do with the configuration of the runner on Windows. Has anyone encountered this error on Gitlab before? Or any suggestions on how to debug?
Here are the steps that I took to install the runner on Gitlab for Windows (see https://docs.gitlab.com/runner/install/windows.html):
Create a folder somewhere in the system: C:\GitLab-Runner.
Download the binary for 64-bit and put it into the folder (see https://gitlab-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-runner-windows-amd64.exe).
Run prompt as an administrator
Run the following command:
cd C:\GitLab-Runner
gitlab-runner.exe register
Enter your GitLab instance URL (see Gitlab > Settings > CI/CD > Runners > Specific runners)
Enter the token to register the runner (see Gitlab > Settings > CI/CD > Runners > Specific runners)
Enter a description for the runner: runner_test for instance
Enter the tags associated with the runner, separated by commas: testing, windows for instance
Provide the runner executor: shell
Install GitLab Runner as a service and start it
cd C:\GitLab-Runner
gitlab-runner.exe install
gitlab-runner.exe start
I also had to install the latest version of pwsh in Windows (see gitlab-runner: prepare environment failed to start process pwsh in windows):
Run prompt as an administrator
Install the newer pwsh.exe:
winget install Microsoft.PowerShell
Restart the runner
cd C:\GitLab-Runner
gitlab-runner.exe restart
This issue was due to my choice of shell for some reason. A Gitlab runner can choose a shell among the following: bash, sh, powershell, pwsh, and cmd (the last one being deprecated now).
As I stated above I had been using pwsh. So, I went after the config.toml file inside of the C:\GitLab-Runner directory to manually make the change from pwsh to powershell.
...
[[runners]]
name = "runner_test"
executor = "shell"
shell = "powershell"
...
I then restarted the runner and got the job to complete properly:
cd C:\GitLab-Runner
gitlab-runner restart
I still get the error (more like a warning now) but it does not prevent the job from finishing anymore. If anyone has a better answer with a proper explanation I would gladly accept it as the answer to this question.
Note that pwsh to powershell are both powershell scripts (see https://docs.gitlab.com/runner/shells/index.html):
powershell Fully Supported PowerShell script. All commands are executed in PowerShell Desktop context. In GitLab Runner 12.0-13.12, this is the default when registering a new runner.
pwsh Fully Supported PowerShell script. All commands are executed in PowerShell Core context. In GitLab Runner 14.0 and later, this is the default when registering a new runner.

Error with Command Line Script in Azure DevOps Pipeline

I'm building my first pipeline using Power Platform Build Tools. I'm trying to export a Dynamics model driven app to a repo. I'm getting an error with in my Command Line Script. The following is the error log:
2021-01-21T08:48:04.6191345Z ##[section]Starting: Command Line Script
2021-01-21T08:48:04.6292483Z
==============================================================================
2021-01-21T08:48:04.6292831Z Task : Command line
2021-01-21T08:48:04.6293131Z Description : Run a command line script using Bash on Linux and macOS
and cmd.exe on Windows
2021-01-21T08:48:04.6293422Z Version : 2.178.0
2021-01-21T08:48:04.6293630Z Author : Microsoft Corporation
2021-01-21T08:48:04.6293952Z Help :
https://learn.microsoft.com/azure/devops/pipelines/tasks/utility/command-line
2021-01-21T08:48:04.6294293Z
==============================================================================
2021-01-21T08:48:05.7216764Z error: pathspec 'master' did not match any file(s) known to git
2021-01-21T08:48:05.7217182Z Generating script.
2021-01-21T08:48:05.7217463Z ========================== Starting Command Output
===========================
2021-01-21T08:48:05.7217952Z ##[command]"C:\windows\system32\cmd.exe" /D /E:ON /V:OFF /S /C "CALL
"D:\a\_temp\93c0ac5e-da28-4265-b4d0-4326b5f38209.cmd""
2021-01-21T08:48:05.7218457Z commit all changes
2021-01-21T08:48:05.7218642Z push code to new repo
2021-01-21T08:48:05.7226781Z fatal: pathspec '-' did not match any files
2021-01-21T08:48:05.7227220Z error: pathspec 'export"' did not match any file(s) known to git
2021-01-21T08:48:06.2395991Z git: 'bearer' is not a git command. See 'git --help'.
2021-01-21T08:48:06.2983259Z ##[error]Cmd.exe exited with code '1'.
2021-01-21T08:48:06.3323471Z ##[section]Finishing: Command Line Script
Based on this output, I don't know what is missing.
According to your screenshots, your default branch is "main", not "master".
The master did not match any branches known to git, so the task failed.
In addition, you need to use origin/{branch} instead of {branch}. As the branch is a remote branch.
Error with Command Line Script in Azure DevOps Pipeline
This may be because your git command has wrong syntax.
According to the image you provided, we could to know the git command line you used is:
This is not the correct syntax to add all files, that also the reason why you get the error:
fatal: pathspec '-' did not match any files
If I use the same syntax git add —— all, I will get the same error message.
To resolve this error, please try to use following correct syntax:
git add --all
Besides, we could not use the AUTHORIZATION:BEARER with $(system.AccessToken), you need to use Authorization: Basic with Base64-encode:
Please check this thread for some more details.
That the reason why you get the error git: 'bearer' is not a git command..
To resolve this issue, you could use git command line with PAT directly to push the files:
git push https://<Your PAT>#dev.azure.com/<YourOrganization>/<YourProject>/_git/MyTestProject HEAD:master
In addition, what Jane said is another issue.

Jenkins console prints only logger commands from main.py

I have a pipeline in Jenkins, which triggers a python file:
status = sh script: '''
python3 main.py --command check_artfiacts
''', returnStatus:true
as long I'm in the main.py, I'm getting the expected result from logger in the console:
2019-11-28 22:14:32,027 - __main__ - INFO - starting application from: C:\Tools\BB\jfrog_distribution_shared_lib\resources\com\amdocs\jfrog\methods, with args: C:\Tools\BB\jfrog_distribution_shared_lib\resources\com\amdocs\jfrog\methods
2019-11-28 22:14:32,036 - amd_distribution_check_artifacts_exists - INFO - Coming from func: build_aql_queries
however, when calling a function that exists on another python file, it doesn't work (it behaves like a normal print):
added helm to aql
amd_distribution_check_artifacts_exists: build_docker_aql_query_by_item
I know for sure it's some pipeline issue, coz when running the code from PyCharm, it prints everything as expected.
Did anyone face such an issue?
I found the solution in this thread:
jenkins-console-output-not-in-realtime
so -u worked for me:
python3 -u main.py --command check_artfiacts

Puppet task run throwing invalid json error

My Puppet sever is PE 2017.3.1
My Agent node is on 5.5 version
I am facing an error while executing the command
/opt/puppetlabs/puppet/bin/puppet-task run sample --nodes puppet-agent
My Sample file is a bash file which contains : -
#!/usr/bin/env bash
hostnamectl
I was able to list my task using cli
The above puppet-task command throws an error on command line: -
1. Starting Job
2. Invalid Json
Puppet access login token expired. So after recreating it, I was able to perform the task.

Resources