I am trying to run a powershell script which first logins to azure and then deploys the zip file to azure using psexec.
I am using the following command:
F:\jenkins\VMScripts\PsExec64.exe \\WINSU9 -u "WINSU9\administrator" -p mypassword /accepteula -h PowerShell -noninteractive -File C:\Shared\Trial\webappscript.ps1
I am getting the output as:
PsExec v2.2 - Execute processes remotely
Copyright (C) 2001-2016 Mark Russinovich
Sysinternals - www.sysinternals.com
[
{
"cloudName": "AzureCloud",
"id": "a7b6d14fddef2",
"isDefault": true,
"name": "subscription_name",
"state": "Enabled",
"tenantId": "b41cd",
"user": {
"name": "username#user.com",
"type": "user"
}
}
]
WARNING: Getting scm site credentials for zip deploymentConnecting to WINSU9...
Starting PSEXESVC service on WINSU9...
Connecting with PsExec service on WINSU9...
Starting PowerShell on WINSU9...
PowerShell exited on WINSU9 with error code 0.
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // node
[Pipeline] End of Pipeline
Finished: SUCCESS
It is just giving the output of az login command but the output of deployment is not showing. Also if the deployment fails, it will still show success. But it should show failure.
Answering my question so that others facing the same issue can get help here. As #Alex said that powershell is exiting with error code 0, I tried to return the error code 1 whenever any command fails. Since the output of Azure CLI is in json format, I stored that output in a variable and checked if it contains anything. The sample of the code is written below.
$output = az login -u "username" -p "password" | ConvertFrom-Json
if (!$output) {
Write-Error "Error validating the credentials"
exit 1
}
The Jenkins job succeeded because PSExec.exe returned exit code 0, which means that no errors were encountered. Jenkins jobs will fail if the underlying scripts fail (e.g. returning non-zero exit codes, like 1). If the PSExec.exe application isn't doing what you want it to - I would wrap it in another script which performs post-deploy validation, and returns 1 if the deployment failed.
See How/When does Execute Shell mark a build as failure in Jenkins? for more details.
You can use powershell step, this should hand out the error directly as well.
Related
I tried executing this command in the Terminal(and my environment is Ubuntu22.04) and it worked well with some Loading Effects, but when I configure this command in my docker container(created by my CI/CD pipeline executor) it does not work properly when the CI/CD pipeline runs. Below is my configuration and execution error:
- nsenter --mount=/host/proc/1/ns/mnt sh -c "cd /home/..USER../Desktop/..PROJECT..&&export https_proxy=..PROXY../&&gh pr create --title "test" --body "test" --base cicd_test"
Error in the CI/CD Console logs which runs properly in the local Terminal with some loading effects
hope run properly in the docker or some other alternative command that may work
I'm running a rather hefty build in my Azure pipeline, which involves processing a large amount of data, and hence requires too much memory for my buildagent to handle. My approach is therefore to start up an linux VM, run the build there, and push up the resulting docker image to my container registry.
To achieve this, I'm using the Azure CLI task to issue commands to the VM (e.g. az vm start, az vm run-command ... etc).
The problem I am facing is that az vm run-command "succeeds" even if the script that you run on the VM returns a nonzero status code. For example, this "bad" vm script:
az vm run-command invoke -g <group> -n <vmName> --command-id RunShellScript --scripts "cd /nonexistent/path"
returns the following response:
{
"value": [
{
"code": "ProvisioningState/succeeded",
"displayStatus": "Provisioning succeeded",
"level": "Info",
"message": "Enable succeeded: \n[stdout]\n\n[stderr]\n/var/lib/waagent/run-command/download/87/script.sh: 1: cd: can't cd to /nonexistent/path\n",
"time": null
}
]
}
So, the command succeeds, presumably because it succeeded in executing the script on the VM. The fact that the script actually failed on the VM is buried in the response "message"
I would like my Azure pipeline task to fail if the script on the VM returns a nonzero status code. How would I achieve that?
One idea would be to parse the response (somehow) and search the text under stderr - but that sounds like a real hassle, and I'm not sure even how to "access" the response within the task.
Have you enabled the option "Fail on Standard Error" on the Azure CLI task? If not, you can try to enable it and run the pipeline again to see if the error "cd: can't cd to /nonexistent/path" can make the task run failed.
If the task still is passed, the error "cd: can't cd to /nonexistent/path" should not be a Standard Error. In this situation, you may need to add more command lines in your script to monitor the output logs of the az command. Once there is any output message shows error, execute "exit 1" to exit the script and return a Standard Error to make the task be failed.
I solved this by using the SSH pipeline task - this allowed me to connect to the VM via SSH, and run the given script on the machine "directly" via SSH.
This means from the context of the task, you get the status code from the script itself running on the VM. You also see any console output inside the task logs, which was obscured when using az vm run-command.
Here's an example:
- task: SSH#0
displayName: My VM script
timeoutInMinutes: 10
inputs:
sshEndpoint: <sshConnectionName>
runOptions: inline
inline: |
echo "Write your script here"
Not that the SSH connection needs to be set up as a service connection using the Azure pipelines UI. You reference the name of the service connection you set up in yaml.
I am attempting to deprovision an Azure Linux instance using the Custom Script Extension.
My Script stored in an Anonymous Access Blob:
sudo waagent -deprovision+user -verbose -force
exit
My Command to apply the extension:
az vm extension set --resource-group rg01--vm-name vm01--name CustomScript --publisher Microsoft.Azure.Extensions --version 2.0 --settings "{'fileUris': [ 'https://mystorageaccount.blob.core.windows.net/scripts/Deprovision.sh' ], 'commandToExecute': 'sh Deprovision.sh'}"
When I run the az command, all the /var/log/azure sub directories and logs disappear!. I can tell from the bastion window, something tried to delete my user account, so I am confident the extension is getting provisioned and run.
Unfortunately, the extension, shows all status information as unavailable, and my az command just sits there. The "Create or Update Virtual Machine Extension" item in the VM's activity log also shows no activity once the deprovision starts. The Azure activity log suggest a restart occurred and my account is no longer valid.
Hopefully Linux/Azure folks have a recipe for this...
-
I saw similar behavior in Windows, and ended up using this script to Sysprep (the -Wait was critical, as it forced the Powershell process to wait for completion, preventing the Agent from returning success/fail until the process completed.), this prevents a restart. I can then script deallocation to occur when the extension completes. I suspect something similar is going on here.
Start-Process -FilePath C:\Windows\System32\Sysprep\Sysprep.exe -ArgumentList '/generalize /oobe /quiet /quit' -Wait
While the command:
sudo waagent -deprovision+user -verbose -force
works via SSH, when run via CustomScript extension this command basically kills everything on the machine. The CustomScript extension is not able to acknowledge completion.
Using this script:
sudo shutdown +3
sudo waagent -deprovision+user -verbose -force -start
Line 1, shuts the VM down in 3 minutes(deprovision command seems very fast)
Line 2, adding '-start', runs waagent as a background process. This allows CustomScript extension to acknowledge completion.
Now this command completes (instead of hangs):
var cmd = "sh Deprovision.sh";
var result = vm.Update()
.DefineNewExtension("deprovision")
.WithPublisher("Microsoft.Azure.Extensions")
.WithType("CustomScript")
.WithVersion("2.1")
.WithMinorVersionAutoUpgrade()
.WithPublicSetting("fileUris", new string[] { blobUri })
.WithPublicSetting("commandToExecute", cmd)
.Attach()
.Apply();
After completion, we must poll Azure for when the VM is Stopped.
WaitForVMToStop(vm);
private void WaitForVMToStop(IVirtualMachine newVM)
{
Context.Logger.LogInformation($"WaitForVMToStop...");
bool stopped = false;
int cnt = 0;
do
{
var stoppedVM = Context.AzureInstance.VirtualMachines.GetById(newVM.Id);
stopped = (stoppedVM.PowerState == PowerState.Stopped);
if (!stopped)
{
cnt++;
Context.Logger.LogInformation($"\tPolling 60 seconds for 'PowerState = Stopped on [{newVM.Name}]...");
System.Threading.Thread.Sleep(60000);
if (cnt > 20)
{
Context.Logger.LogInformation($"\tSysPrep Extension exceeded 20 minutes. Aborting...");
throw new Exception($"SysPrep Extension exceeded 20 minutes on [{newVM.Name}]");
}
}
} while (!stopped);
Context.Logger.LogInformation($"\tWaited {cnt} minutes for 'PowerState = Stopped...");
}
This seems way too complicated to me, but it works. I especially do not like assuming deprovision will occur in 3 minutes or less. If anybody has a better way, please share.
I'm very new to Azure IoT Edge and I'm trying to deploy to my Raspberry PI : Image Recognition with Azure IoT Edge and Cognitive Services
but after Build & Push IoT Edge Solution and Deploy it to Single Device ID I see none of those 2 modules listed in Docker PS -a & Iotedge list
And when try to check it on EdgeAgent Logs there's error message and it seems EdgeAgent get error while creating those Modules (camera-capture and image-classifier-service)
I've tried :
1. Re-build it from fresh folder package
2. Pull the image manually from Azure Portal and run the image manually by script
I'm stuck on this for days.
in deployment.arm32v7.json for those modules I define the Image with registered registry url :
"modules": {
"camera-capture": {
"version": "1.0",
"type": "docker",
"status": "running",
"restartPolicy": "always",
"settings": {
"image": "zzzz.azurecr.io/camera-capture-opencv:1.1.12-arm32v7",
"createOptions": "{\"Env\":[\"Video=0\",\"azureSpeechServicesKey=2f57f2d9f1074faaa0e9484e1f1c08c1\",\"AiEndpoint=http://image-classifier-service:80/image\"],\"HostConfig\":{\"PortBindings\":{\"5678/tcp\":[{\"HostPort\":\"5678\"}]},\"Devices\":[{\"PathOnHost\":\"/dev/video0\",\"PathInContainer\":\"/dev/video0\",\"CgroupPermissions\":\"mrw\"},{\"PathOnHost\":\"/dev/snd\",\"PathInContainer\":\"/dev/snd\",\"CgroupPermissions\":\"mrw\"}]}}"
}
},
"image-classifier-service": {
"version": "1.0",
"type": "docker",
"status": "running",
"restartPolicy": "always",
"settings": {
"image": "zzzz.azurecr.io/image-classifier-service:1.1.5-arm32v7",
"createOptions": "{\"HostConfig\":{\"Binds\":[\"/home/pi/images:/images\"],\"PortBindings\":{\"8000/tcp\":[{\"HostPort\":\"80\"}],\"5679/tcp\":[{\"HostPort\":\"5679\"}]}}}"
}
Error message from EdgeAgent Logs :
(Inner Exception #0) Microsoft.Azure.Devices.Edge.Agent.Edgelet.EdgeletCommunicationException- Message:Error calling Create module
image-classifier-service: Could not create module image-classifier-service
caused by: Could not pull image zzzzz.azurecr.io/image-classifier-service:1.1.5-arm32v7
caused by: Get https://zzzzz.azurecr.io/v2/image-classifier-service/manifests/1.1.5-arm32v7: unauthorized: authentication required
When trying to run the pulled image by script :
sudo docker run --rm --name testName -it zzzz.azurecr.io/camera-capture-opencv:1.1.12-arm32v7
None
I get this error :
Camera Capture Azure IoT Edge Module. Press Ctrl-C to exit.
Error: Time:Fri May 24 10:01:09 2019 File:/usr/sdk/src/c/iothub_client/src/iothub_client_core_ll.c Func:retrieve_edge_environment_variabes Line:191 Environment IOTEDGE_AUTHSCHEME not set
Error: Time:Fri May 24 10:01:09 2019 File:/usr/sdk/src/c/iothub_client/src/iothub_client_core_ll.c Func:IoTHubClientCore_LL_CreateFromEnvironment Line:1572 retrieve_edge_environment_variabes failed
Error: Time:Fri May 24 10:01:09 2019 File:/usr/sdk/src/c/iothub_client/src/iothub_client_core.c Func:create_iothub_instance Line:941 Failure creating iothub handle
Unexpected error IoTHubClient.create_from_environment, IoTHubClientResult.ERROR from IoTHub
When you pulled the image directly with docker run, it pulled but then failed to run outside of the edge runtime, which is expected. But when the edge agent tried to pull it, it failed because it was not authorized. No credentials were supplied to the runtime, so it attempted to access the registry anonymously.
Make sure that you add your container registry credentials to the deployment so that edge runtime can pull images. The deployment should contain something like the following in the runtime settings:
"MyRegistry" :{
"username": "<username>",
"password": "<password>",
"address": "<registry-name>.azurecr.io"
}
As #silent pointed out in the comments, the documentation is here, including an example deployment that includes container registry credentials.
I'm presently hosting an Azure Scale set running Windows Server 2012 R2 that is setup with the Chef extension (Chef.Bootstrap.WindowsAzure.ChefClient). When the VM is provisioned, the extension reports back that it succeeded via the Azure portal however the registered node on the Chef server is not updated to retain the provided run list and the first run isn't fully completed. This is causing subsequent chef-client runs to be performed with an empty run list. When I observe the reports on chef server, I see a run with a status of aborted with no error.
Upon review of the WindowsAzure Plugins chef-client.log file, I can see that it tries to execute the run list but seems to be interrupted with the following FATAL
FATAL: Errno::EINVAL: Invalid argument # io_writev - <STDOUT>
There is no chef-stacktrace.out file created as well. The ARM extension definition looks like:
{
"type": "extensions",
"name": "ChefClient",
"properties": {
"publisher": "Chef.Bootstrap.WindowsAzure",
"type": "ChefClient",
"typeHandlerVersion": "1210.12",
"autoUpgradeMinorVersion": true,
"settings": {
"client_rb": "ssl_verify_mode :verify_none\nnode_name ENV[\"COMPUTERNAME\"]",
"runlist": "recipe[example]",
"autoUpdateClient": "false",
"deleteChefConfig": "false",
"bootstrap_options": {
"chef_server_url": "https://mychefserver.com/organizations/myorg",
"validation_client_name": "myorg-validator",
"environment": "dev"
}
},
"protectedSettings": {
"validation_key": "-----BEGIN RSA PRIVATE KEY----- ... -----END RSA PRIVATE KEY----"
}
}
}
In order to troubleshoot, I've tried to reduce my example cookbook down to a single DSC script which installs IIS. Even this step, I've executed it multiple ways such as using windows_feature, powershell_script, and dsc_script. All result end up with the same error. Here is the current script
powershell_script 'Install IIS' do
code 'Add-WindowsFeature Web-Server'
guard_interpreter :powershell_script
not_if "(Get-WindowsFeature -Name Web-Server).Installed"
end
If I override the run list and call chef-client manually, everything succeeds. I'm having trouble honing in on whether this is the Azure Chef Extension, the Chef client, or the cookbook.
As far as I can tell, communication with the Chef server looks good as the necessary pem files are exchanged, chef-client is installed, and the cookbook is downloaded and cached from the server. The cache gets removed on the subsequent run however with the empty run list. Here are the contents of first-boot.json:
{"run_list":["recipe[example]"]}
Here are the versions in play:
chef-client version: 14.1.12
Azure Chef Extension version: 1210.12.110.1001
Server version: Windows Server 2012 R2
Any ideas what could be going on?
It turns out my analysis was incorrect about which resource was causing the problem. It appears that the first boot run was failing when using dsc_script as the resource to install the web server. When using the following powershell_script resource, it succeeded and the run list attached for future runs.
powershell_script 'Install IIS' do
code 'Add-WindowsFeature Web-Server'
guard_interpreter :powershell_script
not_if "(Get-WindowsFeature -Name Web-Server).Installed"
end