Show progress in a azure-pipeline output - azure

so I have my computer set up as an agent pool in azure-devops. I'm creating a test for latency so the developers can use it in their CI, the script runs in python and test various points in a system I have set up for the company which is connected to the cloud, it's mainly for informative purposes. When I run the script I have to wait some time, so the system I have connected goes through its normal network cycle inspecting all the devices in the local network, not very important for que question, however when I'm waiting I show in the terminal a message with "..." going from "." to ".." to "...", just to show the script didn't crash or anything.
the python code looks like this and works just fine when I run it locally:
sys.stdout.write("\rprocessing queue, timing varies depending on priority" + ("."*( i % 3 + 1))+ "\r")
sys.stdout.flush()
however the output shown in the azure pipeline shows all of the lines without replacing them. Is there a way to do what I want?

I am afraid showing progress is not supported in azure pipeline. Azure pipeline log console isnot user interactive. It just capture the agent machine terminal outputs.
You might have to use a simpler way to indicate that the script is now executing and not finished yet. For simple example:
sys.stdout.write("Waiting for processing queue ..." )
You can report this problem to microsoft development team. Hope they find a way to fix this in the future sprint.

I have seen it once but never actually used it myself, this can be done in both bash and PowerShell, not sure if this works inside a Python script, you might have to call bash/PowerShell from within your Python script.
It is possible to set a progress value in percent that is visible outside of the log, but as I understand it this value is step-spefific, meaning it only applies to the pipeline step you're currently in. You could drag the numeric value (however many percent) along into the next step, but the progress counter would then again show up in the next step. I believe it is not possible to have a pipeline global display of a progress.
If you export a progress value it will show up beside the step name in the left hand side step list.
This setting of a progress (also exporting one variable from one step to another, which is typically done that way) can be done by echoing special logging commands. There's a great description to be found here: Logging commands
What you want to do is something just as it is shown as an example on the linked page:
echo "Begin a lengthy process..."
for i in {0..100..10}
do
sleep 1
echo "##vso[task.setprogress value=$i;]Sample Progress Indicator"
done
echo "Lengthy process is complete."
All of these special logging commands start with ##vso[task... The VSO is a relict to the time when Azure DevOps was called Visual Studio Online.
There are a whole bunch of them, but most of the time what you really need is exporting variables from one build step context to another, which is done with ##vso[task.setvariable]value

Related

Declarative Pipeline using env var as choice parameter value

Disclaimer: I can achieve the behavior I’m looking for with Active Choices plugin, BUT I really want this to work in a Jenkinsfile and controlled with scm because it’s tedious to configure the Active Choices on each job we may need them on. And with it being separate from the Jenkinsfile creation, it’s then one job defined in multiple places. :(
I am looking to verify if this is possible, because I can’t get the syntax right, if it is possible. And I haven’t been able to find any examples online:
pipeline {
environment {
ARTIFACTS = lib.myfunc() // this works well
}
parameters {
choice(name: "Artifacts", choices: ARTIFACTS) // I can’t get this to work
}
}
I cannot use the function inline in the declaration of the parameter. The errors were clear about that, but it seems as though I should be able to do what I’ve written out above.
I am not home, so I do not have the exceptions handy, but I will add them soon. They did not seem very helpful while I was working on this yesterday.
What have I tried?
I’ve tried having the the function return a List Because it requires a list according to the docs, and I’ve also tried (illogically) returning a String in the precise syntax of a list of strings. (It was hacky, like return "['" + artifacts.join("', '") + "']" to look like ['artifact1.zip', 'artifact2.zip']
I also tried things like "$ARTIFACTS" and ${ARTIFACTS} in desperation.
the list of choices has to be supplied as String containing new line characters (\n): choices: 'TESTING\nSTAGING\nPRODUCTION'
I was tipped off by this article:
https://st-g.de/2016/12/parametrized-jenkins-pipelines
Related to a bug:
https://issues.jenkins.io/plugins/servlet/mobile#issue/JENKINS-40358
:shrug:
First, we need to understand that Jenkins starts running your pipeline code by presenting you with Parameters page. Once you've set up the parameters, and pressed Build, then a node is allocated, variables are set, and your code starts to run.
But in your pipeline, as presented above, you want to run some code to prepare the parameters.
This is not how Jenkins usually works. It's definitely not doing the following: allocating a node, setting the variables, running some of your code until parameters clause is reached, stopping all that, presenting you with GUI, and then continuing where it left off. Again, it's not how Jenkins works.
This is why, when writing a new pipeline, your first option to build it is Build and not Build with Parameters. Jenkins hasn't run your code yet; it doesn't have any idea if there are any parameters. When running for the first time, it will remember the parameters (and any choices, if were) as were configured for this (first) run, so in the second run you will see the parameters as configured in the first run. (Generally, in run number n you will see the result of configuration in run number n-1.)
There are a number of ways to overcome this.
If having a "somewhat recent" (and not "current and absolutely up-to-date") situation fits you, your code may need minor changes to work — second time. (I don't know what exactly lib.myfunc() returns but if it's a choice of Development/Staging/Production this might be good enough.)
If having a "somewhat recent" situation is an absolute no-no (e.g. your lib.myfunc() returns the list of git branches, and "list of branches as of yesterday" is unacceptable), then your only solution is ActiveChoice. ActiveChoice allows you to run some code before showing you the Build with Parameters GUI (with script approval etc.).

linux wait command only proceed if previous job successful putty

Im an end user that has access to Putty in order to run selective scripts on our server as they would run during our overnight batch process.
Some of these I run in sequence using
run_process.ksh kj_job1 & wait; run_process.ksh kj_job2
However kj_job1 can fail and kj_job2 would still run. Id like a way for kj_job2 to only proceed if kj_job1 was completed succesfully but i cant find a guide online to walk me through what i need to do.
My knowledge in this area is limited, i simply navigate to the directory where we have the file run_process.ksh and then add the job name i want to run. I recently found out about the & wait command in order to run strings and the () s i can run things in parallel.
Any help is apprecaited.

Is it possible to incorporate an environment variable into a ruby script for Calabash?

I am testing a feature on an app that requires the user to be a certain age. The only time you see the prompt that asks for your age is once you open the app for the first time and once you log out of the app. I don't want my test to only go through my steps to log in and then log out to be able to see this prompt, but I also don't want to manually reset the data in between tests either. Isn't this why we write scripts? Anyways, before I launch the test, I use the environment variable RESET_BETWEEN_SCENARIOS=1 cucumber features/my_feature.feature. Is there a way that I can use this variable INSIDE of my step definition so that it resets the data on its own once I run the script?
I'm not familiar with Calabash, but it appears to be using cucumber. If that is the case, you could handle the action in a before or after hook which would run before or after each scenario.
Within the features/support folder, add a file hooks.rb
Before() do
if ENV['RESET_BETWEEN_SCENARIOS'] == '1'
#code to reset data
end
end
This could also be run after the scenario by using After() do. The same if/then could be used within a scenario step as well.

Auto-correlation callback function issue - loadrunner

I'm working in new application written in Siebel 8.1, issue appears when I'm trying to replay script and I can't handle that.
Replay Output:
Error -27086: Auto-correlation callback function
"flCorrelationCallbackParseWebPage" failed (rc=1) for parameter
"Siebel_Parse_Web_Page40"
web_reg_save_param("Siebel_Parse_Web_Page40",
"LB/IC=",
"RB/IC=",
"Ord=1",
"Search=Body",
"RelFrameId=1",
"AutoCorrelationFunction=flCorrelationCallbackParseWebPage",
"AutoCorrelationDll=LrwiSiebelCorrelationWrapper",
LAST);
I have done all steps for prepare record options from: http://software-qe.blogspot.se/2008/01/siebel-7x-record-and-replay-for.html
I'm using Loadrunner 11.52 (Siebel Web protocol), IE8.
We've been using the autocorrelation library for quite a few years on my team and we see this a lot. Unfortunately, it's not an easy problem to diagnose.
First I would check your test results and your VUser log to see if something happened before the autocorrelation failed. (Make sure your logging is set to parameter substitution in runtime settings).
Check your parameter files for extra spaces, commas, etc. Sometimes I've seen that error right after it rejects something about your parameter file.
Worst case scenario, your script is corrupted and you'll have to start over. We've gotten in the habit of making frequent backups of our scripts just because of this issue. Usually, we'll be able to start from our backup and continue or create a new script and paste the old code in. Autocorrelation error "magically" goes away with the same code in a new script.
If auto(magical)correlation does not work then use manual correlation.
Record twice with same data: Compare. You will find session, state and time data.
Change the credentials: Re-record. Compare. You will find credential related correlation
Change the business record but keep the same business process. Re-Record. You will find the business related correlation.
Do not expect autocorrelation to provide a magical working script. You have about a 0.0001% chance of that happening without LoadRunner script development intervenetion.

Powershell: commandline applications not working after calling method from module

I have created a powershell module (.psm1) file that includes a few other powershell scripts. We use it for sharepoint.
So basically, here's what happens:
I have a deploy script that retrieves the module location from the registry
It loads the module using the Import-Module cmdlet (using -force switch)
This module in turn loads the Sharepoint 2010 snap in and a few other scripts that I created
It runs runs a deployment script that references functions from the included scripts
It also runs a command line application and sends the output directly to the screen
The script will usually work the first time. However, after a few number of tries the commandline tool will stop working and sending output to the screen altogether. And if I try to run a commandline tool (not a cmd-let) after running my script, it don't worky anymore: no output, nothing is done. Its just the same as hitting enter on a blank prompt. anything powershell specific or running GUI applications will work fine but running any console application will not produce any concievable results. the only solution to this, is to just close my powershell and open it again. it will work for usually once and I will have to close it again. our users certainly wont be happy about that..
The most 'notable' things on the script:
scriptblocks are used extensively (for logging), a script block is sent to a handler that executes it using invokecommand and logs the step
its manipulating sharepoint objects
all objects are properly disposed of
no static variables are created nor changed
There are a few global variables shared across all scripts
What I have tried:
I striped my code to a bare minimum: loading an xml file, and restaring a few windows services but I'm still getting this intermittently. I have no idea which part of the code could cause this. I would love to post the code, but our company policy forbids me to. so my aplogies..
Update as per the comment below:
here's roughly how I use codeblocks. I have this function below that is used everytime I want to make the user aware of a task that I'm executing and what it outcome is.
function DoTask($someString, $scriptBlock, $param)
{
try
{
OutputTaskDescription $someString
InvokeCommand $scriptBlock -ArgumentList $param
OutputResultOK
}
catch
{
OutputResultError $_.tostring()
}
}
it could then be used like this:
$stringVar = "something"
$SpSite = new-spsite
deploySomething 'Deploying something' -param $spsite -ScriptBlock {
dosomethingToObject $stringvar
dosomethingToObject $spSite.Name
}
it would then output something like:
Deploying Something ------------- OK
Deploying Something ------------- ERROR
Also notice that I pass the $spsite in the argument list and I just use the string directly. I still don't understand how this works but it seems like I can access all primitive typed variables even without passing them as arguments but I have to pass more complex objects are params, else they dont have any value.
Update:
after much searching and days of pain. I have found others with the same pain. My code exhibits the same exact symptoms as described here:
http://connect.microsoft.com/PowerShell/feedback/details/496326/stability-problem-any-application-run-fails-with-lastexitcode-1073741502
I guess there is no solution yet to this problem.
After a little while I've noticed that if I've ran some very memory intensive functions, I too have gotten that behavior where everything you try to execute just goes to the prompt again. I'd recommend setting Set-PsDebug -Trace 2 to see what those functions are actually doing. I fixed my issue by doing this and figuring out how to make my functions more efficient.

Resources