I'm at a loss with this one. What is the most direct way I can get the outcome of a test run associated with a test case (work item)?
Here is the API call for Get Results:
GET https://dev.azure.com/{organization}/{project}/_apis/test/Runs/{runId}/results/{testCaseResultId}?api-version=5.0
I would need the runId and the testCaseResultId.
I can get a list of test runs:
GET https://dev.azure.com/{organization}/{project}/_apis/test/runs?api-version=5.0
but this doesn't seem to have any association with test cases.
There isn't much to do with the Test Cases section of the API.
I must be missing something, it shouldn't be this hard to get a result of a test case?
Related
Using serenity-js, with cucumber and screenplay, i can't get to pass data from one step to another. E.g. : a "When" step defined by a pretty user.attempsTo() that simply calls a webservice, and a "Then" step that should simply checks the response of the webservice. "TakeNote" forgets the data of the previous step ( as i could see, even surprising) and "LastResponse" context might be lost also...
I guess there is a simple way to store an object (as a Question maybe) in one Activity of the "When" step, and get back this object in an Activity of the "Then" step.
Without Serenity, i would use a World object that is clearly a simple way to do the job, but i'm sure there a Serenity pretty way to do the trick.
Thank you.
Bruno
I tried to pass the object with LastResponse, with TakeNotes. But the object seems to be lost.
I found what was wrong (answering to myself).
user.attempsTo() is actually processed prior to the next step only if a resolution of the promise is asked, so if preceded by return, or await, or if it is declared as an implicit return using arrow function like this ( user :Actor) => attempsTo(...) ).
Without the promise resolution, interactions are not executed in the described sequence, and this is really no good.
The symptom for a bad code is that the interactions are not logged in their steps, but in step further. Hope this helps.
I have a test case in SoapUI NG Pro which has the following steps:
POST REST Request that starts a process
JDBC Request where I check that the process Start Date has been logged to a database table
Delay (to simulate the time it takes for the process to run)
JDBC Request where I check that the End Date and Duration have been logged to the table
I would like to capture the timestamp of the POST Request to use within my assertions in steps 2 and 4.
I have looked around online and some people have mentioned using Events while others have mentioned using a Script TestStep but I haven't been able to get either to work.
I can get the POST Response timestamp but am looking for the Request timestamp in particular. I also noticed that there is a timestamp in the Request Log but again I don't know how to access that.
Any help would be greatly appreciated. Its probably also worth mentioning that I am using JavaScript instead of Groovy.
You can add a Script Assertion for the Soap Request test step and add the below statement in order to show the time taken.
log.info messageExchange.response.timeTaken
If you want the above value to be accessible in other steps, then use below(which stores the value to test case level, so that it is easy to access the test case property in other steps of the same test case):
context.testCase.setPropertyValue('TIME_TAKEN', messageExchange.response.timeTaken.toString())
In the later steps, use Property Expansion to read the test case level property value
def timeTaken = context.expand('${#TestCase#TIME_TAKEN}') as Integer
I have a bit of structural dilemma in soap. When running tests, it can be possible to run tests at project, test suite or test case level.
Now currently what happens is that we can run a whole project via project level and it will display a prompt box to select an endpoint (through a project level setup script and produces a project report using the project level tear down script).
However, it may be possible that the tester may not want to run a whole project and only wants to run a test suite or even a test case. Now it may be possible that the tester may only want to run only a test suite or even only a test case. Now it would be a hassle disabling suites or cases you don't want to run.
Now the problem i have is that if I start putting prompt boxes to select endpoints at suite or case level, everytime we hit a suite or case, it will always ask for an endpoint. Another thing is that I am thinking not creating suite or test case reposts because if running many suites or cases one by one, it is just an overkill on reporting.
I like your thinking on this, but I was speaking with my professional colleague and what we're thinking is this:
Add the below code for all test suites and test case level in their relevant setup scripts where it asks for endpoint (this is same code used in project set up script for selecting endpoint):
import com.eviware.soapui.support.*
def alert = com.eviware.soapui.support.UISupport
def urls = []
project.properties.each
{
if (it.value.name.startsWith("BASE_URL_"))
{
urls.push(it.value.name.replace("BASE_URL_", ""))
}
}
def urlName = alert.prompt("Please select the environment URL", "Enter URL", urls)
if (urlName)
{
def url = project.getPropertyValue("BASE_URL_" + urlName)
def urlBase = "BASE_URL_" + urlName
project.setPropertyValue("BASE_URL", url)
switch (urlBase){
case "BASE_URL_TEST":
project.setPropertyValue("DOMAIN_NAME", "TEST");
break;
case "BASE_URL_STAGE":
project.setPropertyValue("DOMAIN_NAME", "STAGE");
break;
default:
project.setPropertyValue("DOMAIN_NAME", "NO DOMAIN");
break;
}
}
else
{
log.warn 'haven\'t received user input'
log.warn 'No base URL is selected or cancelled, try again'
assert false
}
Now what we add is the following and we may need to use properties but again see what you think is best:
If test is ran at project level, it will prompt to select endpoint through project setup script but it will not ask for selecting endpoint through test suite or test case setup script. So it's only a single endpoint selection
If test is ran at suite level, it will prompt to select endpoint through project setup script but it will not ask for selecting endpoint through test case setup script. So it's only a single endpoint selection
For running at test case level, well it only runs for that test case so it's at the lowest level as it asks for an endpoint for that test case.
We can't have setup scripts disabled at any level because there maybe over code in those setup script that will need to be exectued, we just need a way to say depending on which level, don't ask for selecting endpoints at lower levels.
Seems complicated to implement but does anyone know best way to implement this or do they even have a better idea than this theory?
Thanks
For a moment, let us assume you get it done for all levels (project, suite, and each case). May be you forgot about the step level ;-)
Do you have any Pros in your approach?, for me, NO.
Cons in your approach:
Each time user executes a test (be it project / suite / any test case), engineer needs to select value from the drop down, which is unwanted though testing against the same server as previous test case & little annoying.
Test execution requires manual intervention each time test execution is invoked.
User Interface is required as drop down being used.
Will be come road block / hurdle for end to end automation or to achieve automation.
Test execution can't done in headless mode. And this is important if you need to use Continuous Integration tools.
Proposed Approach :-
If I have to do the above, I would do the following. That would be clean, damn simple, no such complications would arise that you had mentioned in the long summary.
Looks there are following project properties defined with addresses of the test servers:
BASE_URL_TEST
BASE_URL_STAGE
There is also another project property defined BASE_URL and all the above logic is to allow the user to select the value from above properties to base URL value.
Now all user have to do is change the value for project property BASE_URL. I would think just user have to set one of the below value by hand what he / she needed as (one of them) before proceeding with their tests.
${#Project#BASE_URL_TEST} or
${#Project#BASE_URL_STAGE}
NOTE that a property value can be referred into another property by the use of Property Expansion like above.
With the above, user can set whatever is needed and change only if required or have to change the test server.
No setup script at any level is required any more, and just simply change the value of the property.
Properties are given to make to life simple, which can be used in N number of places and maintain the project easily.
Most Importantly, overcome the Cons mentioned in the beginning.
It is general practice that SoapUI is used to design the tests, and SOAPUI_HOME/bin/testrunner.bat or .sh utility to execute the tests in command line mode and that is the way to achieve Continuous Integration.
That's why use of properties helps here to achieve the above without any issues.
Even simple:
Just have one project property BASE_URL (remove others), user have to just edit the property value and have the test server name / IP address and is done for once, say http://testjuniper. Isn't it dead simple?
And I believe, the engineer would definitely know which server he / she is using to execute the tests.
Having said that, now user do not have to bother at all, irrespective of executing a project / suite / test case, as long as testing is carried out against the same server / environment.
Once, the test execution is finished against TEST environment, the engineer may move on to other environment say STAGING, just change BASE_URL property value accordingly.
I am getting this ExpectationNotMetError in my cucumber/watir test,
How to tell the difference?
RSpec::Expectations::ExpectationNotMetError:
expected: "Limits as usual \u0393\u00C7\u00F6 Limits in perspective /\u256B\u00C6\u256B\u00E6\u256B\u00F2\u256B\u00A3\u256B\u00F2\u256B\u00AC\u256B\u00D6\u256B\u00F6 \u256B\u2310\u256B\u00A3 \u256B\u00C9\u256B\u00D6\u256B\u00A3\u256B\u00D6\u256B\u00F6 \u256B\u00BA\u256B\u00F1\u256B\u00D6\u256B\u00FF\u256B\u00F2\u256B\u00A3\u256B\u00D6\u256B\u00E1\u256B\u00F6 \u0393\u00C7\u00F6 \u256B\u20A7\u256B\u00E6\u256B\u00FF \u256B\u20A7\u256B\u00E6\u256B\u00AC\u256B\u00D6 \u256B\u00F6\u256B\u00BA\u256B\u00E6\u256B\u00BF\u256B\u00F2\u256B\u00AC"
got: "Limits as usual \u2014 Limits in perspective /\u05D2\u05D1\u05D5\u05DC\u05D5\u05EA\u05D9\u05D4 \u05E9\u05DC \u05D0\u05D9\u05DC\u05D9\u05D4 \u05E7\u05E4\u05D9\u05D8\u05D5\u05DC\u05D9\u05E0\u05D4 \u2014 \u05DE\u05D1\u05D8 \u05DE\u05D1\u05EA\u05D9 \u05D4\u05E7\u05D1\u05E8\u05D5\u05EA"
The string you are testing contains "\u 2014" but your test expects the string not to contain that. As a result, the test is failing on the expectation.
To simplify:
expected: "............" got "......2014......"
I would like to retrieve the scenario state in the "After" scenario hook. I noticed that the .failed? method does not consider pending steps as failed steps.
So How can I determine that a scenario did not execute completely, because it failed OR because some steps were not implemented/defined.
You can use status method. The default value of status is :skipped, the failed one is :failed and the passed step is :passed. So you can write something like this:
do sth if step.status != :passed
Also, if you use !step.passed? it does the same thing because it only checks for the :passed status.
http://cukes.info/api/cucumber/ruby/yardoc/Cucumber/Ast/Scenario.html#failed%3F-instance_method
On that subject, you can also take a look at this post about demoing your feature specs to your customers: http://multifaceted.io/2013/demo-feature-tests/
LiohAu, you can use the 'status' method on a scenario itself rather than on individual steps. Try this: In hooks, add
After do |scenario|
p scenario.status
end
This will give the statuses as follows:
Any step not implemented / defined, it'll give you :undefined
Scenario fails (when all steps are defined) :failed
Scenario passes :passed
Using the same hook, it'll give you the status for scenario outline, but for each example row (since for each example row, it is an individual scenario). So if at all you want the result of an entire outline, you'll need to capture result for all example rows and compute the final result accordingly.
Hope this helps.