Using automation testing running Cucumber through Jenkins, I am able to get the results for execution, however I would need to accumulate the total results, any idea or tool that could allow me to aggregate the results by day, week, month,...
Related
Current Scenario: We have 5 jobs in RT executions and each job is running on 5 different agents. So we have 5 cucumber reports.
Expected Scenario: We need a consolidated Cucumber HTML Report of all 5 jobs.
How it can be resolved? We are working in azure Devops.
The Cucumber Html report is a single page html application. If you open it up in an editor you'll see that it contains a an array of messages.
These messages can also be obtained by using the messages:target/messages.ndjson plugin. If you can obtain in a single job the messages from each parallel job you can then merge these and pass them to the html-formatter.
To merge the files you may have to filter out some messages. Definitively requires writing some code, don't think anyone has done this before, but shouldn't be impossible.
I am integrating Asana project metrics with our help desk dashboard. I would like to show 3 numbers for each project:
- Total tasks in project
- Total completed tasks in project
- Total incomplete tasks in project
When I call the project/tasks api, I want to simply get a count, and not have to retrieve all the pages and programatically count the tasks. Is there any parameter for the API calls which just gets me a count of how many tasks match the criteria?
Thanks,
Craig
Unfortunately, the Asana API doesn't currently have the type of filtering where you can query to a subset of tasks that match an arbitrary pattern that you specify (i.e. "only the tasks where completed=true"). We also don't have an easy way to only get the completed tasks. You can get all incomplete tasks fairly easily by specifying completed_since=now on the tasks query endpoint - which is admittedly a bit strange, but works - but its converse (get only completed tasks) doesn't.
We are evaluating use cases for more filtering options, so you might see it at some point! For now, however, the only way to go about this is to get all of the tasks for a project and count them on your side.
I have about 6000 specflow [version 1.9.0.77] tests and those tests are split across 10 categories [tags], roughly 600 test cases per categories and takes about an hour to complete. Currently I’m using Nunit 2.6.4 to execute the tests [executing sequentially] and generating the Specflow flow report out of Nunit test report xml.
I’m planning to move the sequential execution model to parallel execution to reduce the test execution time. There are no static references, no feature or scenario context and test data are unique to test case.
I explored Nunit 3.5 with Specflow 2.0 but couldn’t find a solution to run the tests parallel through categories or by tags. Every time it runs sequentially.
I followed the page http://www.specflow.org/documentation/Parallel-Execution/ to setup parallel execution but didn’t worked for me.
Any thoughts?
Two things comes to mind that might go wrong that you don't really mention:
To run in parallel, SpecFlow runs Features in parallel. So it doesn't matter how many tags (categories) you use if the are all in the same feature file.
Another error source is that to run parallel tests you need at least two processors on the machine running the tests. And if you have a lot of things running, consuming processor power, the number of available processors will decrease to 1 which equal sequential execution of tests.
I have a JMeter test that has two thread groups. The first thread group goes out and gets auth and audit tokens. The second requires the tokens to test the APIs on which I'm interested in gathering performance data. I have Listeners set up as children of the samplers in the second thread group only. Running JMeter I get the results I want. But when I execute the same test from Jenkins, I get results from the both of the thread groups. I don't want the results from the first thread group. They clutter up my graphs and since there is only one execution of each they fluctuate, performance wise, enough to trigger my unstable/failed percentages routinely. Is there a way to get Jenkins to report on only the listeners/samplers I want? Do I have to run one test to get the tokens and another to test? If so, how do I pass the tokens from one test to the other?
You can execute 2 jenkins jobs:
First job write to file the tokens using BeanShell/JSR223 PostProcessor
Second job read the tokens from file using CSV Data Set Config
I am trying to do functional testing with JMeter for a Web service. I am trying to build a tractability metrics using JMeter itself for which I need the count of the samplers in a thread, no of passed samplers, failed samplers in a thread.
Is there any way to collect no of the samplers, failed or passed samplers in a thread?
Any help is much appreciated.
Thanks
By adding Aggregate Report listener to your test, you will be able to get the number of passed and failed requests details. Below is the purpose of aggregate report.
The aggregate report creates a table row for each differently named
request in your test. For each request, it totals the response
information and provides request count, min, max, average, error rate,
approximate throughput (request/second) and Kilobytes per second
throughput. Once the test is done, the throughput is the actual
through for the duration of the entire test.
Also, you can leverage View Results Tree, View Results in Table and Debug Sampler to debug your samplers.
To enable functional test mode in JMeter, click on the Test Plan and then check Functional Test Mode check box as shown below.