I have a JMeter test that has two thread groups. The first thread group goes out and gets auth and audit tokens. The second requires the tokens to test the APIs on which I'm interested in gathering performance data. I have Listeners set up as children of the samplers in the second thread group only. Running JMeter I get the results I want. But when I execute the same test from Jenkins, I get results from the both of the thread groups. I don't want the results from the first thread group. They clutter up my graphs and since there is only one execution of each they fluctuate, performance wise, enough to trigger my unstable/failed percentages routinely. Is there a way to get Jenkins to report on only the listeners/samplers I want? Do I have to run one test to get the tokens and another to test? If so, how do I pass the tokens from one test to the other?
You can execute 2 jenkins jobs:
First job write to file the tokens using BeanShell/JSR223 PostProcessor
Second job read the tokens from file using CSV Data Set Config
Related
I have a use case for testing authentication functionality where there are multiple test cases like login into an app, forgetting a password, log in to MFA-enabled applications. I have a set of multiple users that can be used in any of the test cases but then the issue arises when trying to run them in multiple browser contexts. I have stored my test data in a JSON file with the username and password of multiple sample users.
When let's say test runs for login to MFA enabled application, all three browser worker is launched simultaneously and all of them try to get user details from the test data file.
BUT the issue comes here, all of them picked up the first object let's say user A, all three browser test passes till the password step but when MFA is entered, it creates a race condition, the one worker who submitted the OTP first will pass but the rest fails because that 30 seconds window OTP is already redeemed.
I want to have a way that works in the same way as the synchronized method in Java where if a worker is using one user, don't make them available for another user, instead provide them next user from the test data.
Please guide me on how to do that in Playwright!
I'm sure there are far more elegant ways of doing this, but one approach is to use the worker index / parallel index feature described in the docs here:
https://playwright.dev/docs/test-parallel#worker-index-and-parallel-index
It looks like parallel index may be a better fit for your use case.
If each of your rows of test data includes the index of the worker it is intended for, then your code can ensure that worker 0 only picks up worker 0's test data and worker 1 picks up worker 1's test data.
Alternatively you could use testConfig.workers to determine how many workers there are and then use the remainer/modulo operator (%) in JS to split the rows up between the different workers:
https://playwright.dev/docs/api/class-testconfig#test-config-workers
So you would compare TestDataRowNumber % testConfig.workers to process.env.TEST_PARALLEL_INDEX to split the file up amongst however many workers you had.
Let's say I have 6233 simulations to run. The commands are generated and stored in a file, one in each line. I would like to use Slurm to schedule and run these commands. However, the MaxArraySize limit is 2000. So I can't use one job array to schedule all of them.
One solution is given here, where we create four separate jobs and use arithmetic indexing into the file, with the last job having a smaller number of tasks to run (233).
Is it possible to do this using one sbatch script with one job ID?
I set ntasks=1 when using job arrays. Do larger ntasks help in such situations?
Update:
Following Damien's solution and examples given here, I ended up with the following line in my bash script:
curID=$(( ${SLURM_ARRAY_TASK_ID} * ${SLURM_NTASKS} + ${SLURM_PROCID} ))
The same can be done using Python (shown in the referenced page). The only difference is that the environment variables should be imported into the script.
Is it possible to do this using one sbatch script with one job ID?
No. That solution will give you multiple job IDs
I set ntasks=1 when using job arrays. Do larger ntasks help in such situations?
Yes, that is a factor that you can leverage.
Each job in the array can spawn multiple tasks (--ntasks=...). In that case, the line number in the command file must be computed from $SLURM_ARRAY_TASK_ID and $SLURM_PROCID, and the program must be started with srun. Each task in a job member of the array will run in parallel. How large the job can be will depend on the MaxJobsize limit defined on the cluster/partition/qos you have access to.
Another option is to chain the tasks inside each job of the array, with a Bash loop (for i in $seq(...) ; do ...; done). In that case, the line number in the command file must be computed from $SLURM_ARRAY_TASK_ID and $i. Each task in a job member of the array will run serially. How large the job can be will depend on the MaxWall limit defined on the cluster/partition/qos you have access to.
If I go to Test results screen after run of my pipeline, it is showing each test case from Java/Maven/TestNG automated test project duplicated. One instance of each test case shows blank for machine name and the duplicate of that shows a machine name.
Run 1000122 - JUnit_TestResults_3662
There are several possibilities. First, if you added multiple configurations to a test plan, if so, the tests cases will be repeated in the plan with the each of the configurations you have assigned.
Another possibility is that when you passed parameters to the test method, did you use multiple parameters, so the test method was executed two times.
The information you provided is not sufficient. Can you share the code or screenshots of your Test Samples?
I am trying to do functional testing with JMeter for a Web service. I am trying to build a tractability metrics using JMeter itself for which I need the count of the samplers in a thread, no of passed samplers, failed samplers in a thread.
Is there any way to collect no of the samplers, failed or passed samplers in a thread?
Any help is much appreciated.
Thanks
By adding Aggregate Report listener to your test, you will be able to get the number of passed and failed requests details. Below is the purpose of aggregate report.
The aggregate report creates a table row for each differently named
request in your test. For each request, it totals the response
information and provides request count, min, max, average, error rate,
approximate throughput (request/second) and Kilobytes per second
throughput. Once the test is done, the throughput is the actual
through for the duration of the entire test.
Also, you can leverage View Results Tree, View Results in Table and Debug Sampler to debug your samplers.
To enable functional test mode in JMeter, click on the Test Plan and then check Functional Test Mode check box as shown below.
I currently do service using beanstalkd and node.js.
I would like when jobs fail, retry n time before give up the job.
If the job succede i want do it the same job 10 time.
So, what is the best practice, stock in mongo db with the jobId the error and success count, or delete and put a new job with a an error and success count in the body.
I dont know if i'm clear? so tell me , thanks a lot
There is a stats-job <id>\r\n that should also be available via the API library that returns, among other things, how many times the specific job has been reserved, released, buried, and so on.
This allows for a number of retries of failed jobs by checking previous reservation/releases.
To run the same job multiple times, I would personally create either one additional job, with a success count that would then be incremented (into another new job) - or, all nine new jobs, with optional delays before they start.
You have a couple of ways to do this:
you can release the job, and obtain from stats the number of reserves
you can put a new job with a retry count, and keep track of history in the data payload
You should do the later, and you don't need MongoDB as a second dependency.