I am unable to figure out how to filter out tests based on their run statuses in FitNesse. I'm commenting out all the passed tests manually so that the next run can ignore them. How can I only rerun failed tests?
There is no feature in FitNesse to do this. Usually you want to keep running passed tests to ensure that nothing has caused them to break.
Related
I am trying to figure out how to use Jest properly. The interaction of Jest and console.log is very poorly documented (and apparently very inconsistent according to lots of tickets).
The Jest documentation is terrible.
When is console.log output supposed to show up? Always? Sometimes?
Seems to me the desired behavior would be for Jest to show console.log output for the tests that failed. I don't see anything about that in the documentation. What I do see is that console.log output seems to correlate somehow being either before or after tests.
Can you explain?
By default - Jest will print all console log results on the terminal which the tests are executed from. Instances when users do not see their logs are usually caused by the silent property being set to true in their config file or the --silent flag being added to their cli execution, which is not default Jest behaviour.
The ---verbose flag causes the results of all individual it unit tests to be printed in the terminal (whereas by default the terminal would only print the names of the test suites that have passed and only unique unit tests (it) that have failed). Additionally each individual test will also display the length of time they took to execute if they they take longer than 1ms. Another interesting fact is that --verbose will also print logs of console.log executions that are run during the evaluation of a module (when a module is imported - e.g. a console statement that simply exists in the body of a javascript file or an implicitly invoked function) even though it is not used (e.g. the tests that uses it are not executed due to the xdescribe keyword in their test suite).
The ability to print console log results exclusively for failed tests is not present in Jest.
I have a weird behaviour in cypress lately
When i debug, the issue is that the test starts with this url account.domain.com
as the test goes on, it naturally move to app.domain.com.
All is good for the first test.
The second one uses the same logic, start with account.domain.com ...
While the first one ended the test with the url app.domain.com, it seems that cypress is unable to load account.domain.com in the next test, it doesn't show any error, it just keep loading
Do you have any solution for this please ?
I'm using cucumber by the way
Is there any way I can stop the whole robot test execution with PASS status?
For some specific reasons, I need to stop the whole test but still get a GREEN report.
Currently I am using FATAL ERROR which will raise a assertion error and return FAIL to report.
I was trying to create a user keyword to do this, but I am not really familiar with the robot error handling process, could anyone help?
There's an attribute ROBOT_EXIT_ON_FAILURE in BuiltIn.py, and I am thinking about to create another attribute like ROBOT_EXIT_ON_SUCCESS, but have no idea how to.
Environment: robotframework==3.0.2 with Python 3.6.5
There is nothing built-in to support this. By design, a fatal error will cause all remaining tests and suites to have a FAIL status.
Just about your only choice is to write a keyword that sets a global variable, and then have every test include a setup that uses pass execution if to skip the test if the flag is set.
If I understood you correctly, you need to pass the test execution forcefully and return green status for that test, is that right? You have a built in keyword "Pass Execution" for that. Did you try using that?
I have a simple Node JS application and am using Istanbul with Mocha to generate code coverage reports. This is working fine.
If I write a new function, but do not create any tests for it (or even create a test file) is it possible to check for this?
My ultimate goal is for any code which has no tests at all to be picked up by our continuous integration process and for it to fail that build.
Is this possible?
One way you could achieve this is by using code coverage.
"check-coverage": "istanbul check-coverage --root coverage --lines 98 --functions 98 --statements 98 --branches 98"
Just add this in your package.json file, change the threshold if needed. If code is written but no test then the coverage will go down.
I'm not sure if this is the correct way to solve the problem but by running the cover command first and adding the parameter --include-all-sources this then reported on any code without a test file and added them to the coverage.json file it generated.
Then running the check-coverage would fail which is what I'm after. In my CI process I would run cover first, then check-coverage
Personally I find the documentation on Istanbul a little bit confusing/un-clear which is why I didn't see this at first!
Say I have a maven2/3 project in jenkins/hudson and BEFORE I run some goals on a maven project configured in the correspoing config.xml file, I want to run a system groovy script (ref. system groovy plugin) during a prestep and interrupt the whole job and set it to SUCCESS if some condition is met (for example say I find something in the log file of the previous job). I DO NOT WANT MAVEN TO START EXECUTING THE GOALS.
I have tried
import hudson.model.*
def thr = Thread.currentThread()
def build = thr?.executable
build.executor.interrupt(hudson.model.Result.SUCCESS)
out.print "HELLO"
But nothing happens, and even "HELLO" is printed in log. But then the build gets ABORTED.
Parsing POMs
Discovered a new module ...
Modules changed, recalculating dependency graph
...
...jdk1.6.0_22/bin/java -Xmx512m -cp ...
<===[JENKINS REMOTING CAPACITY]===>Build was aborted
Thanks for your time.
I do not understand fully, what you want, since you described here some solution, not your exact problem. I know three plugins, which could be useful for your problem as well:
Fail the build plugin lets you set the result of the job, and stops further processing. Any status can be set including success.
Conditional build step plugin lets you define conditions for its child build steps. If the condition is met, the child(ren) will run.
m2 extra build steps - lets you run build steps before or after maven build in a maven job in jenkins. Note, that recently, this plugin is the part of the core jenkins.
So the basic idea is that you could add conditional build step a pre-build step in your job, and the child step could be one fail-the build instance. See the picture below: