I have some precondition for Suite and they should be runs once per suite, so I will add on Suite Setup.
also, I have some precondition for each test-case and they should run at the start of each test-case.
The question is if I use both of them, which one start first if I only run one of the test-cases? Suite Setup or Test Setup?
something like this:
*** Settings ***
Library ...
Varialbles ...
Suite Setup: suite_precondition
Test Setup: test_precondtion
*** Test Cases ***
TC1
<Some code>
TC2
<Some code>
You know that we can run TC1, or TC2 one by one for checking the test-case PASS or not. So what happened here when I run TC1?
Suite setup always runs before any tests start. After that, the test setup will run for each test when the test first starts.
Related
I wanted to run specific test suite in jest-puppeteer for UI automation.
I am able to run the specific test using below.
jest -t test_name
where test_name is from the "it" not from "describe" without quote.
Is there any other cmd to run all the test under describe too.
Run tests that match this spec name (match against the name in describe or test, basically).
jest -t name-of-it
jest -t "name-of-describe"
I have a project built with Nix on Hydra. When the tests are run for it, it prints the logs to a file, which makes them hard to view from a CI server. Is there a way I can make the project print stdout/err as the test suite runs? This is what I currently get:
Running 1 test suites...
Test suite test: RUNNING...
Test suite test: FAIL
Test suite logged to: dist/test/project-name-0.0.0-test.log
0 of 1 test suites (0 of 1 test cases) passed.
Edit: The cabal new-run test:test part is incorrect. I'm looking into how to get Nix's Haskell builder to stream test results.
You can override the checkPhase in Nix to accomplish this. You can see it defined here: https://github.com/NixOS/nixpkgs/blob/32340793aafec24dcef95fee46a21e634dd63457/pkgs/development/haskell-modules/generic-builder.nix#L357-L361
This checkPhase should work:
checkPhase = ''
runHook preCheck
cabal new-run test:test
runHook postCheck
'';
Circle ci is running the deploy section of my circle.yml depsite the test section failing. I'd expect that if anything goes wrong in the test section, the deploy section wouldn't be run, but it is.
There's two command in the test section the second one fails with :
npm run test:coverage -- --maxWorkers=2 returned exit code 1
I do not seem to be the first one, and the oldest post is a few months old:
https://discuss.circleci.com/t/deployment-triggered-after-tests-fail/12356/3
Is this a bug or am I doing something wrong ?
Ideas ?
circleci v 1.0
I've setup Jenkins to run unit test on NodeJS and deploy it to another servers if the test coverage match with my condition.
I use AWS with 2 instances to host Jenkins and Apps. Below is steps that I follow:
Setup Jenkins on instance 1.
Launch Jenkins and configure the build step
At build step, I ssh to instance 2.
cd to src folder at instance 2 and git pull my repository.
Run the unit test using Istanbul and export to test.TAP, now the test.TAP is placed in instance 2.
Back to Jenkins in instance 1, I configure Publish TAP result on Post-build Actions.
<-- My concerns right here is how can I get the test.tap file in instance 2 to read the report and display in Jenkin?
Please help me.
Thank you.
I am trying to integrate tSQLt / SQLTest with CruiseControl.NET
My tests are running and I've written xsl files to display the results but I need to know how to mark the build as failed if any tests fail.
My CCNet exec is:
<exec executable="$(sqlCmdPath)">
<description>Run Unit Tests</description>
<buildArgs>-E -d MyDatabase
-i "\CruiseControlProjects\Configuration\CI_SQL\RunTests.sql"
</buildArgs>
<baseDirectory>\Artifacts\MyDatabase</baseDirectory>
<successExitCodes>0,63</successExitCodes>
</exec>
RunTests.sql:
IF EXISTS (SELECT * FROM sys.objects WHERE OBJECT_ID = OBJECT_ID(N'[tSQLt].[RunAll]')
AND TYPE IN (N'P',N'PC'))
BEGIN
EXECUTE [tSQLt].[RunAll]
END
The tests are run and I have a subsequent task which produces the results in xml that are then merged into the build log:
<exec executable="$(sqlCmdPath)">
<description>Get Unit Tests</description>
<buildArgs>-E -b -d MyDatabase -h-1 -y0 -I
-i "\CruiseControlProjects\Configuration\CI_SQL\GetTestResults.sql"
-o "\CruiseControlProjects\Configuration\CI_SQL\Results\TestResults.xml"
</buildArgs>
<baseDirectory>\Artifacts\MDatabase</baseDirectory>
<successExitCodes>0,63</successExitCodes>
</exec>
So how do I get the overall build to fail?
If you use the -b parameter to sqlcmd, you should find that it will throw an error with a non-zero code when the batch fails (which will happen if tSQLt fails at least one test).
However, I have one potential suggestion to explore. If you can load the XML file within Cruise Control, then the tests can be loaded in as the XML file is in the same format as an nUnit test output file. (Note - I've used this method on TeamCity and Jenkins, but not tried with Cruise Control). This will treat the tests as tests rather than an 'all-or-nothing' approach, and enabling you to track which tests fail repeatedly.
Hope that helps,
Dave.