Integrating Node.js with QUnit to Jenkins - node.js

This is my first sof post so forgive my format and organization of thought. I've made a great effort to solve my problem before posting this. Part of my issue could be lack of knowledge with packages in Ubuntu or Node.js so please guide me.
I'm trying to create a XUnit xml file for Jenkins from QUnit tests for a Node.js application. I don't have the ability to run a browser or even a headless browser, also don't understand why I'd need one since the Node.js code doesn't deal with the browser.
I've been searching all over and have only been successful using qunit-tap and 'prove' to create an XML file. Prove required downloading a formatter which was a perl file. We are trying to prevent using perl stuff.
My system is an Ubuntu VM. This is a task for work and my boss is asking for the minimal amount of packages and dependencies. Our Node.js server is accepting web socket requests and passing messages back and forth with a legacy system written in php.
QUnit's output seems to be a pretty print format, in a table, when I run my tests in the console. It would be amazing to just get that into a flatter form with a flag!
Thanks in advance!

Well, for NodeJS you can go with Grunt and grunt-contrib-qunit, though I would recommend the following approach:
Leverage JUnit Logger
(https://github.com/jquery/qunit-reporter-junit) plugin for JUnit
compatible report.
Comment out console.log output in the PhantomJS Runner https://github.com/jquery/qunit/tree/master/addons/phantomjs to mute non-XML output produced by the runner
Assign a task for Apache Ant build script:
<target name="qunit" description="runs QUnit tests using PhantomJS">
<echo message="Executing QUnit Javascript Unit Tests..."/>
<exec executable="/usr/local/bin/phantomjs" output="./build/qunit/qunit-results.xml">
<arg value="./vendors/Runner/runner-muted.js" />
<arg value="test-runner.html" />
</exec>
</target>
Jenkins shall look for the report in ./build/qunit/qunit-results.xml

ditto, above answer good
use the junit report out and connect it to the standard jenkins unit testing plugin -> http://wiki.jenkins-ci.org/display/JENKINS/xUnit+Plugin
for nice easy to configure self-installed nodejs on the machine, i have to recommend the excellent -> http://wiki.jenkins-ci.org/display/JENKINS/NodeJS+Plugin
for jshint/csslint reports, i found the https://wiki.jenkins-ci.org/display/JENKINS/Checkstyle+Plugin plugin very nice, jshint and csslint both output to this
jshint : {
options : {
reporter : 'checkstyle',
reporterOutput : 'reports/jshint.xml',
},
src : "..."
},
csslint : {
strict : {
options : {
formatters : [{
id : 'checkstyle-xml',
dest : 'reports/csslint.xml'
}],
csslintrc: '.csslintrc'
},
src : [...],
},
},

Related

How to hide Tomcat version from error messages when using embedded servers in Java

I have a java application where i'm using embedded Tomcat servers,
which looks like this
Tomcat tomcat = new Tomcat()
I'm creating an embedded tomcat server here.
Problem statement
whenever there's an error it displays information on which tomcat version i'm using,
how to hide this in java?
i have a little idea that i need to override ServerInfo.properties, but how do i do this?
I'm not sure how we can do this in java, but if you are using any build scripts like ant / gradle for distribution purpose, we can write a task to override / harden the jar file, and replace the ServerInfo.properties file with the customized value whatever we need.
the code for ant build scripts would look like
<target name="override.tomcat">
<jar destfile="path/to/tomcat-embed-core-9.0.62.jar" update="true">
<fileset dir="src/"> <!-- folder where you keep the directory/file to raplace-->
<include name="org/apache/catalina/util/ServerInfo.properties"/> <!-- file to replace within directory path in side the jar-->
</fileset>
</jar>
</target>
and in gradle
task overRideTomcat(type: Jar) {
from(zipTree(file("path/to/tomcat-embed-core-9.0.62.jar"))) {
exclude '**/org/apache/catalina/util/ServerInfo.properties'
}
from('src/') {
include('/org/apache/catalina/util/ServerInfo.properties')
}
archiveName "tomcat-embed-core-9.0.62.jar"
}
make sure you have the modified ServerInfo.properties file under src directory in the same path as you have mentioned in the include statement.

Rerunning failed scenarios with Selenide/Cucumber - no rerun.txt file

I writing automation tests with Cucumber/Selenide and I want to rerun failed scenarios.
This is part of my project with only two small tests (one is failing) to demonstrate behavior: https://github.com/mtpx/cucumberRerun
I read how to do it on How to rerun the failed scenarios using Cucumber? and https://medium.com/#belek.bagishbekov/how-to-rerun-failed-test-cases-in-cucumber-b7fe9b1dcf9c
In my application.feature test runner(ApplicationTest) in #CucumberOptions's plugins section I have line: "rerun:rerun/failed_scenarios.txt", according to previous urls it should generate text file with failed scenario, but after test execution with 'mvn clean test' (with failed scenarios) - there's no any rerun.txt file.
Do You know what is wrong here? Why after build i dont have rerun.txt file?
I using Selenide instead of Selenium, maybe problem is here?
Create another scenario file as shown below. Let's say this as FailedScenarios.java. Whenever you notice any failed scenario run this file. This file will use target/rerun.txt as an input for running the scenarios.
This line is require:
features = "#target/rerun.txt",
Full CucumberOptions
#CucumberOptions(
monochrome = true,
features = "#target/rerun.txt", //Cucumber picks the failed scenarios from this file
format = {"pretty", "html:target/site/cucumber-pretty",
"json:target/cucumber.json"}
)
public class FailedScenarios {
}
You can use rerun file path other than target if you need to run failed Scenario also trigger from maven , In that case change the path in both file you main runner and failed test runner
problem solved :)
In pom i had line:
-Dcucumber.options="--plugin io.qameta.allure.cucumberjvm.AllureCucumberJvm"
This line overrides all plugins information in TestRunner

Is there a html reporter to use with jest-cucumber which shows all the Gherkin annotations on report

I have started using jest-cucumber (https://github.com/bencompton/jest-cucumber) for automating my ReactJS app.
I have tried these reporters with the framework: https://github.com/jest-community/awesome-jest#reporters but I'm not able to see Gherkin steps on the report.
All reports show the test case level details.
I tried to use cucumber-html-reporter https://www.npmjs.com/package/cucumber-html-reporter but unable to use it with jest-cucumber.
Can you suggest any good library which can report the Gherkin annotations step by step which will look like https://www.npmjs.com/package/cucumber-html-reporter
Steps to be followed:
1) Install jest-cucumber with version v2.0.12 that supports reporting. During this writeup this version was not available as stable version of npm. You need to specifically give the version number when installing this.
2) Add below paramter in jest config to specify the report storage location. The jest-cucumber outputs a json file.
reporters: [
"default",
[
"./node_modules/jest-cucumber/dist/src/reporter", //This is the default path
{
formatter: "json",
path: "./tests/integration-test-results/test-report.json" //specific path
}
]
]
3) Create a index.js file for cucumber-html-report conf and mention the location of above created json file and output when the html file need to be placed. For more info, refer: https://www.npmjs.com/package/cucumber-html-reporter
4) Create package.json command as shown below for ease of use so that once the BDD test completes, the report is automatically generated and displayed.
"test-bdd": "jest --config=jest-cucumber.config.js && node ./cucumber-report-config.js"
5) Run: npm run test-bdd
I got it working with cucumber-html-reporter and jest-cucumber version 2.0.12. Please check the issue #27 of jest-cucumber: https://github.com/bencompton/jest-cucumber/issues/27.

Jenkins failed: Test reports were found but none of them are new. Did tests run?

I'm testing the Jenkins jUnit plugin with a manual jUnit Xml file. And I get following info from console output
failed: Test reports were found but none of them are new. Did tests run?
I've searched the solution for this problem but none of them solve my confuse. I try to run a shell script. Since I'm pretty new to this(might have a lot of errors). Here is a piece of my shell script.
cd /Users/Shared/Jenkins/Home/jobs/jUnitReport
sudo visudo
Jenkins ALL = NOPASSWD: /bin/sh -xe /Users/Shared/Jenkins/tmp/hudson7751940722564747051.sh
touch jUnit.xml
I don't really know how to access Jenkins to touch the file.(one of the solution to "failed: Test reports were found but none of them are new. Did tests run?").
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
<testsuite name="PerformanceTest" errors="0" skipped="0" tests="2" failures="1" time="10.74" timestamp="2016-05-24T10:23:58">
<testcase classname="PerformanceTest" name="testSmoothness" time="15" jank-percentage="75">
<failure message="Too many jank frames." />
</testcase>
<testcase classname="PerformanceTest" name="testResponseTime" time="1.3" response-time="0.3">
</testcase>
</testsuite>
</testsuites>
Any help will be appreciated.
Thank You!
(copied from my comment)
Source. There is a timestamp check but looks like it works on file timestamps (last modified) and uses a jitter of 3000 (presumably milliseconds). The JUnit plugin is ignoring test XML output that is older than the buildtime of the current build minus 3 seconds. If the test run and the JUnit plugin run are more than 3 seconds apart, the plugin will ignore the files.

cucumber jvm CucumberException: No features found at []

In my cucumber -jvm, Maven, junit Setup I have my testRunner file as
package com.lebara.testrunner;
import cucumber.junit.Cucumber;
import org.junit.runner.RunWith;
#RunWith(Cucumber.class)
#Cucumber.Options(
glue = {"com.lebara.stepdefs","com.lebara.framework.main", "com.lebara.testrunner"},
features = "C:/Users/sarthak.dayanand/Documents/WebRefreshTest/CukeAutomation/LebaraWebAutomationTest1/src/main/resources",
format = {"pretty", "html:target/cucumber-html-report", "json-pretty:target/cucumber-report.json"},
tags = {"#UserJourney"}
)
public class RunCukesTest {
}
I have my feature file in the above mentioned directory.
If I run it, I get the following exception:
cucumber.runtime.CucumberException: No features found at [C:/Users/sarthak.dayanand/Documents/WebRefreshTest/CukeAutomation/LebaraWebAutomationTest1/src/main/resources/cucumber]...
If I remove the "features" option in the testrunner, it tries to look for feature files in the same directory as my testrunner.java
cucumber.runtime.CucumberException: No features found at [com/lebara/testrunner]
And if I put the feature files there, it works.
My question is why is my feature file not being picked up from my previous location, which I thought to be the default file structure for cucumber - maven setup.
How do I make it pick up from there? Help appreciated.
Where exactly are your test runner and feature files? I've got the following setup which works perfectly:
src/test/
java/
com/mypackage/
TestRunner.java
StepDefinition.java
resources
com/mypackage/
fancy.feature
The Maven/Cuke conventions will have the tests executed from the tests/java directory and the feature files found in the test/resources directory. My test runner is basically the same as yours but with less options:
import cucumber.api.junit.Cucumber;
import org.junit.runner.RunWith;
#RunWith(Cucumber.class)
#Cucumber.Options(format = {"pretty"})
public class TestRunner { }
Hope this helps if you hadn't already found an answer.
I have a setup similar to yours (not using the Maven/Cucumber conventions). In my options, I don't specify the path from root, but from the project's source folder where the features are held. It makes sense, since otherwise the tests would only be runnable from your machine.
In your case, I think it should be:
features = "src/main/resources"
Just add features = { "classpath:features/feature.feature"}, and the feature must under test/resources/features/feature.feature.
#CucumberOptions(
format = {"pretty", "html:target/html"},
features = {"classpath:features/feature.feature"},
snippets = SnippetType.CAMELCASE
Note classpath.
When you compile your code if you are using maven open up target/test-classes/features and you will see feature.feature
//Removing the space between "**classpath**" and "**:com/**" helped.
#RunWith(Cucumber.class)
#CucumberOptions(
features = {"classpath:com/tk/feature/"}, //NOTE: NO SPACE
glue = {"classpath: com.tk.cucumber"},
plugin = {
"pretty",
"html:build/reports/cucumber"
,"json:build/reports/cucumber-tests/test.json"}
)
public class RunAPITests {}
If you are providing the complete path of the feature file i.e.
"C:/Users/sarthak.dayanand/Documents/WebRefreshTest/CukeAutomation/LebaraWebAutomationTest1/src/main/resources" as in your query, try again by replacing the '/' character with '\\'(double back slash) as below.
"C:\\Users\\sarthak.dayanand\\Documents\\WebRefreshTest\\CukeAutomation\\LebaraWebAutomationTest1\\src\main\\resources\\abc.feature"
This is a git repo which uses the latest cucumber version : Cucumber- Example
Clone this repo and run it in your local machine. The #Given is defined and it should pass. The #Then and #When should be shown as undefined.
This is how the output for it should look :
Output for the Belly feature
Use the structure mentioned :
src / test / java/ io /cucumber / {Step definitions java and run cucumber test files here}
src /test / resources/ io/ cucumber / {feature files here}
You can run the gradle build using ./gradlew clean build
and the cucumber test using ./gradlew clean test --info
If this works, then use the same format in your project.
Just changing .Feature to .feature the problem got resolved for me.
Also make sure the path for feature is righly mention in CucumberOptions as per your feature folder
Some of the online tutorial have mentioned .Feature which brings this problem
so changing the case will solve this problem
There is another instance in which 'Feature Not Found' error occurs. I am posting the solution under this answer as there is no similar question.
I got this error when trying to run the Runner file first time after setting up Cucumber project in Maven. The solution i found was as follows: Go to the folder in which the 'feature' file is present in Windows Explorer. Check the size of the feature file you are trying to run. If the size is '0' KB, it will show the 'Feature Not Found' error. Make some changes to file until a value greater than zero is displayed. Run again after making changes.
#RunWith(Cucumber.class)
#CucumberOptions(
features = {"src/main/resources/cucumber/features"},//your feature path
tags = "not #Wip",
glue = {"classpath:steps"},
plugin = {"pretty", "html:target/cucumber/html"})
You must set the feature directory correctly
By putting the feature file under src/test/java where the runner and steps file or
by putting it under src/main/java the problem will get resolved.

Resources