Here https://support.smartbear.com/readyapi/docs/testing/scripts/libs/groovy-lib.html at the bottom of the page, it says how to dynamically load a library in ReadyApi.
However the solution proposed isn't qualitative; it says to do a Thread.sleep(3000)... But this isn't a guarantee that the lib is effectively loaded!
As proof, I did this in my "Setup script" of my test suite:
String customGroovyLibsPath = (new File(testSuite.project.getPath())).getParent() + File.separator + "GroovyLibs"
if (System.properties[ 'soapui.scripting.library' ] != customGroovyLibsPath) {
System.properties[ 'soapui.scripting.library' ] = customGroovyLibsPath
Thread.sleep(3000)
}
On the first execution of my test suite, most of the time, it ends up in an error.
So, my question is how can I make "absolutely" sure my library is fully loaded, without using Thread.sleep and passing to it an arbitrary big amount of time?
Kind regards.
Related
I am working on a node-based MUD game and I would like to limit the amount of time any one command can execute before it gets killed (e.g. 1000ms). I found a module called Tripwire which seems promising but it does not appear to be actively maintained. Tripwire does work as advertised. It manages to force an exception if someone creates an endless loop, but it does not support any resumption of the original script thread.
I am looking for either:
(1) A similar but actively maintained Node module that can interrupt and resume the original event thread, or,
(2) A working example of V8's Isolate::IsExecutionTerminating + Isolate::CancelTerminateExecution (I forked Tripwire but I haven't done any meaningful C++ in a long time and am now just beating my head against the wall).
I have only been able to find test cases so far (which is at least something). I am really hoping that someone has already tackled this, though.
Test cases:
https://chromium.googlesource.com/v8/v8/+/ad55afcb459dafda1cf48e676985717fd7eae786/test/cctest/test-thread-termination.cc
I know this is a bit vague.
I ended up instrumenting the script by passing it through acorn and generating my own final script. I am hoping that the sandbox is locked down to prevent users from escaping it. Example of "compiled" output:
createPermissions(expr) {
let __mec = __bfc(this || GameMaster, 'public', 'createPermissions', __FILE__, false); try { let parts = expr.split('/');
for (let i = 0; i < parts.length; i++) {
__ala(); let foo = parts.slice(0, i).join('/');
} } finally { __efc(__mec, 'createPermissions'); }
}
This new "language" supports public, protected, package, and private variables/methods (by maintaining its own internal call stack, execution context, etc). The directives are "reserved words" (e.g. __bfc=begin function call, __ala=assert loop alarm).
Thanks #jfriend00 for the suggestion.
For those who are curious: Transpiler Module
I'm working on a project which uses nodejs and nighwatch for test automation. The problem here is that the tests are not reliable and give lots of false positives. I did everything to make them stable and still getting the errors. I went through some blogs like https://bocoup.com/blog/a-day-at-the-races and did some code refactoring. Did anyone have some suggestions to solve this issue. At this moment I have two options, either I rewrite the code in Java(removing nodejs and nightwatch from solution as I'm far more comfortable in Java then Javascript. Most of the time, struggle with the non blocking nature of Javascript) or taking snapshots/reviewing app logs/run one test at a time.
Test environment :-
Server -Linux
Display - Framebuffer
Total VM's -9 with selenium nodes running the tests in parallel.
Browser - Chrome
Type of errors which I get is element not found. Most of the time the tests fail as soon the page is loaded. I have already set 80 seconds for timeout so time can't be issue. The tests are running in parallel but on separate VM's so I don't know whether it can be issue or not.
Edit 1: -
Was working on this to know the root cause. I did following things to eliminate random fails: -
a. Added --suiteRetries to retry the failed cases.
b. Went through the error screenshot and DOM source. Everything seems fine.
c. Replaced the browser.pause with explicit waits
Also while debugging I observed one problem, maybe that is the issue which is causing random failures. Here's the code snippet
for (var i = 0; i < apiResponse.data.length; i++) {
var name = apiResponse.data[i];
browser.useXpath().waitForElementVisible(pageObject.getDynamicElement("#topicTextLabel", name.trim()), 5000, false);
browser.useCss().assert.containsText(
pageObject.getDynamicElement("#topicText", i + 1),
name.trim(),
util.format(issueCats.WRONG_DATA)
);
}
I added the xpath check to validate if i'm waiting enough for that text to appear. I observed that visible assertion is getting passed but in next assertion the #topicText is coming as previous value or null.This is an intermittent issue but on test server happens frequently.
There is no magic bullet to brittle UI end to end tests. In the ideal world there would be an option set avoid_random_failures=true that would quickly and easily solve the problem, but for now it's only a dream.
Simple rewriting all tests in Java will not solve the problem, but if you feel better in java, then I would definitely go in that direction.
As you already know from this article Avoiding random failures in Selenium UI tests there are 3 commonly used avoidance techniques for race conditions in UI tests:
using constant sleep
using WebDriver's "implicit wait" parameter
using explicit waits (WebDriverWait + ExpectedConditions + FluentWait)
These techniques are also briefly mentioned on WebDriver: Advanced Usage, you can also read about them here: Tips to Avoid Brittle UI Tests
Methods 1 and 2 are generally not recommended, they have drawbaks, they can work well on simple HTML pages, but they are not 100% realiable on AJAX pages, and they slow down the tests. The best one is #3 - explicit waits.
In order to use technique #3 (explicit waits) You need to familiarize yourself and be comfortable with the following WebDriver tools (I point to theirs java versions, but they have their counterparts in other languages):
WebDriverWait class
ExpectedConditions class
FluentWait - used very rarely, but very usefull in some difficult cases
ExpectedConditions has many predefinied wait states, the most used (in my experience) is ExpectedConditions#elementToBeClickable which waits until an element is visible and enabled such that you can click it.
How to use it - an example: say you open a page with a form which contains several fields to which you want to enter data. Usually it is enought to wait until the first field appears on the page and it will be editable (clickable):
By field1 = By.xpath("//div//input[.......]");
By field2 = By.id("some_id");
By field3 = By.name("some_name");
By buttonOk = By.xpath("//input[ text() = 'OK' ]");
....
....
WebDriwerWait wait = new WebDriverWait( driver, 60 ); // wait max 60 seconds
// wait max 60 seconds until element is visible and enabled such that you can click it
// if you can click it, that means it is editable
wait.until( ExpectedConditions.elementToBeClickable( field1 ) ).sendKeys("some data" );
driver.findElement( field2 ).sendKeys( "other data" );
driver.findElement( field3 ).sendKeys( "name" );
....
wait.until( ExpectedConditions.elementToBeClickable( buttonOK)).click();
The above code waits until field1 becomes editable after the page is loaded and rendered - but no longer, exactly as long as it is neccesarry. If the element will not be visible and editable after 60 seconds, then test will fail with TimeoutException.
Usually it's only necessary to wait for the first field on the page, if it becomes active, then the others also will be.
I'm trying to create selenium tests that run each step synchronously, without using .then(), or async/await. The reason for this is that I want to create a set of functions that allow pretty much anyone on our test team, almost regardless of tech skills to write easy to read automated tests. It looks to me like webdriver-sync should give me exactly what I want. However, the following dummy code is producing problems:
var wd = require('webdriver-sync');
var By = wd.By;
var Chromedriver = wd.Chromedriver;
var driver = new Chromedriver;
driver.get('https://my.test.url');
var myButton = driver.findElement(By.cssSelector('[id*=CLICK_ME]'));
myButton.click();
It tries to run - browser is launched, and page starts to load... but the steps are not executed synchronously - it goes on and tries to find and click "myButton" before the page has finished loading, throwing a "no such element" error... which to me kinda defeats the point of webdriver-sync?! Can someone tell me where I am going wrong?
FWIW, I have webdriver-sync 1.0.0, node v7.10.0, java 1.8.0_74, all running on CentOS 7.
Thanks in advance!
You need to put double-quotes around "CLICK_ME" as it's a string value.
Generally, though, it's a good idea to Wait for specific elements because dynamic pages are often "ready" before all their elements have been created.
I have 2 nodeunit test cases in which if first fails i don't want to run 2nd test case.(Process should skip testing 2nd test case with proper result.) How do I do the same.
I have written sample code where In first test case I have specified number of expects should be one. Even if first test case fails, second test case still executes. What could be the issue here.
exports.testSomething = function(test){
test.expect(1);
test.ok(false, "this assertion should fail");
test.done();
};
exports.testSomethingElse = function(test){
test.ok(true, "this assertion should pass");
test.done();
};
Generally it is a bad idea to make separate tests dependent on eachother, and testing frameworks don't support this kind of thing because of that. Nodeunit tests are declared and processed for execution before any one test has started so there is no way to change which tests run.
For that specific example, if the add fails, then it makes sense that delete fails. It might be slightly annoying to get more output, but it's doing just what it is supposed to do.
It may not be applicable in this cause, but one option is to have a separate section for your delete tests where you use setUp to set up everything that delete needs without having to actually call add.
Update
test.expect is used to specify how many assertions were expected in a single test. You only really need it if your test has asynchronous logic. For example:
'test': function(test){
test.expect(5);
test.ok(true, 'should pass');
for (var i = 0; i < 4; i++){
setTimeout(function(){
test.ok(false, 'should fail');
}, 5);
}
test.done();
},
Without the test.expect(5), the test would pass because done is called before the line in the setTimeout has run, so the test has no way of knowing that is has missed something. By having the expect, it makes sure the test will properly fail.
In Node.js unit tests, what is the way to create data driven unit tests?
For Example, I've a common function / method, which I want to re-use in multiple unit tests with different sets of data. I tried looking into nodeunit, vows, whiskey, qunit, expresso; But I wasn't able to figure out a way to achieve this functionality.
I was not looking at calling the function literally in multiple tests, but rather use the common method in a loop to pick up the data in each iteration and execute it, as a unittest
The reason for this is, I've atleast 1000 rows of parameterized data, for which I want to write unittest. Obviously I cannot go on writing 1000 unittests physically.
Anyone could you please point me a way to achieve the above.
There is qunit addon which allows to run parameterized quint tests
https://github.com/AStepaniuk/qunit-parameterize
So you can separate test data and test method and run the same test method against different data sets.
This is a pretty old post, but I just hit this problem myself and wasn't able to find a clean solution for QUnit without using the plugin referenced by the other comment (qunit-parameterize). Honestly, I couldn't figure out how to integrate the plugin with my company's project and gave up after about an hour.
This is how I ended up solving it:
Just define an array with your inputs (and expected outputs, if needed), iterate over your array, and define the QUnit test in the callback! Super simple, really, but worked quite well.
const testCases = [
{ input: "01/01/2015", expected: "2015-01-01" },
{ input: "09/25/2015", expected: "2015-09-01" },
{ input: "12/31/2015", expected: "2015-12-01" }
];
testCases.forEach(testCase => {
QUnit.test("gets first of month",
() => {
const actual = new classUnderTest().getFirstOfMonth(testCase.input);
strictEqual(actual, testCase.expected);
});
});
I wasn't sure that QUnit would discover the test if it were nested as such, but it does just fine.
Enjoy!