How to make ZombieJS wait indefinitely for a site - node.js

I'm using zombieJS to scrape a veeeeeeeery slow site. I tried many things to make it go slower, but I'm receiving lots of
TypeError: Cannot use 'in' operator to search for 'compareDocumentPosition' in null
errors.
I tried to add to my pressButton function the following:
browser.wait({waitDuration: '700s', element: "pre"});
while initializing the browser with this configuration:
browser = new Browser();
browser.maxWait = 10000000;
browser.runScripts = false;
browser.loadCSS = false;
browser.waitFor = 500000;
but I'm still receiving the above mentioned error after a few seconds...

I think this might apply to you: Zombiejs jQuery nullTypeError 'compareDocumentPosition'
your site is so slow, that the tag cannot be found early enough from zombie. Would you please so kind, to open a new issue for assaf on github? We tried to track down the cause of this random-error earlier, but now I think it's caused because zombie should wait for the first dom element to be loaded

I also had similar problem and it got solved by removing debug option, while creating instance of browser. Also downgrade to v1.4.1 as 2.0 is in alpha stage

Related

What is Comunication Failure in Shopware 5?

when creating new theme there's error occurred.
0 - Communication Failure
Why this happen? could you please help me?
This usually happens due to a timeout that occurs when the Theme-controller tries to read the Theme's configuration for the first time. Unfortunately, this is quite a resource-heavy process; on weaker servers, timeouts may occur during this process quite often.
You can confirm this by opening the Theme-Manager, opening your browser's developer tools, refreshing the Theme overview and look at the response of the backend/Themes/list-Request.
You can give your server more time with the php-function set_time_limit. In engine/Shopware/Components/Theme/Installer.php, in the synchronize-method, prepend set_time_limit(0):
public function synchronize()
{
set_time_limit(0);
$this->synchronizeThemes();
}
Alternatively, prepend set_time_limit(0); to your shopware.php file, but don't forget to remove it again once the theme-overview loaded successfully.

Random failure of selenium test on test server

I'm working on a project which uses nodejs and nighwatch for test automation. The problem here is that the tests are not reliable and give lots of false positives. I did everything to make them stable and still getting the errors. I went through some blogs like https://bocoup.com/blog/a-day-at-the-races and did some code refactoring. Did anyone have some suggestions to solve this issue. At this moment I have two options, either I rewrite the code in Java(removing nodejs and nightwatch from solution as I'm far more comfortable in Java then Javascript. Most of the time, struggle with the non blocking nature of Javascript) or taking snapshots/reviewing app logs/run one test at a time.
Test environment :-
Server -Linux
Display - Framebuffer
Total VM's -9 with selenium nodes running the tests in parallel.
Browser - Chrome
Type of errors which I get is element not found. Most of the time the tests fail as soon the page is loaded. I have already set 80 seconds for timeout so time can't be issue. The tests are running in parallel but on separate VM's so I don't know whether it can be issue or not.
Edit 1: -
Was working on this to know the root cause. I did following things to eliminate random fails: -
a. Added --suiteRetries to retry the failed cases.
b. Went through the error screenshot and DOM source. Everything seems fine.
c. Replaced the browser.pause with explicit waits
Also while debugging I observed one problem, maybe that is the issue which is causing random failures. Here's the code snippet
for (var i = 0; i < apiResponse.data.length; i++) {
var name = apiResponse.data[i];
browser.useXpath().waitForElementVisible(pageObject.getDynamicElement("#topicTextLabel", name.trim()), 5000, false);
browser.useCss().assert.containsText(
pageObject.getDynamicElement("#topicText", i + 1),
name.trim(),
util.format(issueCats.WRONG_DATA)
);
}
I added the xpath check to validate if i'm waiting enough for that text to appear. I observed that visible assertion is getting passed but in next assertion the #topicText is coming as previous value or null.This is an intermittent issue but on test server happens frequently.
There is no magic bullet to brittle UI end to end tests. In the ideal world there would be an option set avoid_random_failures=true that would quickly and easily solve the problem, but for now it's only a dream.
Simple rewriting all tests in Java will not solve the problem, but if you feel better in java, then I would definitely go in that direction.
As you already know from this article Avoiding random failures in Selenium UI tests there are 3 commonly used avoidance techniques for race conditions in UI tests:
using constant sleep
using WebDriver's "implicit wait" parameter
using explicit waits (WebDriverWait + ExpectedConditions + FluentWait)
These techniques are also briefly mentioned on WebDriver: Advanced Usage, you can also read about them here: Tips to Avoid Brittle UI Tests
Methods 1 and 2 are generally not recommended, they have drawbaks, they can work well on simple HTML pages, but they are not 100% realiable on AJAX pages, and they slow down the tests. The best one is #3 - explicit waits.
In order to use technique #3 (explicit waits) You need to familiarize yourself and be comfortable with the following WebDriver tools (I point to theirs java versions, but they have their counterparts in other languages):
WebDriverWait class
ExpectedConditions class
FluentWait - used very rarely, but very usefull in some difficult cases
ExpectedConditions has many predefinied wait states, the most used (in my experience) is ExpectedConditions#elementToBeClickable which waits until an element is visible and enabled such that you can click it.
How to use it - an example: say you open a page with a form which contains several fields to which you want to enter data. Usually it is enought to wait until the first field appears on the page and it will be editable (clickable):
By field1 = By.xpath("//div//input[.......]");
By field2 = By.id("some_id");
By field3 = By.name("some_name");
By buttonOk = By.xpath("//input[ text() = 'OK' ]");
....
....
WebDriwerWait wait = new WebDriverWait( driver, 60 ); // wait max 60 seconds
// wait max 60 seconds until element is visible and enabled such that you can click it
// if you can click it, that means it is editable
wait.until( ExpectedConditions.elementToBeClickable( field1 ) ).sendKeys("some data" );
driver.findElement( field2 ).sendKeys( "other data" );
driver.findElement( field3 ).sendKeys( "name" );
....
wait.until( ExpectedConditions.elementToBeClickable( buttonOK)).click();
The above code waits until field1 becomes editable after the page is loaded and rendered - but no longer, exactly as long as it is neccesarry. If the element will not be visible and editable after 60 seconds, then test will fail with TimeoutException.
Usually it's only necessary to wait for the first field on the page, if it becomes active, then the others also will be.

webdriver-sync running asynchronously?

I'm trying to create selenium tests that run each step synchronously, without using .then(), or async/await. The reason for this is that I want to create a set of functions that allow pretty much anyone on our test team, almost regardless of tech skills to write easy to read automated tests. It looks to me like webdriver-sync should give me exactly what I want. However, the following dummy code is producing problems:
var wd = require('webdriver-sync');
var By = wd.By;
var Chromedriver = wd.Chromedriver;
var driver = new Chromedriver;
driver.get('https://my.test.url');
var myButton = driver.findElement(By.cssSelector('[id*=CLICK_ME]'));
myButton.click();
It tries to run - browser is launched, and page starts to load... but the steps are not executed synchronously - it goes on and tries to find and click "myButton" before the page has finished loading, throwing a "no such element" error... which to me kinda defeats the point of webdriver-sync?! Can someone tell me where I am going wrong?
FWIW, I have webdriver-sync 1.0.0, node v7.10.0, java 1.8.0_74, all running on CentOS 7.
Thanks in advance!
You need to put double-quotes around "CLICK_ME" as it's a string value.
Generally, though, it's a good idea to Wait for specific elements because dynamic pages are often "ready" before all their elements have been created.

Stop chrome extensions from injecting code into multipart response

Some extensions seem to want to modify the response of a request:
It changes the response from regular JSON:
{"fields":"A, B, C","success":true}
to this:
{"fields":"A, B, C","success":true}<script type="text/javascript">(function () {
return window.SIG_EXT = {};
})()</script>"
This specific one is by the HubSpot Sidekick extension - https://chrome.google.com/webstore/detail/sidekick-by-hubspot/oiiaigjnkhngdbnoookogelabohpglmd
Is there any way to prevent this, aside from uninstalling the extension?
I'm one of the engineers working on the Sidekick extension. I wanted to give you a heads up that I've looked into this problem and diagnosed it as a small error in our JS code that was causing certain sites which fetched JSON to end up with adulterated payloads. This error has been fixed in the latest version of Sidekick (v2.4.49) released this afternoon and should not appear again. My apologies for any inconvenience the problem may have caused in the meantime.

sharepoint: Using a Content Editor Web Part this error occurred:"Cannot retrieve properties at this time."

I have a content editor web part. Whenever I edit the content and then click save, the following errors occurred:
"Cannot retrieve properties at this time."
"Cannot save your changes"
How do you fix this?
I tried googling it.. there are some similar cases but not exactly the same. I tried this link:
www.experts-exchange.com/OS/Microsoft_Operating_Systems/Server/MS-SharePoint/Q_21975446.html
and this one:
support.microsoft.com/kb/830342
and this one:
blogs.msdn.com/gyorgyh/archive/2009/03/04/troubleshooting-web-part-property-load-errors.aspx
I found the answer!! apparently using mozilla firefox it worked. Then I found out that there is a javascript error in IE, this javascript error doesnt happened in firefox. how ironic!
Are you doing anything to modify the URL in an HTTPModule? I ran into this problem on a publishing site where a module was hiding the "/pages" part of the URL. Modifying the CEWP via the page when accessed w/o the "/Pages" wasn't working, but with the "/Pages" it was.
Example:
Got error: http://www.tempura.org/webpartpage.aspx
Worked: http://www.tempuri.org/pages/webpartpage.aspx
I don't see how this is an answer -- "don't use IE".
In my case (and apparently many others) it has something to do with ISA + SharePoint + host headers. I will post the fix if I find one.
I have had problems with this before and have found recycling the Application Pool often corrects the problem.
Rodney
IE8 -->
Tools --> Compatiblity View Settings --> CHECK THIS : Display All Websites in ....
If you are editing a webpart page, make sure that it is checked out. Sometimes the document library the webpart pages are in will have a "force check out to edit" option and it will give you errors if the webpage itself isn't checked out.
I had this same error recently. In javascript, I had written some prototype overrides (see examples below) to add some custom functions to the string and array objects. Both of these overrides interferred with SharePoint's native JavaScript somehow in IE. I removed the references from the master page and this issue was FIXED. Currently trying to find a work-around so I can keep them because things like the string.format function is very nice to have...
//Trim
if (typeof String.prototype.trim !== 'function') {
String.prototype.trim = function(){
return this.replace(/^\s+|\s+$/g, '');
}
}
//Format
String.format = function() {
var s = arguments[0];
for (var i = 0; i < arguments.length - 1; i++) {
var reg = new RegExp("\\{" + i + "\\}", "gm");
s = s.replace(reg, arguments[i + 1]);
}
return s;
}
I also faced the same problem. Finally it worked for me using url /Pages/Contact-Us.aspx instead of clean URL. It worked only with IE browser. Don't know why this was happening but anyhow it worked with me.
Use IE browser
Use Pages in the URLinstead of clean URL.
to me,
compatibility mode in IE8, to work

Resources