I am trying to run webdriverio with PhantomJS/Chrome to load a page and then grab the window object for use with other scripts. For some reason I am unable to get the window object. Everytime I get, I end up seeing output like this:
Title is: XXXXX
{ state: 'pending' }
Using the following script:
var webdriverio = require('webdriverio');
var options = {
desiredCapabilities: {
browserName: 'chrome',
logLevel: 'verbose'
}
};
var client = webdriverio.remote(options);
client
.init()
.url('https://xxxx.com')
.waitUntil(function () {
return client.execute(function () {
return Date.now() - window.performance.timing.loadEventEnd > 40000;
}).then(function (result) {
console.log(window);
return window;
});
})
.end();
Does anyone know how I can fix my code so that the window object is returned to my NodeJS console app after the page is completely loaded?
Thanks!
Window is an object in the browser's DOM, so it's only available inside of the 'execute' function. If you wanted access to it, you could return it from your 'execute' function:
return client.execute(function () {
return window;
}).then(function (result) {
console.log(result);
});
This work as well:
browser.execute('return window');
Related
I am migrating an Angular 6 project to Angular Universal project(SSR). In my project, I need to access window object in order to get "navigator", "location", etc... I am using the following method to polyfill window in Server side. But it is not working.
try {
console.log(window);
} catch (window) {
console.log('Server side window.');
window = {
location: {
replace: () => {},
protocol: 'https'
},
navigator: {
userAgent: '',
appVersion: ''
},
scrollTo: () => {},
open: () => {},
localStorage: {}
};
console.log('Server side window.', window);
}
What is the best way to handle the window object properly on the server side for the server-side rendering web site? I got following confusion errors. It said 'window is not defined' or 'has been declared'. How to use window object in NodeJS?
> console.log(window);
ReferenceError: window is not defined
> let window = 1
SyntaxError: Identifier 'window' has already been declared
The problem with this piece of code is that window is specified as exception identifier in try..catch, basically:
try {
console.log(window);
} catch (window) {
// window instanceof ReferenceError
window = {...};
// window is redefined in this block scope
}
This will work in loose mode:
try {
console.log(window);
} catch (err) {
window = {...};
}
But will result in another ReferenceError in strict mode. It should be instead:
try {
console.log(window);
} catch (err) {
global.window = {...};
}
And a proper way to detect window is:
if (typeof window === 'undefined') {
global.window = {...};
}
This code is limited to browser and Node.js; it will result in an error in a worker.
I'm basing this off of the Page Loading section of http://phantomjs.org/quick-start.html
I want to do something like this:
tests.js
var should = require('chai').should();
var page = require('webpage').create();
describe('test website with phantomJS', function() {
it('should load html from page', function() {
page.open('myHomePageToTest.html', function(status) {
if (status === 'success') {
page.content.should.equal('<!DOCTYPE html>...etc...</html>');
}
});
});
});
If I try to run this with 'mocha-phantomjs test.js' I get the error 'Failed to start mocha: Init timeout'
If I try to run this with 'mocha test.js' I get the error 'Cannot find module "webpage"'
I'm sure those are the expected error messages given the code. It's my understanding that is failing. The code is my description of what I want to do. After several hours of treading water last night, I have no idea how to actually do it.
Thank you for any help or nudge in the right direction.
var assert = require('assert');
var phantom = require('phantom');
describe('Mocha and phantom', function () {
this.timeout(150000);
it('Tweeking with phantomjs', function (done) {
phantom.create(function (ph) {
ph.createPage(function (page) {
page.open('https://www.facebook.com/', function (status) {
page.evaluate(function () {
return document.all[0].outerHTML //can check different elements
}, function (result) {
console.log('----------->>>>result',result);
assert.equal(status,'success','Not appropriate status');
done();
})
})
})
})
})
})
I have an weird error and can't find the cause of it for the last few hours...
I have a meteor app, that scrapes some webpages for information and everything works fine as long as I use reuqest and cheerio for static pages, but now I have a dynamic site and I wanted to use phantomjs, casperjs and spookyjs for this one, but here I get some bug...
My code is as follows, I import the npm modules at the start:
if (Meteor.isServer) {
var cheerio = Meteor.npmRequire('cheerio');
var request = Meteor.npmRequire('request');
var phantomJS = Meteor.npmRequire('phantomjs');
var spooky = Meteor.npmRequire('spooky');
And sometime later I want to use spooky to scrape some webpage:
spooky.start("https://www.coursera.org/");
spooky.then( function () {
this.fill("form", {email: user, password: pass}, true);
});`
But as soon as I call the method I get the following error message:
20150224-21:16:39.100(-5)? Exception while invoking method 'getLecturesCoursera' TypeError: Object function Spooky(options, callback) {
....
I20150224-21:16:39.281(-5)? } has no method 'start'
I20150224-21:16:39.281(-5)? at [object Object].Meteor.methods.getLecturesCoursera (app/moocis.js:72:14)
I am doing something completly wrong and I have no clue why it isn't working...
I tried to verify that spookyjs and phantomjs are installed correctly in my app, but that isn't as easy as it sounds for someone who uses them for the first time...
Like normally with spooky you have to create a new Spooky Object before you can start and run it...
if (Meteor.isServer) {
Meteor.startup( function () {
var Spooky = Meteor.npmRequire('spooky');
var spooky = new Spooky({
child: {
transport: 'http'
},
casper: {
logLevel: 'debug',
verbose: true,
ignoreSslErrors: true,
sslProtocol: 'tlsv1'
}
}, function (err) {
if (err) {
e = new Error('Failed to initialize SpookyJS');
e.details = err;
throw e;
}
spooky.start(
'https://www.google.com');
spooky.then(function () {
this.emit('hello', 'Hello, from ' + this.evaluate(function () {
return document.title;
}));
});
spooky.run();
});
spooky.on('error', function (e, stack) {
console.error(e);
if (stack) {
console.log(stack);
}
});
spooky.on('hello', function (greeting) {
console.log(greeting);
});
spooky.on('log', function (log) {
if (log.space === 'remote') {
console.log(log.message.replace(/ \- .*/, ''));
}
});
})
}
I'm following the tutorial listed here :
http://code.tutsplus.com/tutorials/screen-scraping-with-nodejs--net-25560
When I run the code:
var host = 'http://www.shoutcast.com/?action=sub&cat=Hindi#134';
var phantom = require('phantom');
phantom.create(function(ph) {
return ph.createPage(function(page) {
return page.open(host, function(status) {
console.log("opened site? ", status);
page.injectJs('http://ajax.googleapis.com/ajax/libs/jquery/1.11.0/jquery.min.js', function() {
//jQuery Loaded.
//Wait for a bit for AJAX content to load on the page. Here, we are waiting 5 seconds.
setTimeout(function() {
return page.evaluate(function() {
//Get what you want from the page using jQuery. A good way is to populate an object with all the jQuery commands that you need and then return the object.
console.log(document.getElementsByClassName('transition')[0]);
return document.getElementsByClassName('transition')[0];
}, function(result) {
console.log(result);
ph.exit();
});
}, 5000);
});
});
});
});
I get the following error :
phantom stdout: ReferenceError: Can't find variable: $
phantom stdout: phantomjs://webpage.evaluate():7
phantomjs://webpage.evaluate():10
phantomjs://webpage.evaluate():10
I have no idea what this means and there's no help on how to resolve it ...
How can this be solved ?
Basically I want all the 'a' tags with class transition from the site I'm scraping. All these tags are loaded asynchronously on the site.
The $ is due to jQuery and possible conflicts. You hardly require to inject jQuery just to scrape 'a' tags with class transition. You always have document.querySelector or document.querySelectorAll.
var host = 'http://www.shoutcast.com/?action=sub&cat=Hindi#134';
var phantom = require('phantom');
phantom.create(function(ph) {
ph.createPage(function(page) {
page.open(host, function(status) {
console.log("opened site? ", status);
//Wait for a bit for AJAX content to load on the page. Here, we are waiting 5 seconds.
setTimeout(function() {
page.evaluate(function() {
// here you need to add more code to get the html/text
// more code incase you use querySelectorAll
return document.document.querySelector('a.transition');
//return document.document.querySelectorAll('a.transition');
},
function(result) {
console.log(result);
ph.exit();
});
}, 5000);
});
});
});
However, I am not able to understand the way function (result) { console.log(result); ...} is coded. I am not aware if page.evaluate takes callback function as second parameter. Please check that with the documentation.
I've built a basic demo application using AppJS/NodeJS and edge.
Heres the relevant part of the the app.js config file - Basically it just references the external modules.
.......
window.on('ready', function(){
console.log("Window Ready");
window.process = process;
window.module = module;
window.dns = require('native-dns');
window.edge = require('edge');
window.fs = require('fs');
window.frame.openDevTools();
..........
And heres the relevant javascript part of the main index.html page:
.......
function myFunction1()
{
var question = dns.Question({
name: 'www.google.com',
type: 'A'
});
var req = dns.Request({
question: question,
server: { address: '8.8.8.8', port: 53, type: 'udp' },
timeout: 1000
});
req.on('timeout', function () {
console.log('Timeout in making request');
});
req.on('message', function (err, answer) {
answer.answer.forEach(function (a) {
console.log(a.address);
});
});
req.on('end', function () {
console.log('Finished processing request');
});
req.send();
}
function myFunction2()
{
var helloWorld = edge.func('async (input) => { return ".NET Welcomes " + input.ToString(); }');
helloWorld('JavaScript', function (error, result) {
if (error) throw error;
console.log(result);
});
}
..........
If I call myFunction1() which uses another nodejs module (DNS lookup) it works perfectly. However if I call myFunction2() which uses edge I get the following error!
Uncaught TypeError: Property 'func' of object [object Object] is not a function
I've spent hours on this and for cannot work out why this happening!
Have you tried running the same myFunction2 code inside app.js i.e. in nodejs? Maybe the func function does not exist on the edge object. Check the docs maybe you need to do something like
window.edge = require('edge').Edge;
or something similar to get hold of the object that you think you have at the moment. You can also do console.log(window.edge) and see what it outputs (both in node and in browser running dev tools (F12)).
/Simon