I need to run Selenium (or another webscraping tool that can handle javascript) on a remote linux host (Webfaction). I am using Python.
Is this possible? The server is gui-less so I can't run browsers. Or can I, if I use PyVirtualDisplay?
What about running Selenium with HtmlUnit?
I have tried using Selenium with Selenium/PyVirtualDisplay/ChromeDriver, but keep getting various error messages. So I'm wondering if this is even possible before I continue to debug something impossible.
If you need to handle JavaScript Selenium/Webdriver seems to be a good solution.
If you need to run headless, GhostDriver (instead of ChromeDriver) is an excellent alternative. It is based on PhantomJS, a headless browser based itself on Webkit. It has full JS-support.
Related
I couldn't find it referenced in the documentation but since PhantomJs is now at 2.1, Is there a way to use a WebSocket to interface NodeJs and PhantomJs directly or am I better going with StdIn/StdOut?
I'd like to skip the step of spawning a webserver plus a fake webpage to catch the events like I've seen on many github. If I can I would avoid those ninja tricks.
You really don't need WebSockets or the Web Server module.
There are basically two ways to use PhantomJS from node.js:
Write a plain PhantomJS script and spawn a PhantomJS process that executes that script, but this doesn't provide easy interactivity
Interface with PhantomJS through the many available bridges so that you can mix PhantomJS code with node.js code:
phantom, node-phantom, nightmare (version <1.8 used PhantomJS, now it's Electron), Horseman, navit, phridge, node-phantom-simple, jquery.go.js, SpookyJS (wrapper around CasperJS, which is a wrapper around PhantomJS), Selenium-Webdriver (JavaScript bindings), webdriver.io (other JS bindings for Selenium), etc.
I would like to get the list of all webbrowsers installed using nodejs.
If possible on different OS like Windows,OSX and Linux.
For example, if Chrome is installed, I want to run it otherwise run Firefox or IE or Safari depending on the OS.
I can't find any module or internal function to get the webbrowsers.
Thanks in advance for your help.
This isn't possible because you are then saying you would be controlling a user's computer and what software it runs. If you are able to do that, then anyone could.
This assumes you are talking about general usage and not those in a closed and controlled environment, such as a group of users inside a company.
browser-launcher enable you to Detect, launch and stop browsers cross platform
I'm planing to make a web app which will allow you to have a Linux Terminal on a web page so that you can execute any command an get the response as if you were in front of your linux terminal.
I planed to use NodeJS as it is server side JavaScript, asynchronous and fast.
Also I saw this wich does exactly what i'm trying to do, I peeked in the source code, but didn't found something useful, I also analysed it with google chrome developer tools on the network tab, but there is absolutely nothing even while executing some commands and getting responses. How is this possible ? what technology do you think they used ?
So I wanted to get your advice, your experience in order to start it the right way.
I firstly decided to use NodeJS, but if there is another programming language or Framework more appropriate for this kind of application please let me know.
If you want a real terminal in the browser using node.js on the backend, you might give tty.js a try.
Alternatively you can use the pty.js module manually which is used by tty.js. Along with that, you could also use xterm for doing the browser-side terminal emulation.
I've been using Zombie.js with my cucumber-js scripts but having a bit of trouble and can't get my issues resolved as of yet. It's extremely late in my game to switch to a different headless browser, but, if I can't solve my issues in zombie, I might need to make the jump.
However, I can't seem to figure out if I can integrate it into my cucumber-js scripts the way I integrated zombie. PhantomJS seems to be a standalone thing, but I need it to run as a required module.
I've scanned the documentation and can't figure out if I can do this.
Is this possible?
Yes. We created an open source package that allows you to do exactly this, it runs Phantom in GhostDriver mode so it runs without selenium.
https://github.com/xolvio/chimp
You also get synchronous javascript for Webdriver (no callbacks, no promises) ;)
I've used Python webbrowser module and I would love something equivalent in Erlang. What I am trying to do is open a browser window/tab from Erlang.
I can't find anything in the official Erlang documentation. Is there such a thing?
I assume you mean the Python webbrowser module, not webserver. The code for this module is instructive; it looks for various browsers and picks the best option. It interacts with the browser by running a shell command. Firefox supports a -remote command line argument to control an existing Firefox process. (IE seems to be handled simply by passing the url to the iexplore.exe)
The module also detects if it's running on Windows and tries to find out the Windows default browser and use that. Likewise, on Mac OS X, it uses the open command, which will use the default browser.
You should be able to implement this with open_port/2.