When running Intern functional tests in the browser (not headless), I notice that a bunch of different things happen before the tests are actually run. Some things that happen are navigation to different pages, a dropdown with "foo" and "bar", and several other things. Once these finish, Intern then navigates to my webpage and runs the functional tests.
My question is, why does all this happen before the actual functional tests are run?
The functionality of a remote browser can be affected by the version of Selenium, the particular WebDriver server (e.g. chromedriver), the host OS, and proxies or other services being run on the remote host. Intern performs feature tests for the browser to determine what flavor of the WebDriver protocol it supports and what features (like mouse support) may be broken or have non-standard behaviors.
These tests are most useful for older browsers or web driver servers, and can often be disabled when current versions of everything are being used. You can disable these tests by specifying "fixSessionCapabilities": false or "fixSessionCapabilities": "no-detect" in your environment specifier.
Related
I have a HapiJS server running with a variety of plugins and modules used throughout. Sometimes API calls take longer than I would expect and I'm looking to instrument the code to be able to get a timing breakdown of what downstream service/call is taking up all the time.
What options are there for doing this in an on-demand fashion? Something similar to this mojito-waterfall?
To generally inspect network traffic on node.js you can use node-inspector. Just install it then run your application with node-debug and connect to the debugger with Google Chrome (not sure if it works with other browsers because it uses Chrome's developer tools). You can then open the network tab and see all network connections just like you would debugging your browser code.
Note that some versions of node.js may not work with node-inspector. The latest version that works that I know of to date is 6.3.1. Downgrade/upgrade node.js as necessary.
I was wondering if anyone else knows a good way to start individual browser stack tests sequentially using Capybara/Browserstack/Cucumber.
I'm having issues with using Capybara in the sense that browserstack doesn't get updated with my new capabilities for every run, even when I shut down my browser, i.e: The two test runs are started sequentually in Browserstack, but with the same browser and OS-settings.
Abstract Scenario: Run login tests
Given that I want to test x website with capabilities og
Examples:
|browser|browser_version| os |os_version|resolution|
|IE| 11.0 | Windows |8.1 |1024x768 |
|Firefox| 45.0 | Windows |10 |1024x768 |
I've checked that every value successfully gets sent through to the next step, but it seems like Browserstack doesn't update its new capabilities that I'm trying to set.
I know I can probably manage to do parallell runs setting capabilities through settings instead, but we have a limit to how many parallell runs using Browserstack's license. That's why I want to run them sequantually and figured this could be a way to do it.
As per my experience, BrowserStack initiates a test on a particular OS/browser capability that it receives from your tests. Thus, it seems your setup is sending the same capability for both the runs of the test.
I believe you want to run tests sequentially and on different OS/browser combinations. In that case you can refer to the BrowserStack's documentation for configuring Parallel Cucumber tests using Rake file in the "Parallel tests" section. After creating all the files, you can run the following command to run tests sequentially:
rake BS_USERNAME=<username> BS_AUTHKEY=<access_key> nodes=1
Well, we all know about headless servers. Actually, probably the vast majority of servers out there are headless.
As usual (it seems), my situation asked for quite something else. Basically, the proposed architecture looks more or less like:
The app server (node.js) is situated on a physical machine physically connected to two screens.
Between this machine and the 'net there are all sorts of regular networking layers. Please keep in mind that one of the main reasons for this setup is physical portability: ie, the client gets the necessary hardware as the product. The server itself relies on CDN for static files etc.
Each monitor/screen needs to show something different, produced by the same node server.
For now this server will probably run on Windows, but given a concept (which is what my question is after), I can change the code to run on the target platform. Well, depending on my code, this could even be done automatically.
So, my actual question. Node is quite flexible in that it can be run by anything - even custom made software (C++, Delphi, even GM). Just shell_exec('node server.js') and we're off.
But the screens themselves need to be quite dynamic. So node needs to influence both screens in some way. A few options I'm considering:
A custom app which creates two resizable, featureless windows with an embedded chromium browser to be controlled by the node server somehow (how node react with these browsers?)
A custom app which, according to node CLI output, updates the two screens' UI. Since I need something flashy as the UI, this app would be created in something like GameMaker, or a similar engine.
PS: Just in case you're asking; the physical connection opposed to a network one (eg; web-based GUI frontend) is by design.
I'd just wire up the result/monitoring screens as regular HTML pages. In your Node app, create a second HTTP server (on a non-standard port, firewalled from the public) that serves up the monitoring page.
Use socket.io to to send the realtime data to the monitoring page, which can make everything look pretty. Fire it up in a full-screen instance of Chrome.
This approach completely frees you from any kind of platform dependency, and decouples the monitoring app from the server app. It leaves you the latitude to run the monitoring app on a separate box if necessary.
I need to handle authenticated multiple users running parallel tests on the selenium standalone server, and discovered two webdriver clients on nodejs. There's webdriver-js and wd-js. Which is more active and reliable? Any experiences? I'm a bit concerned about them breaking down when node or selenium updates or removes features.
I don't think any of those packages mention automatically starting Xvfb on a unique display number per test. So start shell commands to run xvfb before driving the browser?
The following process is what I am trying to build in nodejs (it's essentially like Grid 2 but on nodejs purpose of continuous integration of tests running) and looking for any packages or suggestions for any of the following part.
First authenticate the user(s) using a persistent bi-directional connection (WebSockets or HTTP 1.1)
Start/queue tests requested to run by the user on available hardware nodes (I will add more linux boxes so need a package to distribute parallel tests across the "grid")
Monitor the running selenium browser tests and send client status updates (ex) running/stop)
Tests submitted by the users need to be persistent and accessible for future or continuous integration (couchdb or mysql)
Scheduling of jobs to be run on a continuous basis (ex. run every set interval of time).
Is nodejs a bit overkill? should I focus on Java only for the backside?
https://github.com/LearnBoost/soda
This is for vanilla Sauce Labs/Selenium RC integration. I'd imagine when you're running in a browser instance like Selenium RC, websockets should just work, as the javascript on the page is executed. If you're authenticating a user, you want to just fill out whatever form and submit (which triggers your WS auth) as normal.
I don't think nodejs is overkill for this. Node is lightweight. I don't know that I'd add node to my stack ONLY for this, but its certainly convenient and if you have a commitment to javascript, its no big deal.
I'm using the yii framework and trying to get its unit tests running while connected over ssh on a CentOS server. When I run phpunit, it tries to launch Firefox, which fails with the error "no display specifiied"
General theory
Error: no display specified
To understand that error message you first have to understand how the X Windowing System works - that is the name of the framework used by Linux (and other types of Unix) systems used to display graphical user interfaces.
X consists of two parts - there is a client and a server. Client is the program that wants to draw the interface - in your case that would be Firefox. Server is a program that makes drawing possible. There are X servers available for all the major operating systems. Linuxes and OSX usually ship with one, on Windows you will have to find and install one - Cygwin/X is one option, but there are others.
So why is this client/server architecture even necessary?
Most of the time its not even needed. If you happen to run Linux locally, then you will not even notice that there is any kind of client/server communication happening somewhere - but there is.
Where X shines though is that this architecture means that network capabilities are built right into it. You can run a client (Firefox) on one machine and display the GUI on a completely different machine. You could run 10 different clients on 10 different machines and have them all display output on a single machine thanks to X. Think VNC or Remote Desktop - X is somewhat similar, but you could say that its on steroids compared to those. And X has had this ability for a really long time.
Whenever your start up a X client (a program that wants to display graphical user interface) it looks for an X server. One possibility for the client to find one is an environment variable named DISPLAY. I'm on OSX and this is what I see.
[~]> echo $DISPLAY
/tmp/launch-ihNtDq/org.x:0
This point to my local X server. It could point to any server on my local network. When the client finds this environment variable it will connect to it and the user interface pops up.
If client cannot find this environment variable - you will get the familiar
Error: no display specified
Back to Yii
Looks like Yii has Selenium tests bundled. PHPUnit needs to start up Selenium RC to control a Firefox instance to run those tests. Selenium RC (or maybe Firefox itself) fails to find DISPLAY environment variable. And dies with the above error.
How do you solve this issue?
There are 3 options
1) install Yii, PHPUnit and all their dependencies locally. Selenium runs just fine on Windows. It wont use the X protocol on Windows, so none of that business with X clients and X servers. And you can then run Yii testsuite locally.
2) install an X server on your Windows box. Then enable 'X Forwarding' in your ssh client settings (or use the -X command line parameter for ssh). When you do that, then there will be a DISPLAY variable set when you are logged in to that CentOS server. You can verify it by typing the echo command above. Then the X client on CentOS can talk (show GUI) to the X server on your Windows machine - all the X traffic is tunneled over the ssh connection. This however means that you need java (which selenium RC is built in) and Firefox on that CentOS server. You may or may not have them there.
3) use a virtual framebuffer - for example Xvfb - an X server that performs all drawing operations in memory, not showing any output anywhere.
What good is that? Selenium has commands for taking screenshots at any point during the test run and saving them to files. For example a typical Selenium test would check whether an element exist on the page - and to make a screenshot when it does not. Screenshot would then be saved in a file, which you can view later to determine what the reason for the failure was. Making screenshots works just fine with a virtual framebuffer.
Final clarification
Note that Selenium tests are merely one type of tests that PHPUnit can run. Selenium in not required to write PHPUnit tests, its an optional add-on. But Yii testsuite apparently relies upon it.
Last, but not least
Integration tests (which Selenium tests are) are not usually ran on production systems, because there is a chance that test data is left behind in your production database. Also, getting good test results means isolating from external factors as much as possible - the contents of your production database will be constantly changing and this may affect your tests.
Normally all tests will be executed somewhere else (your development machine, dedicated QA server, whatever you have), before the new code is deployed to production servers. After all the point of tests is to verify that the system works after the changes. There is not much value in running them on production systems - the code does not change after it has been deployed.
Of course - its up to you - if you see value in doing those tests on production system, go right ahead.
Its simpler than you think. Run Selenium locally on your desktop, make sure phpunit is setup on the remote server. Then start a reverse SSH tunnel in your SSH connection. This varies depending on your SSH client. In PuTTY, there is a setting for SSH tunnels and you can reverse the direction by selecting the remote option. Check out this page for details. With OpenSSH from the command line, its done like this:
ssh -R 4444:localhost:4444 user#remoteserver
This will listen on the remote server on port 4444 and forward it to your selenium server running on localhost on your desktop port 4444.
Once you've done that, you'll need to change the TEST_BASE_URL setting in yourproject/protected/tests/WebTestCase.php to go to the remote server's URL for your yii project.
The simplest way to run a gui test on from another agent machine on a Windows client is to use "psexec" (http://technet.microsoft.com/en-us/sysinternals/bb897553.aspx).