I have a set of Test Scripts which I need to test on a few different operating systems. I have connected the remote machines with these OS using Test agents to my local machine. I will run through Visual Studio by adding a test settings file.
Doing this, it distributes the Test Scripts set among the remote machine, but I want each of the remote machine to run each of the test. i.e. each script must execute on every remote system.
In the test mix of the scenario set the tests to run in Sequential order. This will make each virtual user run all of the tests.
Related
I'm new to gitlab and am asking for advice / best practice here.
I have a program that I build on my build machine. The program cant run on the build machine, as it needs to be installed to a test machine that has special hardware/enviornment that the program needs. I want to run some system tests (memory leak tests etc) on the test machine. How is this best done?
I think this can be accomplished with the "multi project pipeline" feature. Is this the simplest/best way?
Here is my plan:
I could have one (shh/shell) runner that build my program on my build machine, and a different runner that runs tests on my test machine. The two would be connected using "multi project pipeline" feature. The artifacts from the build pipeline would be installed on the test machine and then system tests would run on the test machine.
Is this the best way to solve this? Or is there a simpler/better way?
Answering my own question here. "Multi project pipeline" is not necessary here. You simply have a single project and mark jobs with different "tags". You can then register runners for these different tags, on different machines.
(Artifacts are transferred from one job to the next the same, regardless if the runners are run on the same machine or on different ones)
https://docs.gitlab.com/ee/ci/yaml/README.html#tags
I have multiple Linux GCE machines and there's a bash script I want to run on some of them from time to time. Right now I'm connecting manually to each machine separately in order to run the script.
How do I run the script on several machines concurrently without connecting to each machine individually each time? How do I configure on which machines it should run?
you can use csshx to ssh into multiple servers at once. Once logged in into servers, you can execute script as per your need. you can follow this link to install it on mac.
Other alternative could be that you schedule a cronjob at all servers so they will run at specific time. you can edit these multiple cronjobs using csshx.
Try fabric. You can write simple scripts in Python that you can run on multiple hosts (which you will need SSH access to).
I have Jenkins installed on a Linux server. It can run builds on itself. I want to create either a Freestyle Project or an External Job that transfers a bash script and runs it on two separate linux servers. Where in the GUI do I configure the destination server when I create a build? I have added "nodes" in the GUI. I can see the free space of the servers in the Jenkins GUI, so I know the credentials work. But when I create a build, I see no field that would tell Jenkins to push the bash scripts and run them on certain servers.
Are Jenkins nodes just servers that lend computing power to the master server? Or are they the targets of Jenkins builds? I believe that Jenkins "slaves" provide computing power to the Jenkins master server.
Normally Jenkins is used to integrate code. What do you call the servers that Jenkins pushes code into? They would be called Chef clients or Puppet agents if I was using Chef or Puppet for integrating code. I've been doing my own research, but I don't seem to know the specific vocabulary.
I've been working with such tools for several years. And for as far as I know there isn't a Ubiquitous Language for this.
The node's you can configure in Jenkins itself to add 'computing power' are indeed called build slaves.
Usually, external machines you will copy to, deploy to or otherwise use in jobs are called "target machine". As it will be the target of an action in your job.
Nodes can be used in several forms, you can use agents, which will require a small installation on the node machine. Which will create a running agent service with which Jenkins can communicate.
Another way is simply allow Jenkins to connect to a machine via ssh and let it execute commands there. Both are called nodes and could be called build slaves. But the first are usually dedicated nodes while the second can be any kind of machine as long as the ssh user can execute the build.
I also have not found any different terms for these two types.
It's probably not a real answer to your questions, but I do hope it helped.
I am developing a distributed program that will run on Amazon EC2 machines.
Ideally I would develop on my local machine, trigger a script to deploy the source on the remote machines (All Linux machines on which I have ssh access), trigger a compile command on each of the remote machines and then run the program on each instance while having control over the running programs (being able to suspend them).
I am wondering if there exist already any tools for such a task (except using MPI, but this is for another question), and if not, what best practices should I follow.
There are many alternatives depending on your scale of deployment. An interesting one that I have not tried personally is glu. There are also the well known puppet, chef family of config management tools that have some process control components.
Can't you write Perl?
Net::OpenSSH::Parallel allows to write scripts that run commands in several servers in parallel via SSH quite easily:
#!/usr/bin/perl
use Net::OpenSSH::Parallel;
my #hosts = (...);
my $pssh = Net::OpenSSH::Parallel->new;
$pssh->add_host($_) for #hosts;
$pssh->all(rsync_put => '/local/path', '/server/path');
$pssh->all(cmd => 'cd /server/path && make');
$pssh->all(join => '*'); # waits for all the servers to reach this point.
$pssh->all(cmd => 'cd /server/path && ./your_program');
$pssh->run;
I'm using the yii framework and trying to get its unit tests running while connected over ssh on a CentOS server. When I run phpunit, it tries to launch Firefox, which fails with the error "no display specifiied"
General theory
Error: no display specified
To understand that error message you first have to understand how the X Windowing System works - that is the name of the framework used by Linux (and other types of Unix) systems used to display graphical user interfaces.
X consists of two parts - there is a client and a server. Client is the program that wants to draw the interface - in your case that would be Firefox. Server is a program that makes drawing possible. There are X servers available for all the major operating systems. Linuxes and OSX usually ship with one, on Windows you will have to find and install one - Cygwin/X is one option, but there are others.
So why is this client/server architecture even necessary?
Most of the time its not even needed. If you happen to run Linux locally, then you will not even notice that there is any kind of client/server communication happening somewhere - but there is.
Where X shines though is that this architecture means that network capabilities are built right into it. You can run a client (Firefox) on one machine and display the GUI on a completely different machine. You could run 10 different clients on 10 different machines and have them all display output on a single machine thanks to X. Think VNC or Remote Desktop - X is somewhat similar, but you could say that its on steroids compared to those. And X has had this ability for a really long time.
Whenever your start up a X client (a program that wants to display graphical user interface) it looks for an X server. One possibility for the client to find one is an environment variable named DISPLAY. I'm on OSX and this is what I see.
[~]> echo $DISPLAY
/tmp/launch-ihNtDq/org.x:0
This point to my local X server. It could point to any server on my local network. When the client finds this environment variable it will connect to it and the user interface pops up.
If client cannot find this environment variable - you will get the familiar
Error: no display specified
Back to Yii
Looks like Yii has Selenium tests bundled. PHPUnit needs to start up Selenium RC to control a Firefox instance to run those tests. Selenium RC (or maybe Firefox itself) fails to find DISPLAY environment variable. And dies with the above error.
How do you solve this issue?
There are 3 options
1) install Yii, PHPUnit and all their dependencies locally. Selenium runs just fine on Windows. It wont use the X protocol on Windows, so none of that business with X clients and X servers. And you can then run Yii testsuite locally.
2) install an X server on your Windows box. Then enable 'X Forwarding' in your ssh client settings (or use the -X command line parameter for ssh). When you do that, then there will be a DISPLAY variable set when you are logged in to that CentOS server. You can verify it by typing the echo command above. Then the X client on CentOS can talk (show GUI) to the X server on your Windows machine - all the X traffic is tunneled over the ssh connection. This however means that you need java (which selenium RC is built in) and Firefox on that CentOS server. You may or may not have them there.
3) use a virtual framebuffer - for example Xvfb - an X server that performs all drawing operations in memory, not showing any output anywhere.
What good is that? Selenium has commands for taking screenshots at any point during the test run and saving them to files. For example a typical Selenium test would check whether an element exist on the page - and to make a screenshot when it does not. Screenshot would then be saved in a file, which you can view later to determine what the reason for the failure was. Making screenshots works just fine with a virtual framebuffer.
Final clarification
Note that Selenium tests are merely one type of tests that PHPUnit can run. Selenium in not required to write PHPUnit tests, its an optional add-on. But Yii testsuite apparently relies upon it.
Last, but not least
Integration tests (which Selenium tests are) are not usually ran on production systems, because there is a chance that test data is left behind in your production database. Also, getting good test results means isolating from external factors as much as possible - the contents of your production database will be constantly changing and this may affect your tests.
Normally all tests will be executed somewhere else (your development machine, dedicated QA server, whatever you have), before the new code is deployed to production servers. After all the point of tests is to verify that the system works after the changes. There is not much value in running them on production systems - the code does not change after it has been deployed.
Of course - its up to you - if you see value in doing those tests on production system, go right ahead.
Its simpler than you think. Run Selenium locally on your desktop, make sure phpunit is setup on the remote server. Then start a reverse SSH tunnel in your SSH connection. This varies depending on your SSH client. In PuTTY, there is a setting for SSH tunnels and you can reverse the direction by selecting the remote option. Check out this page for details. With OpenSSH from the command line, its done like this:
ssh -R 4444:localhost:4444 user#remoteserver
This will listen on the remote server on port 4444 and forward it to your selenium server running on localhost on your desktop port 4444.
Once you've done that, you'll need to change the TEST_BASE_URL setting in yourproject/protected/tests/WebTestCase.php to go to the remote server's URL for your yii project.
The simplest way to run a gui test on from another agent machine on a Windows client is to use "psexec" (http://technet.microsoft.com/en-us/sysinternals/bb897553.aspx).