I have the following setup:
Windows Server 2016 VM in Azure with Jenkins and .NET.
The nighlty job fetches the repo and runs the automated E2E tests using Specflow and xUnit. The web-related steps run silently (i.e. without displaying the browser window). How can I configure the job to display the browser during test run?
P.S. When I run the same suite using xunit console runner the window gets displayed.
Related
I have a suite of working Coded UI tests, written in C#, all marked as [TestMethod], written in VS 2013 and running on a set of VMWare machines, using the MS Test Agent (version 12.0).
I am building a release pipeline in Azure DevOps and use the following tasks:
Visual Studio Test Platform Installer (selected version 15.9.2)
VsTest - Test Assemblies (selected Visual Studio 2015 as this is the lowest version)
All of the test DLLs are at a specific folder on the VM I'm running and they are all copied from the deployment folder of an actual working VMWare machine. The test directories of the tasks are pointing to that folder.
I tried running .orderedtests file in the test assemblies section of the VsTest, but got an error that no tests were found.
I then switched to a *.dll so I can just see some tests recognized and running.
At this point I just want to see tests running. I don't care for the results.
In this scenario - letting the VsTest task discover the available tests, I get the following error:
##[error]The slice of type 'Discovery' is 'Aborted' because of the error : System.Exception: No test is available in [some
folder]\Some-File.dll. Make sure that test discoverer & executors are
registered and platform & framework version settings are appropriate
and try again.
What am I missing and what is needed to be able to run those 2013 (12.0) tests, which are working perfectly on local VMs, on the Azure DevOps pipeline?
Thanks!
The slice of type 'Discovery' is 'Aborted' because of the error : System.Exception: No test is available in [some folder]\Some-File.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
Based on the error message, you can try to can specify the appropriate framework and platform in the vstest task.
For example:
/Platform: x64 /Framework: framework40
For more info, you could refer to this doc: VSTest.Console.exe command-line options
On the other hand, since you can run the test on your local machine successfully, you can also try to use the self-hosted agent.
I'm setting azure pipeline release by selenium webdriver continous integration. I have added "Visual studio test platform installer" and "vstest- TestAssemblies" in release to get test result. But I'm getting this warning while "VStest-TestAsseblies run".
and after succesfull deployment I get this result in Test Tab
"No test runs are available for this build. Run tests and get rich
reports by using an appropriate built-in task such as the Visual
Studio Test task. Alternately, run tests using a runner of your choice
such as JUnit, xUnit, mocha, jest, pytest, rspec, etc. and use the
Publish Test Results task to get rich reports."
Even I have one test case which work fine on visual code locally.
I'd like to be able to run my Coded-UI tests on a headless browser. Is there some way for me to hide the IE browser instance when executing my tests?
You are asking two questions.
Can you run Coded UI tests on a headless browser?
Can you hide the IE browser instance when executing tests?
Coded UI tests can be run on a Virtual Machine (VM) (or even a physical computer) where no display is connected. To do this both the application under test and the Visual Studio that runs the tests must be running on that VM. The application under test could be a browser. Visual Studio also has an "agent" software that can run a test under the control of Visual Studio, or MTM or programs like mstest.exe.
Coded UI runs tests by running the application under test. If that application is a browser then that browser must be running and must be displaying its output on a screen. The screen may on a remote physical or virtual computer and does not need to be shown on any real screen. It is common to have a remote desktop to access the computer running the tests, but to have that desktop not in focus while the tests run. If the remote desktop is minimised or if it starts running a screensaver then the test will fail.
i have TFS express 2015 running on a local machine, i'm trying to get it to run a local process (a calculator window) as a second step after building.
i've tried command line, tried batch script and even tried powershell. the second step seems to go through just fine but no calculator is beeing opened.
is there anything i'm missing here? the TFS Express is installed on the same machine that runs visual studio.
please help!
For XAML build
If you want to open the local process (for example notepad） during Team Build, you need set your Build Service running as Interactively, launch your TFS Admin Console>>Build Configuration>>Build Service Properties>>Stop the service>>selected Run the Service Interactively.
Then you need to customize your build process template to create a InvokeProcess activity. You can use the InvokeProcess activity to start a process (run a program) on the build server. This activity is essentially a wrapper over Start. In your InvokeProcess activity, set FileName = “C:\Windows\notepad”.
After above two steps, you can open the notepad during Team Build. When running your build definition, the notepad will be opened on your build agent machine, and your build keeps on running this open notepad activity. You need logon your build agent using your TFS Build Service account, then you will see the notepad be opened on build agent machine, after you close this notepad, your build will continue run next steps.
I am having difficultly trying to come up with a working automated build and automated testing solution for testing several applications remotely.
I have several different applications which install onto Windows test machine VMs (Win 7) and an automated test solution which is built using a test settings file and test case filter and run automated tests against this application.
I have two groups of VMs, one group to build the automated tests solution which then sends the tests remotely to the second group which has the application installed on.
Currently this is done by a default template build definition (VS 2012) which builds the test solution on a build VM and sends the tests remotely to a test VM (has application needed manually installed prior to starting build), defining a test controller in the test settings file (there is a test settings file per test VM / build definition) the tests are sent to that test controller. Each test VM has a test controller and test agent on the same VM to prevent the tests going to multiple machines, the tests need to be sent to one machine with that specific application installed.
I am wanting to scale this process and allow for complete automation, so I can just kick off a specific application build which will install the required application on a free test VM, build the test solution code on a build VM and send the tests remotely to the test VM which now has the application needed installed on it, run the tests and send the results back.
I am having issues with doing this between the build and test VMs and having the test settings file on the build machine updated with the test controller name of the free test VM.
Is something like this possible to do and if so what would be the best way to go about this?
Any help or feedback would be greatly appreciated.
Setup:-
TFS / Visual Studio 2012 and Windows 7 build / test machines
Reference links:-
How can TFS build process be configured to execute tests on Test Agents through a Test Controller?
Ok, so you need to look at Lab Management for configuring environments. TFS has a built in method of saying that a set of machines go together and that you want to run tests against it. You can then use MTM to push groups of automated tests to environments and record the data.
In addition you should look at Release Management for VS 2013 (works with 2012) for pushing out the bits to your environments.
With those two tools you can maintain release pipelines where not only can you push tests but they pull tests as part of the release process.
http://nakedalm.com/execute-tests-release-management-visual-studio-2013/
You can have a build kick off the process of build->deploy->test and repeat the process easily...