Should I install the TFS Test Controller on the same machine as the Build Controller?
and also let it launch the browser on this same machine?
What is the best practice?
We have a TFS 2013 Build Controller on a local machine.
We have CodedUI tests which launch a browser which we want to get going again (worked on tfs2010).
Also where exactly do I get the installer for the Test Controller? I presume its inside the TFS 2013 server ISO.
Yes, I think you can install test controller and build controller on the same machine. You can consider installing build agent on a different machine than TFS Application Tier for performance consideration especially you would like to set up some Gated Check-in or CI builds.
In addition, as you would like to run Coded Ui test, you should configure the test agent to interact with the Desktop. Check https://msdn.microsoft.com/en-us/library/Ee291332(v=vs.120).aspx
In addition, you can download the Test Controller here. (Both test controller and test agent included) https://www.microsoft.com/en-us/download/details.aspx?id=40750
Related
I read about Microsoft Visual Studio App Center, but it seems to only apply to developing native apps.
I need to realize a Selenium-based CI/CD workflow, like this:
Programmers develop a web application locally, pushing their changes to a local GitLab repository.
The on-premise "service" watches the GitLab repository for changes. If changes have been detected, then:
Pull project.
Compile project in Release configuration.
Run NodeJS unit tests and validate results.
Deploy to on-premise test site.
Using local (non-public) URLs, run Selenium web tests in parallel on different devices (e.g. Windows 10 desktops, tablets and smartphones; Apple tablets and smartphones; Android tablets and smartphones) with different browsers (i.e. Edge, FireFox, Chrome, Internet Explorer) and validate results.
Using "local" URLs stands for something similar to BrowserStack Local.
This includes the ability to take screenshots and to compare them to reference screenshots of a previous test on the same device/browser.
Show test results of all the tests.
If all the tests have run successfully,
Request confirmation to deploy to production site.
Deploy to production site.
Does Microsoft offer on-premise services for this? Which are they?
You can still get TFS from Microsoft as an On-premise variant of VSTS. Both have the same codebase, but the SaaS variant is released more often (every three weeks!).
VSTS/TFS can handle Git repositories out of the box, with all steps you described. Even uploading a new software version to all the stores :-)
My company is searching for a test management and test automation tool. Since we already use TFS 2015 for version control of our internal documents and product documentation, the question arose whether or not TFS is a suitable candidate for our testing needs.
Our situation is the following: We produce a software system that runs on our custom Linux-based OS, so all test systems are Linux machines. Each member of our test center has a personal Windows computer as well. We want test automation primarily for our functional tests, since our unit tests are already integrated into our build process.
I have done some research into this matter and found out that test management and manual testing should work quite fine with TFS via the Web Access. However, I cannot find reliable statements on the question whether or not the test automation via Microsoft Test Manager works with (remote) Linux machines.
So this is my question: Is there a way to use TFS and the Microsoft Test Manager to automate tests under Linux? If TFS itself cannot do this, are there integrations with external test automation frameworks?
For my research so far, I have used the following resources:
the official documentation on testing with TFS
https://www.visualstudio.com/en-us/docs/test/overview
this official site on how to automate a test case with Microsoft Test Manager
https://msdn.microsoft.com/en-us/library/dd380741.aspx
I would appreciate any kind of help provided.
Afraid not, Install and Deploy test agent is not support in Linux System.
What system requirements do I need to install my test test agents?
Windows 10
Windows 8, Windows 8.1
Windows 7 Service Pack 1
Windows XP Service Pack 3
Windows Server 2012, Windows Server 2012 R2
Windows Server 2008 Release 2, Service Pack 1
Source Link: Install and configure test agents
Features such as diagnostic data collection and functional test would not work on these machines since the test agent is not supported on those OSes.
More detailed info about TFS, MTM Test Management, Automation test, you can refer my answer in this question: Test Automation tools shipped with Visual Studio 2015 Enterprise?
Per my view, we don’t have to run automation tests via TFS. You can use command line to run test. I guess you just want to reflect the test result in the TFS/build. If that’s the case, you can
Run your test under linux which will generate trx file
Publish the test result to the TFS/build. See
https://learn.microsoft.com/en-us/previous-versions/ms243151(v%3dvs.140)
I want to run my coded UI test on a testing machine, but I wish If I can escape from installing VS2012 on my Test machine. Is there any any way to do it?
According to this blog you only need to install the test controller software on a machine: http://blogs.microsoft.co.il/shair/2015/04/08/automation-runner-running-tests-without-visual-studio/
Also there is a tool that wraps around VSTEST.Console to simplify running tests without VS:
Automation Runner only Requires Test Controller to be installed on the
running machine – No MTM or Visual Studio Required.
I used VS Test Agent and it ran the tests for me without Visual studio
I am having difficultly trying to come up with a working automated build and automated testing solution for testing several applications remotely.
I have several different applications which install onto Windows test machine VMs (Win 7) and an automated test solution which is built using a test settings file and test case filter and run automated tests against this application.
I have two groups of VMs, one group to build the automated tests solution which then sends the tests remotely to the second group which has the application installed on.
Currently this is done by a default template build definition (VS 2012) which builds the test solution on a build VM and sends the tests remotely to a test VM (has application needed manually installed prior to starting build), defining a test controller in the test settings file (there is a test settings file per test VM / build definition) the tests are sent to that test controller. Each test VM has a test controller and test agent on the same VM to prevent the tests going to multiple machines, the tests need to be sent to one machine with that specific application installed.
I am wanting to scale this process and allow for complete automation, so I can just kick off a specific application build which will install the required application on a free test VM, build the test solution code on a build VM and send the tests remotely to the test VM which now has the application needed installed on it, run the tests and send the results back.
I am having issues with doing this between the build and test VMs and having the test settings file on the build machine updated with the test controller name of the free test VM.
Is something like this possible to do and if so what would be the best way to go about this?
Any help or feedback would be greatly appreciated.
Setup:-
TFS / Visual Studio 2012 and Windows 7 build / test machines
Reference links:-
How can TFS build process be configured to execute tests on Test Agents through a Test Controller?
Ok, so you need to look at Lab Management for configuring environments. TFS has a built in method of saying that a set of machines go together and that you want to run tests against it. You can then use MTM to push groups of automated tests to environments and record the data.
In addition you should look at Release Management for VS 2013 (works with 2012) for pushing out the bits to your environments.
With those two tools you can maintain release pipelines where not only can you push tests but they pull tests as part of the release process.
http://nakedalm.com/execute-tests-release-management-visual-studio-2013/
You can have a build kick off the process of build->deploy->test and repeat the process easily...
We are having compilations problems in a TFS server and it's because the server lacks several libraries built in the default VS2012 Premium installation (Microsoft Fakes in this case).
I'm unsure of going ahead installing a full instance of VS, but first I want to know what is the best practice in this regard?
What is recommended?
Since we are talking a sandbox, do whatever and don't worry about it. If we are talking best practices, it's not a good idea to put your build tier on the app tier / data tier. Any developer could check in code that gets run on the server during the compile and trash your entire environment.
Have you looked at Visual Studio Online? It's a hosted TFS service and you can use their hosted build controller or configure your own. That makes for a very good sandbox IMO.
I don't see any issue installing VS on the TFS server(I assume you run your builds on that server too and that's when you are seeing the problem. Ideally tfs server and build box should be separate but some people use the same box.)
I have used Visual Studio on the build box several times to debug issues with builds. You just need to make sure you close the VS instance (if it has a solution open) once you are done with debugging otherwise your builds can fail when they try to clean up the project directory at the start of the build.
We run a single server TFS instance which has everything - sql, SharePoint and tfs - running on it. It is also a build server so it has to have VS 2010 and 2012 installed. We've done this with all versions since 2005 and have had no issues with it at all.