I have created sample Coded UI Test in VS15 and checked in to the TFS15. Made a build definition using XAML and associate test there.
At MTM frount, I have ready environment with Test controller and agent. Plan and Test tasks are also completed in MTM Testing Center but When I move to Track in Assign Build to assign build I see Available builds drop down is disabled when i refresh i got below error message looking for help on this matter.
I am using Test Controller and Test Agent version 2013.
Application detected an unexpected fault.
Could not load type
'Microsoft.TeamFoundation.Build.WebApi.QueryDeletedOption' from
assembly 'Microsoft.TeamFoundation.Build2.WebApi, Version=14.0.0.0,
Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'.
Have u tried running MTM on a different machine, maybe your MTM installation was not succesful.
Try that and let me know.
Pablo
Related
We are trying to generate code coverage on performing manual tests for application hosted in Azure app services.
I have a trail account in portal.azure.com with 'user' privilege.
Tried the below options:
A. Use VS tools in Azure console and local
B. Use OpenCover in Azure console
Both the above mentioned options using VS tools and Opencover has been used in the localhost (IIS server) to successfully generate code coverage on runnign manual tests.
A. Steps for code coverage using VS tools in Azure:
1. cd D:\Program Files (x86)\Microsoft Visual Studio 15.0\Team Tools\Performance Tools
2. vsinstr /coverage D:\home\site\wwwroot\bin\abc.dll
3. vsperfcmd /start:coverage /output:D:\home\LogFiles\Testing123.coverage
4. vsperfcmd -shutdown
Issue1: The generated code coverage file has error: Empty results generated: No binaries were instrumented. Make sure the tests ran, required binaries were loaded, had matching symbol files, and were not excluded through custom settings. For more information see https://go.microsoft.com/fwlink/?LinkID=253731
However I'm able to generate code coverage on the same application which has been hosted in my local IIS server. The same 4 steps mentioned above is working. The issue is that the instrumented dll doesn't log the transactions performed on the hosted application.
Reference: https://gurunadhduvvuru.wordpress.com/2014/08/01/generating-functional-testing-code-coverage/
PS: We have tried starting the coverage process before and after publish the application.
B. Steps for code coverage using Open Cover in Azure
1. OpenCover.Console.exe -target:C:\Windows\System32\inetsrv\w3wp.exe -targetargs:"-debug -s 1" -targetdir:C:\inetpub\wwwroot\bin\ -filter:+[] -register:user -output:C:\Users\coverage2.xml
Issue 2: Access denied is displayed when the application is not running.
Issue 3: The generated coverage file is empty and doesn't has the same error mentioned in Issue 1, on executing the opencover command when the application is running.
Reference: https://automationrhapsody.com/code-coverage-manual-automated-tests-opencover-net-applications/
Any help on the reason for the generated coverage file to be empty ? or any reference to suffice my requirement would be much appreciated?
It appears that it is a limitation of Azure WebApp Sandbox, all Azure Web Apps run in a secure environment called a sandbox. Each app runs inside its own sandbox, isolating its execution from other instances on the same machine as well as providing an additional degree of security and privacy which would otherwise not be available.
You cannot install arbitrary software when running on Azure App Service. Generally, you cannot make any machine level changes (e.g. registry changes).
Reference: https://github.com/projectkudu/kudu/wiki/Azure-Web-App-sandbox. If your requirement fits, try this on an Azure VM running IIS, where you have greater control on managing the VM.
Using the Test hub in the Web Portal why is the "Associated Automation" tab completely empty, greyed out and prevented from being used?
What do I need to do to enable all this and run a coded ui test? I have hand coded, Coded UI tests that are build and in dlls but need to run the tests in those dlls in my test suite.
For background, I can run the tests in the dlls using a batch file and have done so successfully but struggling to get started running them from my test plan! My Microsoft "Test Manager" has the same problem, all greyed out but as Test Manager is being deprecated I am less concerned about that.
EDIT:
Another way of viewing this problem is to ask why does the Automation Status not show the option "Automated"? See picture below ...
It is not possible per MSDN documentation.
In the visual studio development environment, try right clicking on a test in test explorer, clicking something such as "Associate to Test Case", then entering in the test case number and see if that works. I believe it is greyed out because there is no valid input options through the web right now.
I have created a number of (standalone) automated test-cases captured using CodedUITest in Visual Studio 2013, to test webpages.
They work fine within Visual Studio, individually; and when several of them are put into a single project as a solution, to create a kind of playlist.
However, I'm trying to use Microsoft Test Manager as a 'front-end' - in order to be able to select which tests, create play-lists, decide how many times a specific test-case should be run etc, with the results stored.
TFS is being used to both store my (individual) test-cases, and, where I'd like to deposit the resultant output of pass/fail etc.
Trouble is, even though the test-automation part functions very well within Visual Studio, getting Microsoft Test Manager to function with what I have, and its associated environment is proving a COMPLETE NIGHTMARE.
My system is simple; I have a virtual machine setup with the testing environment which allows (within VS) for me to run these automated tests.
Why is this proving so difficult to work with MTM? It should be easy - I should simply be having to point MTM at the folder my test-cases are stored, and use its GUI to tell it what and how many tests I want to run.
Anyone else have a similar problem, or a similar setup?
All MS do is point me to (countless) pages which I've already read - for which the whole lot seems to be much deeper than it needs to be.
You cant just point MTM at a folder and tell it to run the tests as it would have no idea where to put the data. The results of each codedui running are associated with a corresponding test case in mtm.
You also need to have an automated build create the output (your assemblies) for you. Idealy everything goes together with your application. As your application changes, so will your tests.
You should add your CodedUI projects to the same solution that is used to build the application that you are testing. Then when the automated build for that application kicks off your code is picked up to. Both things, test and application, end up in a drop location. It is that drop location that MTM will use to find your test assemblies.
If, while you have the main solution open, you open/create a Test Case you can go to the automation tab in Visual Studio and associate one of your CodedUI tests with that Test Case. The test case will then show up as Automated in MTM.
Now that we have the versioned bits and know where to find them MTM needs somewhere to run them. If you open MTM and switch to the Lab center you can create a Standard environment to run your tests. This will automatically go install the agents required so you wikk need admin on those boxes.
Now that we have both versioned tests and an environment you can find the Test Case in MTM and see that it is "automated". If you right-click on it and say run you will get a box requesting an environment, and which version of the bits to go run. It will then go off and run those tests against that environment and feed the result back into the Test Case.
Does that help?
You'll need to setup Test controller and Test agent and associate you build with a Test Plan in MTM.In Test Lab you need to associate Test Agent to the Test Controller.
Once done you'll need to allow File & Printer sharing exception
And then you will have to add Test Controllers and Agents to Appropriate Groups.
Control Panel->All Control Panel Items->Administrative Tools->Computer
Management
Environment: TeamCity 6.5.1 on Win2k3, BuildAgent(s) on Win2k3, Visual Studio 2k10, .NET v4, Nant 0.91
I'm completing the setup of TeamCity and am trying to lock down the BuildAgent account on the build machine(s) per our security guidelines. The build is crashing the first time "devenv.exe /build" is called via the Nant script:
Faulting application devenv.exe, version 10.0.30319.1, stamp 4ba1fab3,
faulting module msenv.dll, version 10.0.30319.1, stamp 4ba1fd94,
debug? 0, fault address 0x0000c36b.
I had no luck googling that message. However, if I change the BuildAgent Service from the Local Network Account to the Administrator account, things work. However, if I use another domain account, it fails. Also fails if I add that domain account to the local Administrators group.
Any ideas on what I'm missing? Is there a specific privilege you need to have in order for a "DevEnv /build" to work without crashing?
Yuck, I just went through this recently. First, use devenv.com, not devenv.exe. The devenv with the com extension can build a solution and send all output to the console, without using the gui. As the TeamCity agent is a service, it may not be allowed to interact with the gui at all.
Second, and I realize that this might not be possible for you (especially if you are building an MSI), but consider doing whatever you need to do to use the built in Visual Studio build runner that comes with TeamCity. It does utilize MSBuild to do its work. If you go this route and you still need devenv, then go find MSBuild Extensions Pack, which has already solved a lot of these issues with their own devenv build task.
Honestly, I ended up replacing Microsoft's installation projects with alternatives (InstallShield or WiX), and never looked back.
I'm trying to create a worker process for Azure and I'd like to test it via nUnit. I need to write a process which connect to storage and uploads data to it. The problem is I can't find any references to how you actually set up the testing framework to start dev fabric etc.
I've created the Work Processor role and then created a testing project, wired up nunit to start and for the project to pick up the test project dll. This all works fine with nUnit opening, I'm able to see the test dll and run the test.
My issue is "How do I get the dev fabric to start up when running the project through nUnit?" I'm presuming that I need to put something into the setup routine within the test project but I can't see what I should put in.
If anyone has any tips, experience etc., links to how to etc. I'd be very grateful. I'm sure that I can't be the first person who want to put Azure projects under test.
Haven't done this myself, but the DevFabric (compute and store) has a command-line interface.
You would just invoke the commdn from your test stand-up routine and it should be good to go.
CSrun.exe is the command, please see MSDN for usage reference:
http://msdn.microsoft.com/en-us/library/gg433001.aspx