When I'm running the test in Microsoft Test Manager I get an error: The build directory of the test run either does not exist or access permission is required.
What does it mean? Where can I check build directory?
I suppose you are running automated tests.
If you are running automated test in Microsoft Test Manageg (MTM) you have Test Case(s) automated by your test implementation.
So MTM need to know where is the drop location of the build building your tests.
That means you have to:
Create a Build definition that build the code implementing your tests.
Configure MTM to use the drop location of this build definition.
You can check which build MTM is currently using here:
(I suppose you will have "undefined" standing there.)
You can configure which build is to use here:
Just check here how to configure your Test Plan which build to use when running tests: Creating and Defining a Test Plan
Additionally I would highly recommend you this free e-book: Testing for Continuous Delivery with Visual Studio 2012
Related
Currently I am trying to using Azure Test Plans to manage all test cases, but I just found how to build / generate manual test cases. I also make some research about Pipeline and I just write some "hello world" test scripts and using Pipeline to automatically test the code. My question is: How could I combine TestPlans with Pipeline to make the test in pipeline recordable? Thank you so much!
this is how the pipeline looks like
this is how the test code be checked
My question is: How could I combine TestPlans with Pipeline to make
the test in pipeline recordable?
I'm a bit confused about what the recordable mean in your question. To combine TestPlan with Pipeline, you should configure the Test Plan Settings. For example, set a build pipeline(build and generates the test binaries) and a release pipeline with test-related task in your Test Plan Settings.
The normal logic is:
A test plan containing your automated tests, which you have associated with automated test methods using Visual Studio 2017, or Visual Studio 2015 or earlier.
A Team Build pipeline that generates builds containing the test binaries.
The app to test. You can deploy the app as part of the build and release workflow and also use it for on-demand testing.
For more details please check Run automated tests from test plans.
I have downloaded and installed in a windows server 2016 a vs test agent, which was installed by a tool called Test Agent Configuration Tool. I then followed this guide to configure the agent to run tests that interact with desktop - I want to run CodedUI Tests for web projects from whithin my release pipeline.
This is the output from the Test Agent Configuration Tool
When I run my release pipeline I get this error during the test phase:
Error calling Initialization method for test class
Captura.PDV.Web.Test.CodedUI.CodedUITest1:
Microsoft.VisualStudio.TestTools.UITest.Extension.UITestException: To
run tests that interact with the desktop, you must set up the test
agent to run as an interactive process. For more information, see "How
to: Set Up Your Test Agent to Run Tests That Interact with the
Desktop" (https://go.microsoft.com/fwlink/?LinkId=255012)
If you are
running the tests as part of your team build, you must also set up the
build agent to run as an interactive process. For more information,
see "How to: Configure and Run Scheduled Tests After Building Your
Application" (https://go.microsoft.com/fwlink/?LinkId=254735)
Here is my test task configuration:
Some notes and questions:
I can run the tests calling vstest.console.exe manually from within the server with success
My release pipeline runs on VSTS
I am not using Microsoft Test Manager - Is it really necessary?
Do I need to set up a test lab enviroment?
Do I need to have a controller for the agent?
I just want to run my tests in first place to see how it goes and then proceed to more advanced tests scenarios using MTM
You're mixed up between the old way of doing things and the new way.
You don't need to use MTM -- MTM is deprecated.
You don't need to set up a lab environment. Lab environments are deprecated.
You don't need special test controllers or agents.
All you should need to do in order to run UI tests is:
Set up a build/release agent that runs interactively.
Run the tests on that agent with the Visual Studio Test task with the "Test Mix Contains UI tests" check box checked.
That's it.
I'd also recommend against using Coded UI and instead use Selenium (or Appium for desktop applications); Microsoft is no longer investing in the Coded UI platform and recommends using Selenium as an alternative.
For creating a Build-Deploy-Test Workflow for my Wpf application, I ma using TFS Build definition and MTM to create Environment.
I am using following Steps:
I am Using LabDefaultTemplate.11.xaml in Build Definition
In the Lab Process Settings, I have given the required details as:
Environment: I have created an environment using MTM on the VM
in the environment I have added a Desktop Client(the same VM only) in the machines List
Please note that my Test Controller, agent, MTM all are on the VM only
Build: I have selected an existing build definition and it's working fine.
Deploy: I have specidfied command for application installation(running of msi file)
also, In the deploy section only, at the end I have specified a batch file where I am calling my CodedUI tests via command line(using VSTEST.CONSOLE.EXE)
Test: Not using it i.e. Not checked the "Run these Tests in Environment"
When I run this build definition, build creation, and installation are working fine but failes to run the codedui tests. Getting following error:
Error Message:
Error calling Initialization method for test class CodedUITestProject1.AISTest: Microsoft.VisualStudio.TestTools.UITest.Extension.UITestException: To run tests that interact with the desktop, you must set up the test agent to run as an interactive process. For more information, see "How to: Set Up Your Test Agent to Run Tests That Interact with the Desktop" (http://go.microsoft.com/fwlink/?LinkId=255012)
If you are running the tests as part of your team build, you must also set up the build agent to run as an interactive process. For more information, see "How to: Configure and Run Scheduled Tests After Building Your Application" (http://go.microsoft.com/fwlink/?LinkId=254735)
Stack Trace:
at Microsoft.VisualStudio.TestTools.UITesting.Playback.Initialize()
at Microsoft.VisualStudio.TestTools.UITesting.CodedUITestExtensionExecution.BeforeTestInitialize(Object sender, BeforeTestInitializeEventArgs e)
at Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestExecution.RaiseBeforeTestInitialize(BeforeTestInitializeEventArgs args)
at Microsoft.VisualStudio.TestTools.TestTypes.Unit.UnitTestExecuter.RunInitializeMethod()
Test Run Failed.
Also, I have made sure that test agent is configured as an interactive process.
Any Help would be highly appreciated.
The issue got resolved after setting the build controller also to run as a interactive process.
How to setup the properly the continuous integration when we have 1 solution (BuildDeploy.sln) and 2 cloud services (DataHora.ccproj and Maiuscula.ccproj):
When I try to run the default continuous deploy script created for visual studio online of maiuscula the cloud service is replaced by DataHora cloud service.
Seems that he gets only the datahora ccproj, because he is the first cloud project service of the solution.
To test my theory I create a custom entry in my configuration manager tha only builds maiusculo and change the Build Definition to get only Any CPU|X_Maiusculo
When I run the default build defition for maiusculo that was created by visual studio continuous integration I receive the following error:
Built $/ContinousDeployment/BuildDeploy/BuildDeploy.sln for target(s) DataHoraCloudService:Publish.
C:\a\src\BuildDeploy\BuildDeploy.sln.metaproj: The target "DataHoraCloudService:Publish" does not exist in the project.
So my question was: How to setup my continous integration script to build and deploy only one project of my solution?
If you have more than one deployable code project in the solution then the default that's will not work. It was created as a quick deploy for good with some solutions.
If you instead switch to the default template and create a PowerShell that does your deployment of both services you can call It from the post-test location. This is a better way anyway as It is possible to debug outside of a build.
I have a Visual Studio solution housing an Azure Cloud Service with the following projects:
CloudService
CloudServiceRole
Tests
Where the Tests project is a standard MSTest project that contains unit tests for the business logic in the CloudServiceRole project.
The code is stored in Visual Studio Online and I have hooked up the automated CI build deployment that Azure offers. When I check in code, my staging deployment of the cloud service is automatically updated. However, the Tests project is never even built during the CI builds! This, of course, means that no unit tests run during the build as the "run unit tests" part of the build process finds no assemblies with tests.
My goal is to change this so the tests project is built and all the unit tests executed.
Looking at the MSBuild arguments that the CI deployment process uses, it appears that only the CloudService:Publish target is executed. The CloudService project has no dependency on the Tests project so MSBuild never even builds the latter.
What I have tried
I cannot manually add a CloudService->Tests dependency because when I add dependencies on projects that are not Cloud Service Role projects, I get an error during build (The item "C:\a\src\MyProject\Tests\Tests.csproj" in item list "ProjectReferenceWithConfiguration" does not define a value for metadata "Name".) and I cannot add a CloudServiceRole->Tests dependency because that would make a circular dependency.
Instructing MSBuild to build the full solution by manually adding a /t:Build parameter resulted in yet another error: C:\a\bin\ServiceDefinition.csdef: Need to specify the physical directory for the virtual path 'Web/' of role Web.
Adding the Tests project as a separate build target, alongside the solution, results in the tests getting built! However, at the same time, it disables the Continuous Deployment functionality: More than one solution found. Continuous Deployment skipped.
Trying to make a fake Cloud Service Role project that references the Tests project but has zero instances configured results in a build error: WAT100: The following roles 'Tests.FakeRole' have an instance count of 0. Instance count of 0 is not supported in deployments to Microsoft Azure. Attempting to disable this validation results in a build error due to a defect in the Azure SDK.
You need to run a Build and a Publish separately. I ran into the same problem on my VSO (now VSTS) project and this fixed it. This happens because your cloud service doesn't depend on your unit test project.
1) Visual Studio Build (or MSBuild) action with arguments /t:Build (clean here)
2) Visual Studio Build (or MSBuild) action with arguments /t:Publish (do not clean here)
Note: I had to run these actions separately (not /Build;Publish) otherwise I got an error about the cloud service entry point.
Pieced this together from this question and from here and here.
One workaround that appears to bring results is to add a pre-MSBuild script to the build definition and explicitly build the Tests project in that script.
:: This is used as a pre-build script in Continuous Deployment builds because on their own, they do not build the Tests project.
"c:\Windows\Microsoft.NET\Framework64\v4.0.30319\MSBuild.exe" %~dp0\..\..\Tests\Tests.csproj /t:Build /p:Configuration=Debug;Platform=AnyCPU;OutDir="%~dp0\..\..\..\..\bin\\"
It appears to do the job, although I am not sure what side-effects I should be aware of. My main concern is that the binaries from this build go into the same directory as the binaries from the Cloud Service build - is there perhaps some possibility of conflict here?