I am trying to configure and run selenium test cases from Azure Devops Release pipeline. I did configure Visual studio Test Platform installer followed by Visual studio test. However, while running the 'test task', its erroring out stating
2019-12-22T11:31:50.7602521Z ##[warning]No test sources found matching the given filter '**\*csproj'
Am I missing something here or not doing proper configuration?
My main test case is being written in .cs file and tried including it in path as well but still same error.
Can someone please guide me?
thanks,
Pankaj
You have to add the path to the assemblies - dll. cs files are not used here as they are not compiled.
Provide the search pattern in the Test files field
Example search pattern:
**\publish\Test.Project.Name.dll
!**\*TestAdapter.dll
!**\obj\**
Have you built your project? Since you said you can see only files which are .cs or .csproj. You should first build your project to generate .dll files which are used in your vstest test files.
You can add the Visual Studio Build Task above the Visual Studio Test Task, then choose your project(.csproj or .sln) in the Solution Tag.
After building, you will see the .dll files in your Artifact, then, as Jonah said, you need to specify the location of the .dll file in Test files.
Related
My ASP .NET Solution (net framework 4.7.2) consists of several projects. I am interested in having a build definition (Azure) that will run my tests from a specific folder ( that folder contains some tests projects -nunit tests )
I have searched for information about this, but most solutions seemed constructed towards building & releasing, with all the tests in the whole solution being run.
How should I approach this? I will need to build my solution (Visual studio build ) as a task (?), but what should be the steps(tasks) to select and run my test folder ?
You could actually do something like this How to run a list of tests with VsTest.Console on windows agents if you prepare such file with list of tests from that folder.
You can find vstest.console.exe on this location on host agents
C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\Extensions\TestPlatform\vstest.console.exe
In that case you just need to prepare such file, put it in source control and run in your pipeline.
I will need to build my solution (Visual studio build ) as a task (?)
The answer is yes. That because the folder **\bin\** is ignored by source control by default. We need to build the solution in our build pipeline to generate dll files.
but what should be the steps(tasks) to select and run my test folder ?
We could specify the specific folder in the Visual Studio Test task in the Search folder:
Or specify that folder in the option Test files:
I have added a release definition in Visual Studio Team Services to deploy an application to a Azure cloud service. I'm trying to run some integration tests after a successful deployment:
But unfortunately tests are not run:
Warning: No test is available in C:\a\301532a74\Build\Api.Web.IntegrationTests\Api.Web.IntegrationTests.dll. Make sure that installed test discoverers & executors, platform & framework version settings are appropriate and try again.
Information: Additionally, you can try specifying '/UseVsixExtensions' command if the test discoverer & executor is installed on the machine as vsix extensions and your installation supports vsix extensions. Example: vstest.console.exe myTests.dll /UseVsixExtensions:true
##[warning]No results found to publish.
I have installed the NUnit Test Adapter (nuget package) for all projects containing tests (unit tests and integration tests). The weird thing is that I am able to run the tests as part of a build but not as part of a release.
I also tried to set the Path to Custom Test Adapters but it doesn't seem to make a difference.
What I am missing here? Is there a better way to run some integration tests as part of a release?
This is usually caused by the NUnit test adapter cannot be found by VSTest.
Usually, In "Build" environment, NUnit test adapter is restored in working folder by nuget, so VSTest can find it during the testing. But in "Release" environment, NUnit test adapter does not exist by default. So you need to update your build definition to upload the NUnit test adapter files to "drop" and the NUnit test adapter will be avaiable in "Release" environment after the artifacts is downloaded. Then you can set the "Path to Custom Test Adapters" to "$(System.DefaultWorkingDirectory)\" and try again.
If you are using your own build agent, you can also place the NUnit Test Adapter in a permanent path that the build agent can access and set the "Path to Custom Test Adapters" to it. Or install NUnit test adapter extension for VS on the build agent and enter "/UseVsixExtensions:true" in "Visual Studio Test\Other console options".
I have found a solution for my problem, based on Eddie's suggestions:
Step 1 - Copy all NUnit assemblies to the output folder of the test project (set the Copy Local property to True)
Step 2 - Create an artifact that will contain all files of the bin folder of the test project. This will contain not only the test assemblies but also the NUnit assemblies
Step 3 - In the test task, set the Path to Custom Test Adapters to the artifacts folder that contains the test assemblies and the NUnit adapters assemblies
That's it! Next time you create a release and deploy to any environment tests will be executed.
My problem was that i didn´t have a testrunner for NUnit installed at all, since I´m using resharpers testrunner.
Therefore my solution was as simple as adding the nuget package, "NUnit3TestAdapter" version="3.4.0", to my test project.
In visualstudio.com (VS online) "Nuget restored adapters are automatically searched for." and you need not/should not specify a path for "Custom Test Adapters"
- I don´t know if this also works for NUnit test adapter version 2.
I am have built an MSBuild project that has
the main .proj file with several Targets inside,
several .targets files
and several .rsp response files to run this project in several ways. Such as
Build, Clean/Build/Deploy, Clean/Build/Test/Deploy.. You get the idea.
What I now want to do is take this MSBuild and run it in TFS scheduler after I pull the source code. So the workflow should be
1. Pull the source code in TFS
2. Run MSBuild project in the scheduler so I might set up tasks to run hourly and nightly.
The MSBuild will take care of Deploying to IIS, unless someone has a more efficient way of deploying after an hourly build.
How can I accomplish this in TFS?
Thank you
You can use the Team Explorer UI in Visual Studio to define a "scheduled" build definition.
http://visualstudiomagazine.com/articles/2012/04/11/creating-a-build-definition-in-tfs.aspx
If you want to take this to the next level, research "continuous integration" which is also a built in capability.
Just wondering if there is a preference between sln or csproj files when building projects in TFS 2012. We typically build sln files in TFS to support our dev teams and that's usually the standard, but one team is asking why we can't build csproj files instead in TFS.
I noticed that to build a csproj file you have to provide arguments in the MSBuild Arguments field for a TFS build definition, and you do not have to provide these arguments when building a solution file. So other than this small detail I'm not sure what the pros and cons are between building an sln vs csproj.
Can someone please shed some light on the benefits, pros/cons of building a sln vs csproj in TFS, is there a common practice, a standard, or does it really matter?
Just wondering if there is a preference between sln or csproj files
when building projects in TFS 2012. We typically build sln files in
TFS to support our dev teams and that's usually the standard, but one
team is asking why we can't build csproj files instead in TFS.
Why do they want to do this? Is there some sort of advantage articulated in this inquiry?
I noticed that to build a csproj file you have to provide arguments in
the MSBuild Arguments field for a TFS build definition, and you do not
have to provide these arguments when building a solution file. So
other than this small detail I'm not sure what the pros and cons are
between building an sln vs csproj.
A .sln is a master project that is converted to an msbuild script at runtime and will use a metadata file using the same schema as the .csproj projects in your solution.
To see what I'm talking about, open a command prompt, type "SET MSBUILDEMITSOLUTION=1" then "msbuild.exe solution.sln", then parse the new "solution.metaproj" and "solution.metaproj.tmp" files.
Can someone please shed some light on the benefits, pros/cons of
building a sln vs csproj in TFS, is there a common practice, a
standard, or does it really matter?
A .csproj would be a single project while a .sln would be a collection of projects. Building a single .csproj would yield the output binaries of that project (along with dependencies) and building the entire solution would yield the outputs of the entire solution.
I had the same question. Build times may be a bit slower but I haven't tested this myself (probably a negliable difference). I believe that when you build using the solution file it will automatically set 'BuildInParallel' to true.
Faster Buidls with MSBuild
I am currently working on the TFS 2012 deployment build. Based on what I experience, you don't have to provide MSBuild Arguments unless it is really required.
Lets take the following example: You have a solution with 2 projects,
Soln
> Web Proj 1
> Console Proj 2
> Dependency Library Folder
case 1: building a sln
a. In source settings, you have to mention only your solution folder. The MSBuild will automatically take care of any dependencies as they are within the solution.
b. In Process -> Items to build, you have to mention your solution file.
c. For example, if your solution has a web project, then the MSBuild output will be as shown below,
\\<build server> \d$\Builds\<Build Name>\<Build File Name>_20141210.6\_PublishedWebsites\<web proj>
Case 2: building a csproj
a. In source settings, you have to mention only your csproject folder and also the dependency folder path separately.
b. In Process -> Items to build, you have to mention your csproject file.
If you are looking for something more specific, please let us know.
I have a publish profile set up in a VS 2012 project. When I right click on the project in VS, select Publish and click on the [Publish] button, it publishes the project to the server using the settings provided in the Publish Profile.
When I use msbuild and the command line, with the following syntax:
msbuild.exe .\mvc.csproj /p:PublishProfile=DevServer
/p:DeployOnBuild=True /p:Password=MyPassword /p:AllowUntrustedCertificate=true
Then it builds the project, and gives me a message:
Package "mvc.zip" is successfully created as a single file at the following location: file:///c:/code/mvcsite/obj/Debug/Package
And then provides info on how to deploy the package.
How can I deploy from the command line? My ultimate goal is to run the deployment through TeamCity, and am right now trying to get my command line properties correct. However, the most that I can do from the command line right now is to create the deployment package, but not to run the actual deployment. How can I do both (preferably with one statement, to duplicate what happens in VS2012 when I deploy from there)?
Since you are building the .csproj you missed one important property
/p:VisualStudioVersion=11.0
This property was introduced in MSBuild 4.5 to facilitate project sharing between VS 2010 and VS 2012. A drawback; when building the .csproj you need to specify the value for this property. When building the solution file the value can be derived from the solution file version. Read more at my blog http://sedodream.com/2013/01/06/CommandLineWebProjectPublishing.aspx.