I have added a release definition in Visual Studio Team Services to deploy an application to a Azure cloud service. I'm trying to run some integration tests after a successful deployment:
But unfortunately tests are not run:
Warning: No test is available in C:\a\301532a74\Build\Api.Web.IntegrationTests\Api.Web.IntegrationTests.dll. Make sure that installed test discoverers & executors, platform & framework version settings are appropriate and try again.
Information: Additionally, you can try specifying '/UseVsixExtensions' command if the test discoverer & executor is installed on the machine as vsix extensions and your installation supports vsix extensions. Example: vstest.console.exe myTests.dll /UseVsixExtensions:true
##[warning]No results found to publish.
I have installed the NUnit Test Adapter (nuget package) for all projects containing tests (unit tests and integration tests). The weird thing is that I am able to run the tests as part of a build but not as part of a release.
I also tried to set the Path to Custom Test Adapters but it doesn't seem to make a difference.
What I am missing here? Is there a better way to run some integration tests as part of a release?
This is usually caused by the NUnit test adapter cannot be found by VSTest.
Usually, In "Build" environment, NUnit test adapter is restored in working folder by nuget, so VSTest can find it during the testing. But in "Release" environment, NUnit test adapter does not exist by default. So you need to update your build definition to upload the NUnit test adapter files to "drop" and the NUnit test adapter will be avaiable in "Release" environment after the artifacts is downloaded. Then you can set the "Path to Custom Test Adapters" to "$(System.DefaultWorkingDirectory)\" and try again.
If you are using your own build agent, you can also place the NUnit Test Adapter in a permanent path that the build agent can access and set the "Path to Custom Test Adapters" to it. Or install NUnit test adapter extension for VS on the build agent and enter "/UseVsixExtensions:true" in "Visual Studio Test\Other console options".
I have found a solution for my problem, based on Eddie's suggestions:
Step 1 - Copy all NUnit assemblies to the output folder of the test project (set the Copy Local property to True)
Step 2 - Create an artifact that will contain all files of the bin folder of the test project. This will contain not only the test assemblies but also the NUnit assemblies
Step 3 - In the test task, set the Path to Custom Test Adapters to the artifacts folder that contains the test assemblies and the NUnit adapters assemblies
That's it! Next time you create a release and deploy to any environment tests will be executed.
My problem was that i didn´t have a testrunner for NUnit installed at all, since I´m using resharpers testrunner.
Therefore my solution was as simple as adding the nuget package, "NUnit3TestAdapter" version="3.4.0", to my test project.
In visualstudio.com (VS online) "Nuget restored adapters are automatically searched for." and you need not/should not specify a path for "Custom Test Adapters"
- I don´t know if this also works for NUnit test adapter version 2.
Related
I am trying to configure and run selenium test cases from Azure Devops Release pipeline. I did configure Visual studio Test Platform installer followed by Visual studio test. However, while running the 'test task', its erroring out stating
2019-12-22T11:31:50.7602521Z ##[warning]No test sources found matching the given filter '**\*csproj'
Am I missing something here or not doing proper configuration?
My main test case is being written in .cs file and tried including it in path as well but still same error.
Can someone please guide me?
thanks,
Pankaj
You have to add the path to the assemblies - dll. cs files are not used here as they are not compiled.
Provide the search pattern in the Test files field
Example search pattern:
**\publish\Test.Project.Name.dll
!**\*TestAdapter.dll
!**\obj\**
Have you built your project? Since you said you can see only files which are .cs or .csproj. You should first build your project to generate .dll files which are used in your vstest test files.
You can add the Visual Studio Build Task above the Visual Studio Test Task, then choose your project(.csproj or .sln) in the Solution Tag.
After building, you will see the .dll files in your Artifact, then, as Jonah said, you need to specify the location of the .dll file in Test files.
I am attempting to move a solution from TFS 2012 to TFS 2018 SP2RC2 but I can't get the unit tests to run correctly. All projects have been re-targeted to 4.7.1 and are built as x86 platform. We have a testsettings file that supplies nothing but deployment items. I am using the new VSTest Platform Installer task (as directed by MS) and the VS Test Task. At the start of the test run I get the following message:
Test run will use DLL(s) built for framework .NETFramework,Version=v4.5 and platform X86. Following DLL(s) do not match framework/platform settings.
So all of the test are skipped as they target 4.7.1. Where is this 4.5 setting coming from? I cannot find it specified anywhere and can't figure out how to change it.
There is a /Framework: parameter that you can specify to VSTest.
In your case, you should specify /Framework:.NETFramework,Version=v4.7.1
See more at https://msdn.microsoft.com/en-us/library/jj155796.aspx?f=255&mspperror=-2147217396
To add this parameter in a Azure DepOps yaml pipeline, use the otherConsoleOptions argument
- task: VSTest#2
otherConsoleOptions: '/Framework:.NETFramework,Version=v4.7.1'
The fix I found for this in Visual Studio is way easier than I thought:
Exit all instances of VS
Open your project folder in Windows Explorer, find the .vs folder, delete it
Restart VS, the folder rebuilds itself, tests work again.
Apparently there are some settings that the NUnit plugin stashes in this folder and they are in binary so you can't edit them. This happened to me after I updated to NUnit3TestAdapter version 3.17.
I had trouble running a test project developed on another machine where no tests where able to be run using the MSTest test runner. In addition to message in your question I also got the messages:
Make sure that test discoverer & executors are registered and platform
& framework version settings are appropriate and try again.
Discover test finished: 0 found
In my case I resolved it by unloading and updating the .csproj file adding the following import directly under the root Project tag.
<Import Project="..\packages\MSTest.TestAdapter.1.3.2\build\net45\MSTest.TestAdapter.props" Condition="Exists('..\packages\MSTest.TestAdapter.1.3.2\build\net45\MSTest.TestAdapter.props')" />
Hope this answer will help save you some time.
I got this error using Visual Studio 2019 with NUnit 3.12.0:
Test run will use DLL(s) built for framework
.NETFramework,Version=v4.5 and platform X86. Following DLL(s) do not
match framework/platform settings.
Project.UnitTests.dll is built for Framework 4.5.2
and Platform AnyCPU.
Installed NUnit3TestAdapter 3.13.0 and then everything started working. Did not need to modify Framework version or CPU settings.
Had the exact same issue as Bill, working on legacy code in VS2019, nothing else had worked. I simply changed the Run Settings to use the Auto Detect.
In the menu go to Test -> Configure Run Settings -> Auto Detect runsettings File
For me skipping tests in local VS2022 run was related to testsettings file. I deleted .vs folder of my project then selected
Test ->Configure Run Settings -> Select Autodetect run settings file
After that I can debug unit tests
TLDR: If you use *.runsettings file for your test projects, try removing TargetFrameworkVersion node
I had similar problem but with .Net Core
Test run will use DLL(s) built for framework
.NETFramework,Version=v4.0 and platform X64. Following DLL(s) do not
match framework/platform settings. MyProject.Tests.dll is
built for Framework .NETCoreApp,Version=v3.1 and Platform AnyCPU.
Turns out there was another problem. For my other .NET Framework test projects I had defined *.runsettings file
And in this file I had
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<!-- Parameters used by tests at runtime -->
<TestRunParameters>
<Parameter name="ConnectionString" value="Data Source=localhost;Initial Catalog=MyDatabase;Integrated Security=True" />
</TestRunParameters>
<RunConfiguration>
<!-- Framework35 | [Framework40] | Framework45 | FrameworkCore10-->
<TargetFrameworkVersion>Framework40</TargetFrameworkVersion>
</RunConfiguration>
</RunSettings>
Supported values for TargerFrameworkVersions based on https://learn.microsoft.com/en-us/visualstudio/test/configure-unit-tests-by-using-a-dot-runsettings-file?view=vs-2019
are:
FrameworkCore10 for .NET Core sources, FrameworkUap10 for UWP-based
sources, Framework45 for .NET Framework 4.5 and higher, Framework40
for .NET Framework 4.0, and Framework35 for .NET Framework 3.5.
But for Core 3.1, FrameworkCore10 doesnt work!
So I removed node TargetFrameworkVersion completely
I had the same problem in VS 2019 on a legacy project.
Restarting Visual Studio did not help, nor did build as Release then rebuild as Debug, nor did deleting the .vs folder. I did those things and still was unable to run tests, getting a message like the OP's during test detection each time. I mention these here for completeness.
I deleted the LocalTestRun.testrunconfig file and the *.vsmdi file from Solution Explorer, cleaned and rebuilt the solution, and the unit tests worked again.
I was running into the same issue with the Visual Studio Test Task version 2 in an Azure Devops 2019 pipeline after a .csproj test project was upgraded from .NetFramework 4.6.1 to .NetCore 3.1.
The fix for us was to be more specific when specifying the test files. The default of **\*test*.dll was finding several other assemblies with "test" in their names that didn't exist before the upgrade. Using an explicit **\<MyTestAssemblyName>.dll fixed the issue.
I am running NUnit 2.6.3 in VS2012 with NUnit Test Adapater 1.1.0.0, on 64 bit Windows 7. The project is C# winforms.
My project's unit tests have stopped running from within VS. I can launch NUnit directly and run the project OK, but running or debugging tests from within VS just greys them out with this message in the output window:
No test is available in
[Your Output Path].
Make sure that installed test discoverers & executors, platform &
framework version settings are appropriate and try again.
Any pointers of what that means? Where can I find the "platform and framework version settings" - does it refer to NUnit settings, project settings, VS settings, or one of the NUnit plugin settings?
I think that it's related to me updating to the latest version of the Test Adapter, but unfortunately I'm not sure :(
Have fixed it by:
- Updating my project target to .NET framework 4.5 from 3.5 (which my project was in - not my choice!). I cannot find any documentation stating this as a limitation, so perhaps it's a bug.
(I have also changed to using NuGet to install NUnit to the project instead of the VS plugin, but I'm not sure that had an effect).
I had a similar problem but when I tried to run my tests just showed that yellow inconclusive circle.
I ended up solving my problem by going into the bin folder for my project, deleting the NUnit framework dll files and then re-running the tests. Worked like a charm.
So I've downloaded VS2012 RTM and upgraded my Coded UI test projects to the VS2012 versions.
The reason I have done this is because I was having issues with VS2010 Coded UI tools due to partial support of IE9.
So I ran the tests from my developer machine both locally and remotely (making use of the new Visual Studio agent tools), and the tests worked successfully. Presumably, the reason my tests now work is that VS2012 upgraded them so that they now work with IE9, right?
So - this is my problem. I want to automate these tests, by launching them from our build machine - but our build machine still runs on VS2010, and for the time-being this is not going to change.
So I checked in my new VS2012 Coded UI test solution into TFS and queued a new build - so the build machine built my solution. And the build was successful. All good there.
So, next I created a new test case in Microsoft Test Manager and associated with the ordered test list in my new solution. Then I launched the tests (using the existing VS2010 test controller) on my remote test environment (which has the existing VS2010 test agent tools).
But the tests failed - the same issues that affected my tests with the VS2010 Coded UI tests (due to lack of full support for IE9)
Why did they fail?
Do I need the new VS 2012 agent tools for my test controller and agents? Must I build my solution in VS2012?
Ideally, I don't want to have to install VS2012 RTM on my build machine - I want to do the minimum possible to get my tests working and automating.
Is there a way around this?
Your coded ui tests reference assemblies that come with VS installation or your agent's installation (the assemblies defininf the WpfControl, Mouse, Keyboard, Playback and other classes).
So if you run it on a build/test machine with older versions of the dll's the same issues are going to stay, the tests will not start using the newer fixed assemblies thatt were shipped with VS2012.
As a temporary workaround you can check what dll's you reference and make sure that the build process puts them in the same directory with your tests assembly. That way the lookup for them will find the in the current directory without using the /path and finding them in the VS installation directory.
I was just upgrading my project from sdk 1.3 to 1.7 and I noticed that now when I build the application the package(cspkg) is not created with the build. I have go click on publish to create a package. Is there a way to tell VS to create a package everytime a build is triggered.
Also using msbuild is there a way to do the same thing. I have multiple projects under a solution, Most of which are just libraries and then there is this azure app. Is there a way to specify a single msbuild statement with params to tell the azure app to create the package as well as build the other projects. Also when I specify debug the debug package should be created and when I specify release switch the release package should be created.
How could I do the same thing on my build server as well where I have a .proj file which specifies the sln to build. How could I mention a switch to build the package there.
Thanks,
Kunal
You can configure CSPACK command (Be sure to have CSPACK.exe launched from SDK 1.7 Path otherwise you will get some schema related errors with SDK 1.6 project) as Post Build event in your Windows Azure Application Build settings. This way when you will build, after successful build CSPACK command will run and package your application. Same way you can configure your MSBuild configuration. I just tested and it worked for me.
Visit this MSDN article on packaging a cloud service to learn more.
You can do this using msbuild as well. See the Resolution section of this question.