Testing using an upgraded VS2012 solution, but building in VS2010 - coded-ui-tests

So I've downloaded VS2012 RTM and upgraded my Coded UI test projects to the VS2012 versions.
The reason I have done this is because I was having issues with VS2010 Coded UI tools due to partial support of IE9.
So I ran the tests from my developer machine both locally and remotely (making use of the new Visual Studio agent tools), and the tests worked successfully. Presumably, the reason my tests now work is that VS2012 upgraded them so that they now work with IE9, right?
So - this is my problem. I want to automate these tests, by launching them from our build machine - but our build machine still runs on VS2010, and for the time-being this is not going to change.
So I checked in my new VS2012 Coded UI test solution into TFS and queued a new build - so the build machine built my solution. And the build was successful. All good there.
So, next I created a new test case in Microsoft Test Manager and associated with the ordered test list in my new solution. Then I launched the tests (using the existing VS2010 test controller) on my remote test environment (which has the existing VS2010 test agent tools).
But the tests failed - the same issues that affected my tests with the VS2010 Coded UI tests (due to lack of full support for IE9)
Why did they fail?
Do I need the new VS 2012 agent tools for my test controller and agents? Must I build my solution in VS2012?
Ideally, I don't want to have to install VS2012 RTM on my build machine - I want to do the minimum possible to get my tests working and automating.
Is there a way around this?

Your coded ui tests reference assemblies that come with VS installation or your agent's installation (the assemblies defininf the WpfControl, Mouse, Keyboard, Playback and other classes).
So if you run it on a build/test machine with older versions of the dll's the same issues are going to stay, the tests will not start using the newer fixed assemblies thatt were shipped with VS2012.
As a temporary workaround you can check what dll's you reference and make sure that the build process puts them in the same directory with your tests assembly. That way the lookup for them will find the in the current directory without using the /path and finding them in the VS installation directory.

Related

TFS Tests do not match framework settings

I am attempting to move a solution from TFS 2012 to TFS 2018 SP2RC2 but I can't get the unit tests to run correctly. All projects have been re-targeted to 4.7.1 and are built as x86 platform. We have a testsettings file that supplies nothing but deployment items. I am using the new VSTest Platform Installer task (as directed by MS) and the VS Test Task. At the start of the test run I get the following message:
Test run will use DLL(s) built for framework .NETFramework,Version=v4.5 and platform X86. Following DLL(s) do not match framework/platform settings.
So all of the test are skipped as they target 4.7.1. Where is this 4.5 setting coming from? I cannot find it specified anywhere and can't figure out how to change it.
There is a /Framework: parameter that you can specify to VSTest.
In your case, you should specify /Framework:.NETFramework,Version=v4.7.1
See more at https://msdn.microsoft.com/en-us/library/jj155796.aspx?f=255&mspperror=-2147217396
To add this parameter in a Azure DepOps yaml pipeline, use the otherConsoleOptions argument
- task: VSTest#2
otherConsoleOptions: '/Framework:.NETFramework,Version=v4.7.1'
The fix I found for this in Visual Studio is way easier than I thought:
Exit all instances of VS
Open your project folder in Windows Explorer, find the .vs folder, delete it
Restart VS, the folder rebuilds itself, tests work again.
Apparently there are some settings that the NUnit plugin stashes in this folder and they are in binary so you can't edit them. This happened to me after I updated to NUnit3TestAdapter version 3.17.
I had trouble running a test project developed on another machine where no tests where able to be run using the MSTest test runner. In addition to message in your question I also got the messages:
Make sure that test discoverer & executors are registered and platform
& framework version settings are appropriate and try again.
Discover test finished: 0 found
In my case I resolved it by unloading and updating the .csproj file adding the following import directly under the root Project tag.
<Import Project="..\packages\MSTest.TestAdapter.1.3.2\build\net45\MSTest.TestAdapter.props" Condition="Exists('..\packages\MSTest.TestAdapter.1.3.2\build\net45\MSTest.TestAdapter.props')" />
Hope this answer will help save you some time.
I got this error using Visual Studio 2019 with NUnit 3.12.0:
Test run will use DLL(s) built for framework
.NETFramework,Version=v4.5 and platform X86. Following DLL(s) do not
match framework/platform settings.
Project.UnitTests.dll is built for Framework 4.5.2
and Platform AnyCPU.
Installed NUnit3TestAdapter 3.13.0 and then everything started working. Did not need to modify Framework version or CPU settings.
Had the exact same issue as Bill, working on legacy code in VS2019, nothing else had worked. I simply changed the Run Settings to use the Auto Detect.
In the menu go to Test -> Configure Run Settings -> Auto Detect runsettings File
For me skipping tests in local VS2022 run was related to testsettings file. I deleted .vs folder of my project then selected
Test ->Configure Run Settings -> Select Autodetect run settings file
After that I can debug unit tests
TLDR: If you use *.runsettings file for your test projects, try removing TargetFrameworkVersion node
I had similar problem but with .Net Core
Test run will use DLL(s) built for framework
.NETFramework,Version=v4.0 and platform X64. Following DLL(s) do not
match framework/platform settings. MyProject.Tests.dll is
built for Framework .NETCoreApp,Version=v3.1 and Platform AnyCPU.
Turns out there was another problem. For my other .NET Framework test projects I had defined *.runsettings file
And in this file I had
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<!-- Parameters used by tests at runtime -->
<TestRunParameters>
<Parameter name="ConnectionString" value="Data Source=localhost;Initial Catalog=MyDatabase;Integrated Security=True" />
</TestRunParameters>
<RunConfiguration>
<!-- Framework35 | [Framework40] | Framework45 | FrameworkCore10-->
<TargetFrameworkVersion>Framework40</TargetFrameworkVersion>
</RunConfiguration>
</RunSettings>
Supported values for TargerFrameworkVersions based on https://learn.microsoft.com/en-us/visualstudio/test/configure-unit-tests-by-using-a-dot-runsettings-file?view=vs-2019
are:
FrameworkCore10 for .NET Core sources, FrameworkUap10 for UWP-based
sources, Framework45 for .NET Framework 4.5 and higher, Framework40
for .NET Framework 4.0, and Framework35 for .NET Framework 3.5.
But for Core 3.1, FrameworkCore10 doesnt work!
So I removed node TargetFrameworkVersion completely
I had the same problem in VS 2019 on a legacy project.
Restarting Visual Studio did not help, nor did build as Release then rebuild as Debug, nor did deleting the .vs folder. I did those things and still was unable to run tests, getting a message like the OP's during test detection each time. I mention these here for completeness.
I deleted the LocalTestRun.testrunconfig file and the *.vsmdi file from Solution Explorer, cleaned and rebuilt the solution, and the unit tests worked again.
I was running into the same issue with the Visual Studio Test Task version 2 in an Azure Devops 2019 pipeline after a .csproj test project was upgraded from .NetFramework 4.6.1 to .NetCore 3.1.
The fix for us was to be more specific when specifying the test files. The default of **\*test*.dll was finding several other assemblies with "test" in their names that didn't exist before the upgrade. Using an explicit **\<MyTestAssemblyName>.dll fixed the issue.

NUnit has stopped working in Visual Studio

I am running NUnit 2.6.3 in VS2012 with NUnit Test Adapater 1.1.0.0, on 64 bit Windows 7. The project is C# winforms.
My project's unit tests have stopped running from within VS. I can launch NUnit directly and run the project OK, but running or debugging tests from within VS just greys them out with this message in the output window:
No test is available in
[Your Output Path].
Make sure that installed test discoverers & executors, platform &
framework version settings are appropriate and try again.
Any pointers of what that means? Where can I find the "platform and framework version settings" - does it refer to NUnit settings, project settings, VS settings, or one of the NUnit plugin settings?
I think that it's related to me updating to the latest version of the Test Adapter, but unfortunately I'm not sure :(
Have fixed it by:
- Updating my project target to .NET framework 4.5 from 3.5 (which my project was in - not my choice!). I cannot find any documentation stating this as a limitation, so perhaps it's a bug.
(I have also changed to using NuGet to install NUnit to the project instead of the VS plugin, but I'm not sure that had an effect).
I had a similar problem but when I tried to run my tests just showed that yellow inconclusive circle.
I ended up solving my problem by going into the bin folder for my project, deleting the NUnit framework dll files and then re-running the tests. Worked like a charm.

How can we use VS2012 Static Analysis Rules with FxCop 10 Standalone

We're struggling to get some rules that work fine in the VS2012 code analysis tool, to appear in FxCop 10.
Specifically we're trying to get CA2100 - "Review sql queries for security vulnerabilities"
We've tried adding the DLLs from VS2012 (11) but it doesn't seem to work.
Our goal is to ultimately run the VS2012 ruleset as part of a TeamCity build, so we'll probably end up with the cmd line version, but our build agents do not have vs installed and hence we need to use the standalone version.
Any help much appreciated
dan
The VS 2012 rule assemblies are compiled against a different version of the FxCop core assemblies than FxCop 10 (which is the same as that used for VS 2010), so no. However, if your goal is to run this as part of an automated build, the version of fxcopcmd.exe (plus its dependencies) found in your VS 2012 installation should work fine on your build server. Licensing considerations may, however, apply.

NUnit not loading latest version of a Console Application

I am re-writing a console application and trying to use Unit Tests. I'm using NUnit, C#, .NET Framework 4 in Visual Studio 2010 on a Windows 7 PC.
When I change code within the application my unit tests don't pick up the latest changes. I can see that NUnit is reloading but for some reason it's not getting the latest version. The only way to get NUnit to work with the latest version is to remove the reference to the Console app, add it back and compile.
Is there a way to avoid doing this? I think the problem is that the the console app is compiled as an EXE, whereas with most applications I've unit-tested they have tended to be DLLs.
EDIT
I had hoped that this question: Unit Testing a Console Application inside Visual Studio contained the answer. If you look at the accepted answer this was the way I'd been proceeding but my test project wasn't picking up the changes. I've now created a class library and I'm applying my tests to that. My console app will simply load, then call, this class library.
Second Edit
I've now got my code to a stage where I can test functionality and I've noticed the same problem with my console app. Although it 'says' it is referencing the compiled DLL, it only picks up the latest change when I delete and re-attach the reference. So, I'm thinking that the problem isn't necessarily connected to NUnit, rather an issue/quirk with console apps.

Is code coverage available in VS2012 for metro-style apps?

I am using Windows 8 Pro and Visual Studio 2012 Ultimate (both RTM).
I create a Metro-style class library, and then a corresponding unit test library. Both target WinRT (not the .NET framework).
I can run unit tests fine, but when I attempt to "Analyze code coverage for all tests", the output window comes up with the results of the rebuild, and that's it. The Code Coverage Results window is all grayed out.
From reading on the web, some have alluded to the idea that code coverage is not enabled for WinRT assemblies, but I cannot confirm. I have also tried creating a .testsettings file, like was needed in VS2010 and explicitly turn on code coverage, and I got the same results.
Can anyone confirm or deny that code coverage is not available for WinRT-targeted assemblies, in VS2012 RTM?? I'm just trying to figure out if I'm missing some setting - or if it's not possible.
It's not possible at this stage due to the sandboxed nature of WinRT apps.
P.S. It's on the list of things the team is looking to resolve in the near future (no, I don't have a timeframe for it)

Resources