3

I have below error when try to execute integration tests under VSTS CD-task Visual Studio Test.

No test is available in D:\a\r1\a\SiteSearchFramework\e2e\XXX.SiteSearch.EndToEnd.Integration.Tests.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again

I tried few options in below solution, didn't work though.

Solution-1

Logs:

2018-12-30T01:15:07.5727307Z ##[section]Starting: VsTest - End2End Tests 2018-12-30T01:15:07.5735098Z ============================================================================== 2018-12-30T01:15:07.5735255Z Task : Visual Studio Test 2018-12-30T01:15:07.5735424Z Description : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest, xUnit, NUnit, Chutzpah (for JavaScript tests using QUnit, Mocha and Jasmine), etc. can be run. Tests can be distributed on multiple agents using this task (version 2). 2018-12-30T01:15:07.5735595Z Version : 2.144.9 2018-12-30T01:15:07.5735680Z Author : Microsoft Corporation 2018-12-30T01:15:07.5735860Z Help : [More information](https://go.microsoft.com/fwlink/?LinkId=835764) 2018-12-30T01:15:07.5735989Z ============================================================================== 2018-12-30T01:15:08.7136918Z SystemVssConnection exists true 2018-12-30T01:15:08.7303782Z SystemVssConnection exists true 2018-12-30T01:15:08.7401053Z Running tests using vstest.console.exe runner. 2018-12-30T01:15:08.7401220Z ====================================================== 2018-12-30T01:15:08.7401424Z Test selector : Test assemblies 2018-12-30T01:15:08.7401563Z Test filter criteria : null 2018-12-30T01:15:08.7401702Z Search folder : D:\a\r1\a 2018-12-30T01:15:08.8673368Z Run in parallel : false 2018-12-30T01:15:08.8673783Z Run in isolation : false 2018-12-30T01:15:08.8674157Z Path to custom adapters : D:\a\r1\a 2018-12-30T01:15:08.8675167Z Other console options : null 2018-12-30T01:15:08.8675533Z Code coverage enabled : false 2018-12-30T01:15:08.8676084Z Diagnostics enabled : true 2018-12-30T01:15:08.8676957Z SystemVssConnection exists true 2018-12-30T01:15:08.8680294Z Run the tests locally using vstest.console.exe 2018-12-30T01:15:08.8680531Z ======================================================== 2018-12-30T01:15:08.8680903Z Test selector : Test assemblies 2018-12-30T01:15:08.8681301Z Test assemblies : **\*\*.EndToEnd.Integration.Tests.dll 2018-12-30T01:15:08.8681656Z Test filter criteria : null 2018-12-30T01:15:08.8682253Z Search folder : D:\a\r1\a 2018-12-30T01:15:08.8682609Z Run settings file : D:\a\r1\a 2018-12-30T01:15:08.8683225Z Run in parallel : false 2018-12-30T01:15:08.8683646Z Run in isolation : false 2018-12-30T01:15:08.8686856Z Path to custom adapters : D:\a\r1\a 2018-12-30T01:15:08.8689056Z Other console options : null 2018-12-30T01:15:08.8689537Z Code coverage enabled : false 2018-12-30T01:15:08.8689740Z Diagnostics enabled : false 2018-12-30T01:15:08.8690648Z Rerun failed tests: false 2018-12-30T01:15:09.1368849Z VisualStudio version selected for test execution : toolsInstaller 2018-12-30T01:15:09.4782611Z ======================================================== 2018-12-30T01:15:09.4808305Z ====================================================== 2018-12-30T01:15:09.4945232Z [command]C:\hostedtoolcache\windows\VsTest\16.0.0-preview-20181205-02\x64\tools\net451\Common7\IDE\Extensions\TestPlatform\vstest.console.exe @D:\a\_temp\5987b5c1-0bd0-11e9-ab8a-2da31cef68f3.txt 2018-12-30T01:15:09.6112080Z Microsoft (R) Test Execution Command Line Tool Version 16.0.0-preview-20181205-02 2018-12-30T01:15:09.6112672Z Copyright (c) Microsoft Corporation. All rights reserved. 2018-12-30T01:15:09.6112863Z 2018-12-30T01:15:09.6577532Z vstest.console.exe 2018-12-30T01:15:09.6578059Z "D:\a\r1\a\SiteSearchFramework\e2e\XXX.SiteSearch.EndToEnd.Integration.Tests.dll" 2018-12-30T01:15:09.6578280Z /logger:"trx" 2018-12-30T01:15:09.6578485Z /TestAdapterPath:"D:\a\r1\a" 2018-12-30T01:15:11.6317213Z Starting test execution, please wait... 2018-12-30T01:15:14.2322930Z No test is available in D:\a\r1\a\SiteSearchFramework\e2e\XXX.SiteSearch.EndToEnd.Integration.Tests.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again. 2018-12-30T01:15:14.3430162Z Results File: D:\a\r1\a\TestResults\VssAdministrator_fv-az137_2018-12-30_01_15_14.trx 2018-12-30T01:15:14.3453569Z 2018-12-30T01:15:15.9470428Z ##[warning]Invalid results file. Make sure the result format of the file 'D:\a\r1\a\TestResults\VssAdministrator_fv-az137_2018-12-30_01_15_14.trx' matches 'VSTest' test results format. 2018-12-30T01:15:15.9596641Z ##[section]Async Command Start: Publish test results 2018-12-30T01:15:15.9596804Z ##[section]Async Command End: Publish test results 2018-12-30T01:15:15.9597264Z ##[section]Finishing: VsTest - End2End Tests

enter image description here

Test Project (XXX.SiteSearch.EndToEnd.Integration.Tests) references :

enter image description here

Shabar
  • 2,617
  • 11
  • 57
  • 98
  • Does the artifact containing your tests have the appropriate test runner and framework assemblies alongside the test assemblies? – Daniel Mann Dec 30 '18 at 01:31
  • @DanielMann, Test runner : `C:\hostedtoolcache\windows\VsTest\16.0.0-preview-20181205-02\x64\tools\net451\Common7\IDE\Extensions\TestPlatform\vstest.console.exe` and Test Assemblies : `D:/a/1/a/src/XXX.SiteSearch.EndToEnd.Integration.Tests/bin/Release/XXX.SiteSearch.EndToEnd.Integration.Tests.dll` . Only doubt is should test Assemblies in `D:\a\r1\a..` path? If so where to change this? – Shabar Dec 30 '18 at 01:44
  • 1
    VSTest is the test runner, but it's not the test framework. You need more than just an assembly with tests in order to run it. When you build your test project, there are a bunch of other assemblies that are required to actually discover and run the tests. You have to publish **all** of them as part of the artifact. What test framework are you using? Did you install the appropriate NuGet packages for discovering and running those tests? Did the contents of those NuGet packages end up published in your artifact? If not, that's the problem – Daniel Mann Dec 30 '18 at 02:24
  • @DanielMann All the related assemblies used in integration tests added to the questions. Are you able to give an example `build.yaml` code snippet to show how I can add them. – Shabar Dec 30 '18 at 05:51
  • @SMPH that is not part of the yaml but part of the csproj of the testproject. select the xunit-dependencies and ensure that in the proeprties 'Copy local' is set to true – D.J. Dec 30 '18 at 07:06
  • @D.J. Integration tests runs fine when execute locally pointing the environment through `app.config` using `VS2107` IDE. But here what I after is run them in `VSTS` via CD pipeline. If you get the question correctly? Not sure what you meant by "ensure that in the properties 'Copy local' is set to true" – Shabar Dec 30 '18 at 10:54
  • select the xunit-dependencies -> right click on them -> properties -> Check if 'Copy local' is set to true. @Daniel Mann asked if you installed the appropriate nuget packages, which packages have you installed ? you might be missing one like this -> https://www.nuget.org/packages/xunit.runner.visualstudio/ – D.J. Dec 30 '18 at 20:02
  • @D.J. I have checked all the property settings dependencies, all set to `Copy local = true`. To the second point @Daniel Mann pointed out, All appropriate nuget packages are there in the solution. Sorry not sure whether I have to do anything in VSTS CI/CD pipeline to add those? – Shabar Dec 30 '18 at 22:44
  • One more doubt I have, i.e. in `build.yaml` I have `task: PublishBuildArtifacts@1 PathtoPublish: 'D:/a/1/a/src/XXX.SiteSearch.EndToEnd.Integration.Tests/bin/Release/AGL.SiteSearch.EndToEnd.Integration.Tests.dll ArtifactName: e2e' ` and VSTS CD logs it says `No test is available in D:\a\r1\a\SiteSearchFramework\e2e\XXX.SiteSearch.EndToEnd.Integration.Tests.dll` Is this conflicting? – Shabar Dec 31 '18 at 01:00
  • 2
    @SMPH You have to publish **the entire folder containing your tests** to pick up the dependencies, not just the test assembly. – Daniel Mann Dec 31 '18 at 02:13
  • @DanielMann I added entire folder. After the build script I checked the artefact and it contains everything but still in CD pipeline I get the same message in the logs with different context `No test is available in D:\a\r1\a\SiteSearchFramework\e2e\bin\Release\XXX.SiteSearch.EndToEnd.Integration.Tests.dll D:\a\r1\a\SiteSearchFramework\e2e\obj\Release\XXX.SiteSearch.EndToEnd.Integration.Tests.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.` I can see `.Integration.Tests.dll` in both `bin` & `obj` folders – Shabar Dec 31 '18 at 04:55
  • 3
    @SMPH Is this nuget package installed and copied over to your build artifact outputs? https://www.nuget.org/packages/xunit.runner.visualstudio/ – Daniel Mann Dec 31 '18 at 19:09
  • @DanielMannIn CD pipeline I have two tasks one is `Visual Studio Test Platform Installer` which describe "Acquires the test platform from nuget.org or the tools cache. Satisfies the ‘vstest’ demand and can be used for running tests and collecting diagnostic data using the Visual Studio Test task. Test platform package on NuGet" and the other is `Visual Studio Test` says "Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest, xUnit, etc. can be run." Is that cover? – Shabar Dec 31 '18 at 22:53
  • @DanielMann If above doesn't help, Can you pl give me sample code to do the step what you mentioned i.e. "nuget package installed and copied over to your build artifact outputs"? – Shabar Dec 31 '18 at 23:19
  • @SMPH Look at your project's NuGet dependencies. Is the package I mentioned installed? If not, install it. The problem you're encountering is that the test runner can't find an appropriate test adapter, so it can't discover the tests in your test assembly. – Daniel Mann Dec 31 '18 at 23:37
  • @DanielMann It is installed. Otherwise how can I run these tests locally using `VS2107 IDE`? – Shabar Dec 31 '18 at 23:51
  • 1
    @DanielMann I added `xunit.runner.visualstudio` nuget to the `E2E` project under the main solution. Hooray!!!! it worked. Thanks guys. If you can add this as a solution it will be great. But still have the previous question which is "how can I run these tests locally successfully using VS2107 IDE without `xunit.runner.visualstudio` in the `e2e` project? – Shabar Jan 02 '19 at 01:31
  • This gives the solution for my question in my above comment https://developercommunity.visualstudio.com/content/problem/416493/publish-your-test-assemblies-as-separate-artefact.html – Shabar Jan 12 '19 at 01:31

1 Answers1

4

Add xunit.runner.visualstudio to the solution and make sure it was copied to build artifact output.

Shabar
  • 2,617
  • 11
  • 57
  • 98