Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Not all test results shown in test explorer, when multiple tests fail and fixture contains 10 or more #1625

Closed
JoeRobich opened this issue Nov 20, 2024 · 6 comments
Closed

Not all test results shown in test explorer, when multiple tests fail and fixture contains 10 or more #1625

JoeRobich opened this issue Nov 20, 2024 · 6 comments
Assignees
@ocallesp
Labels
area-test Test discovery, execution, debugging bug Something isn't working

Comments

@JoeRobich
Copy link
Member

This issue originally reported at dotnet/vscode-csharp#7811

Type: Bug

Issue Description

I have an NUnit fixture with 10 tests in them. If more than 1 of them fails (in my example, all fail), and there are at least 10 tests in total, then the test explorer will not show the run results for all tests. In the output of the test explorer, however, the test explorer indicates that it saw 10 tests and that they failed.

Steps to Reproduce

Create an NUnit project (NUnit 4.2.2, NUnit3TestAdapter 4.6.0, Microsoft.NET.Test.Sdk 17.6.0) with the following test file:

using NUnit.Framework;

namespace Tests;

public class VSCodeTestExplorerBugExample
{
    [Test]
    public void Test1()
    {
        Assert.Fail();
    }

    [Test]
    public void Test2()
    {
        Assert.Fail();
    }

    [Test]
    public void Test3()
    {
        Assert.Fail();
    }

    [Test]
    public void Test4()
    {
        Assert.Fail();
    }

    [Test]
    public void Test5()
    {
        Assert.Fail();
    }

    [Test]
    public void Test6()
    {
        Assert.Fail();
    }

    [Test]
    public void Test7()
    {
        Assert.Fail();
    }

    [Test]
    public void Test8()
    {
        Assert.Fail();
    }

    [Test]
    public void Test9()
    {
        Assert.Fail();
    }

    [Test]
    public void Test10()
    {
        Assert.Fail();
    }
}

Then run all tests in the file at once, either by:

  • Clicking "Run test" on the VSCodeTestExplorerBugExample test group
  • Executing "Test: Run tests in current file"
  • Executing "Test: Run all tests"

Expected Behavior

All tests in the test explorer show failure

Actual Behavior

Only some tests show failure, the other tests do not show that they have run. In my case, Test1, Test9 and Test10 show failure.
If those other tests have run before, they will continue to show their previous test result; e.g. if Test5 passed before but has been set to fail, it will continue to show "pass" in the test explorer.

Image

C# Dev Kit - Test Explorer

========== Starting test run ==========
NUnit Adapter 4.6.0.0: Test execution started
Running selected tests in /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/bin/Debug/net8.0/VSCodeTestExplorerBugExample.dll
   NUnit3TestExecutor discovered 10 of 10 NUnit test cases using Current Discovery mode, Non-Explicit run
NUnit Adapter 4.6.0.0: Test execution complete
========== Test run finished: 10 Tests (0 Passed, 10 Failed, 0 Skipped) run in 162.2 ms ==========

C# log

```2024-11-20 14:58:34.100 [info] [LanguageServerHost] No request parameters given, using default language handler 2024-11-20 14:58:34.100 [info] [LanguageServerHost] [01:58:34.097][Start]workspace/buildOnlyDiagnosticIds 2024-11-20 14:58:34.100 [info] [LanguageServerHost] [01:58:34.098][End]workspace/buildOnlyDiagnosticIds 2024-11-20 14:58:34.301 [info] [LanguageServerHost] /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs found in workspace Host 2024-11-20 14:58:34.301 [info] [LanguageServerHost] /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs found in workspace Host 2024-11-20 14:58:34.302 [info] [LanguageServerHost] /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs found in workspace Host 2024-11-20 14:58:34.302 [info] [LanguageServerHost] /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs found in workspace Host 2024-11-20 14:58:34.302 [info] [LanguageServerHost] /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs found in workspace Host 2024-11-20 14:58:34.302 [info] [LanguageServerHost] /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs found in workspace Host 2024-11-20 14:58:34.302 [info] [LanguageServerHost] /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs found in workspace Host 2024-11-20 14:58:34.302 [info] [LanguageServerHost] /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs found in workspace Host 2024-11-20 14:58:34.302 [info] [LanguageServerHost] /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs found in workspace Host 2024-11-20 14:58:34.302 [info] [LanguageServerHost] [01:58:34.300][Start]textDocument/diagnostic 2024-11-20 14:58:34.302 [info] [LanguageServerHost] PublicDocumentPullDiagnosticsHandler(category: WorkspaceDocumentsAndProject) started getting diagnostics 2024-11-20 14:58:34.302 [info] [LanguageServerHost] previousResults.Length=0 2024-11-20 14:58:34.302 [info] [LanguageServerHost] Processing 0 documents 2024-11-20 14:58:34.302 [info] [LanguageServerHost] Microsoft.CodeAnalysis.LanguageServer.Handler.Diagnostics.Public.PublicDocumentPullDiagnosticsHandler finished getting diagnostics 2024-11-20 14:58:34.303 [info] [LanguageServerHost] [01:58:34.300][End]textDocument/diagnostic 2024-11-20 14:58:34.303 [info] [LanguageServerHost] [01:58:34.300][Start]textDocument/diagnostic 2024-11-20 14:58:34.303 [info] [LanguageServerHost] PublicDocumentPullDiagnosticsHandler(category: DocumentCompilerSemantic) started getting diagnostics 2024-11-20 14:58:34.303 [info] [LanguageServerHost] previousResults.Length=1 2024-11-20 14:58:34.303 [info] [LanguageServerHost] Processing 1 documents 2024-11-20 14:58:34.303 [info] [LanguageServerHost] [01:58:34.300][Start]textDocument/diagnostic 2024-11-20 14:58:34.303 [info] [LanguageServerHost] [01:58:34.300][Start]textDocument/diagnostic 2024-11-20 14:58:34.303 [info] [LanguageServerHost] PublicDocumentPullDiagnosticsHandler(category: HotReloadDiagnostics) started getting diagnostics 2024-11-20 14:58:34.303 [info] [LanguageServerHost] Diagnostics were unchanged for DocumentDiagnosticSource: /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs in VSCodeTestExplorerBugExample 2024-11-20 14:58:34.303 [info] [LanguageServerHost] previousResults.Length=0 2024-11-20 14:58:34.303 [info] [LanguageServerHost] Microsoft.CodeAnalysis.LanguageServer.Handler.Diagnostics.Public.PublicDocumentPullDiagnosticsHandler finished getting diagnostics 2024-11-20 14:58:34.304 [info] [LanguageServerHost] PublicDocumentPullDiagnosticsHandler(category: enc) started getting diagnostics 2024-11-20 14:58:34.304 [info] [LanguageServerHost] previousResults.Length=1 2024-11-20 14:58:34.304 [info] [LanguageServerHost] Processing 0 documents 2024-11-20 14:58:34.304 [info] [LanguageServerHost] Microsoft.CodeAnalysis.LanguageServer.Handler.Diagnostics.Public.PublicDocumentPullDiagnosticsHandler finished getting diagnostics 2024-11-20 14:58:34.304 [info] [LanguageServerHost] [01:58:34.300][End]textDocument/diagnostic 2024-11-20 14:58:34.304 [info] [LanguageServerHost] [01:58:34.300][Start]textDocument/diagnostic 2024-11-20 14:58:34.304 [info] [LanguageServerHost] PublicDocumentPullDiagnosticsHandler(category: DocumentAnalyzerSemantic) started getting diagnostics 2024-11-20 14:58:34.304 [info] [LanguageServerHost] previousResults.Length=1 2024-11-20 14:58:34.304 [info] [LanguageServerHost] Processing 1 documents 2024-11-20 14:58:34.304 [info] [LanguageServerHost] Processing 1 documents 2024-11-20 14:58:34.304 [info] [LanguageServerHost] Diagnostics were unchanged for DocumentDiagnosticSource: /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs in VSCodeTestExplorerBugExample 2024-11-20 14:58:34.304 [info] [LanguageServerHost] Microsoft.CodeAnalysis.LanguageServer.Handler.Diagnostics.Public.PublicDocumentPullDiagnosticsHandler finished getting diagnostics 2024-11-20 14:58:34.304 [info] [LanguageServerHost] [01:58:34.300][End]textDocument/diagnostic 2024-11-20 14:58:34.304 [info] [LanguageServerHost] [01:58:34.300][End]textDocument/diagnostic 2024-11-20 14:58:34.304 [info] [LanguageServerHost] Diagnostics were unchanged for OpenDocumentSource: /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs in VSCodeTestExplorerBugExample 2024-11-20 14:58:34.304 [info] [LanguageServerHost] [01:58:34.300][Start]textDocument/diagnostic 2024-11-20 14:58:34.304 [info] [LanguageServerHost] [01:58:34.300][Start]textDocument/diagnostic 2024-11-20 14:58:34.304 [info] [LanguageServerHost] PublicDocumentPullDiagnosticsHandler(category: NonLocal) started getting diagnostics 2024-11-20 14:58:34.304 [info] [LanguageServerHost] Microsoft.CodeAnalysis.LanguageServer.Handler.Diagnostics.Public.PublicDocumentPullDiagnosticsHandler finished getting diagnostics 2024-11-20 14:58:34.304 [info] [LanguageServerHost] PublicDocumentPullDiagnosticsHandler(category: syntax) started getting diagnostics 2024-11-20 14:58:34.304 [info] [LanguageServerHost] previousResults.Length=0 2024-11-20 14:58:34.304 [info] [LanguageServerHost] previousResults.Length=1 2024-11-20 14:58:34.304 [info] [LanguageServerHost] Processing 0 documents 2024-11-20 14:58:34.304 [info] [LanguageServerHost] Microsoft.CodeAnalysis.LanguageServer.Handler.Diagnostics.Public.PublicDocumentPullDiagnosticsHandler finished getting diagnostics 2024-11-20 14:58:34.304 [info] [LanguageServerHost] [01:58:34.300][End]textDocument/diagnostic 2024-11-20 14:58:34.304 [info] [LanguageServerHost] Processing 1 documents 2024-11-20 14:58:34.304 [info] [LanguageServerHost] [01:58:34.300][Start]textDocument/diagnostic 2024-11-20 14:58:34.304 [info] [LanguageServerHost] PublicDocumentPullDiagnosticsHandler(category: DocumentAnalyzerSyntax) started getting diagnostics 2024-11-20 14:58:34.304 [info] [LanguageServerHost] previousResults.Length=1 2024-11-20 14:58:34.305 [info] [LanguageServerHost] Processing 1 documents 2024-11-20 14:58:34.305 [info] [LanguageServerHost] Diagnostics were unchanged for DocumentDiagnosticSource: /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs in VSCodeTestExplorerBugExample 2024-11-20 14:58:34.305 [info] [LanguageServerHost] Diagnostics were unchanged for DocumentDiagnosticSource: /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs in VSCodeTestExplorerBugExample 2024-11-20 14:58:34.305 [info] [LanguageServerHost] Microsoft.CodeAnalysis.LanguageServer.Handler.Diagnostics.Public.PublicDocumentPullDiagnosticsHandler finished getting diagnostics 2024-11-20 14:58:34.305 [info] [LanguageServerHost] Microsoft.CodeAnalysis.LanguageServer.Handler.Diagnostics.Public.PublicDocumentPullDiagnosticsHandler finished getting diagnostics 2024-11-20 14:58:34.305 [info] [LanguageServerHost] [01:58:34.300][End]textDocument/diagnostic 2024-11-20 14:58:34.305 [info] [LanguageServerHost] [01:58:34.300][End]textDocument/diagnostic 2024-11-20 14:58:34.305 [info] [LanguageServerHost] [01:58:34.300][Start]textDocument/diagnostic 2024-11-20 14:58:34.305 [info] [LanguageServerHost] [01:58:34.300][End]textDocument/diagnostic 2024-11-20 14:58:34.305 [info] [LanguageServerHost] PublicDocumentPullDiagnosticsHandler(category: XamlDiagnostics) started getting diagnostics 2024-11-20 14:58:34.305 [info] [LanguageServerHost] previousResults.Length=0 2024-11-20 14:58:34.305 [info] [LanguageServerHost] Processing 0 documents 2024-11-20 14:58:34.305 [info] [LanguageServerHost] Microsoft.CodeAnalysis.LanguageServer.Handler.Diagnostics.Public.PublicDocumentPullDiagnosticsHandler finished getting diagnostics 2024-11-20 14:58:34.305 [info] [LanguageServerHost] [01:58:34.301][End]textDocument/diagnostic ```

C# LSP Trace Logs

``` [Trace - 2:59:14 PM] Sending request 'textDocument/_vs_getProjectContexts - (1899)'. Params: { "_vs_textDocument": { "uri": "file:///home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs" } }

[Trace - 2:59:14 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs found in workspace Host"
}

[Trace - 2:59:14 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] [01:59:14.796][Start]textDocument/_vs_getProjectContexts"
}

[Trace - 2:59:14 PM] Received response 'textDocument/_vs_getProjectContexts - (1899)' in 5ms.
Result: {
"_vs_projectContexts": [
{
"_vs_label": "VSCodeTestExplorerBugExample",
"_vs_id": "5b1f9d31-7f5f-4440-94ff-221b420ccde1|VSCodeTestExplorerBugExample",
"_vs_kind": 2,
"_vs_is_miscellaneous": false
}
],
"_vs_defaultIndex": 0
}

[Trace - 2:59:14 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] [01:59:14.797][End]textDocument/_vs_getProjectContexts"
}

[Trace - 2:59:16 PM] Sending request 'workspace/buildOnlyDiagnosticIds - (1900)'.
[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] No request parameters given, using default language handler"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] [01:59:16.317][Start]workspace/buildOnlyDiagnosticIds"
}

[Trace - 2:59:16 PM] Received response 'workspace/buildOnlyDiagnosticIds - (1900)' in 4ms.
Result: {
"ids": [
"CS1607",
"CS0169",
"CS0414",
"CS0067",
"CS0649",
"CS0656",
"CS0518",
"CS5001",
"CS0028",
"CS0017",
"CS7022",
"CS1556",
"CS0402",
"CS1558",
"CS1555",
"CS8892",
"CS0148",
"CS8078",
"CS7038",
"CS0204",
"CS0570",
"CS8004",
"CS8006",
"CS8005",
"CS8008",
"CS4007",
"CS8178",
"CS4013",
"CS1969",
"CS9026",
"CS9068",
"CS9144",
"CS9148",
"CS9149",
"CS9153",
"CS9154",
"CS9155",
"CS9156",
"CS9158",
"CS9159",
"CS9160",
"CS9163",
"CS9177",
"CS9178",
"CS9207",
"CS8419",
"CS8420",
"CS9217",
"CA1016",
"CA1014",
"CA1017",
"CA1017",
"CA1724",
"CA1724",
"CA1501",
"CA1502",
"CA1505",
"CA1506",
"CA1509",
"CA1852",
"CA5360",
"CA5399",
"CA5400",
"CA5380",
"CA5381",
"CA5367",
"CA5401",
"CA5402",
"CA2301",
"CA2302",
"CA2321",
"CA2322",
"CA2329",
"CA2330",
"CA2311",
"CA2312",
"CA2327",
"CA2328",
"CA5387",
"CA5388",
"CA5362",
"CA5396",
"CA5391",
"CA5395",
"CA5382",
"CA5383",
"CA1812",
"CA1824",
"RS1022",
"RS1022",
"RS1010",
"RS1011",
"RS1016",
"RS1019",
"RS2002",
"RS2003",
"RS2005",
"RS2006",
"RS2007",
"RS2007",
"RS2007",
"RS2007"
]
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] [01:59:16.318][End]workspace/buildOnlyDiagnosticIds"
}

[Trace - 2:59:16 PM] Sending request 'textDocument/diagnostic - (1901)'.
Params: {
"identifier": "HotReloadDiagnostics",
"textDocument": {
"uri": "file:///home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs"
}
}

[Trace - 2:59:16 PM] Sending request 'textDocument/diagnostic - (1902)'.
Params: {
"identifier": "enc",
"textDocument": {
"uri": "file:///home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs"
},
"previousResultId": "PublicDocumentPullDiagnosticsHandler(category: enc):4"
}

[Trace - 2:59:16 PM] Sending request 'textDocument/diagnostic - (1903)'.
Params: {
"identifier": "syntax",
"textDocument": {
"uri": "file:///home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs"
},
"previousResultId": "PublicDocumentPullDiagnosticsHandler(category: syntax):27"
}

[Trace - 2:59:16 PM] Sending request 'textDocument/diagnostic - (1904)'.
Params: {
"identifier": "XamlDiagnostics",
"textDocument": {
"uri": "file:///home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs"
}
}

[Trace - 2:59:16 PM] Sending request 'textDocument/diagnostic - (1905)'.
Params: {
"identifier": "DocumentAnalyzerSyntax",
"textDocument": {
"uri": "file:///home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs"
},
"previousResultId": "PublicDocumentPullDiagnosticsHandler(category: DocumentAnalyzerSyntax):7"
}

[Trace - 2:59:16 PM] Sending request 'textDocument/diagnostic - (1906)'.
Params: {
"identifier": "NonLocal",
"textDocument": {
"uri": "file:///home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs"
}
}

[Trace - 2:59:16 PM] Sending request 'textDocument/diagnostic - (1907)'.
Params: {
"identifier": "DocumentAnalyzerSemantic",
"textDocument": {
"uri": "file:///home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs"
},
"previousResultId": "PublicDocumentPullDiagnosticsHandler(category: DocumentAnalyzerSemantic):20"
}

[Trace - 2:59:16 PM] Sending request 'textDocument/diagnostic - (1908)'.
Params: {
"identifier": "DocumentCompilerSemantic",
"textDocument": {
"uri": "file:///home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs"
},
"previousResultId": "PublicDocumentPullDiagnosticsHandler(category: DocumentCompilerSemantic):42"
}

[Trace - 2:59:16 PM] Sending request 'textDocument/diagnostic - (1909)'.
Params: {
"identifier": "WorkspaceDocumentsAndProject",
"textDocument": {
"uri": "file:///home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs"
}
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs found in workspace Host"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs found in workspace Host"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs found in workspace Host"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs found in workspace Host"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] [01:59:16.518][Start]textDocument/diagnostic"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] [01:59:16.518][Start]textDocument/diagnostic"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs found in workspace Host"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs found in workspace Host"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] PublicDocumentPullDiagnosticsHandler(category: HotReloadDiagnostics) started getting diagnostics"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] PublicDocumentPullDiagnosticsHandler(category: enc) started getting diagnostics"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs found in workspace Host"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] previousResults.Length=0"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] previousResults.Length=1"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs found in workspace Host"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs found in workspace Host"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] Processing 0 documents"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] Microsoft.CodeAnalysis.LanguageServer.Handler.Diagnostics.Public.PublicDocumentPullDiagnosticsHandler finished getting diagnostics"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] [01:59:16.518][Start]textDocument/diagnostic"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] PublicDocumentPullDiagnosticsHandler(category: WorkspaceDocumentsAndProject) started getting diagnostics"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] previousResults.Length=0"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] Processing 1 documents"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] Processing 0 documents"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] Microsoft.CodeAnalysis.LanguageServer.Handler.Diagnostics.Public.PublicDocumentPullDiagnosticsHandler finished getting diagnostics"
}

[Trace - 2:59:16 PM] Received response 'textDocument/diagnostic - (1901)' in 2ms.
No result returned.

[Trace - 2:59:16 PM] Received response 'textDocument/diagnostic - (1909)' in 2ms.
No result returned.

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] [01:59:16.518][End]textDocument/diagnostic"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] [01:59:16.518][End]textDocument/diagnostic"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] Diagnostics were unchanged for OpenDocumentSource: /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs in VSCodeTestExplorerBugExample"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] [01:59:16.518][Start]textDocument/diagnostic"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] Microsoft.CodeAnalysis.LanguageServer.Handler.Diagnostics.Public.PublicDocumentPullDiagnosticsHandler finished getting diagnostics"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] PublicDocumentPullDiagnosticsHandler(category: DocumentCompilerSemantic) started getting diagnostics"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] previousResults.Length=1"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] Processing 1 documents"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] [01:59:16.518][Start]textDocument/diagnostic"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] PublicDocumentPullDiagnosticsHandler(category: syntax) started getting diagnostics"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] previousResults.Length=1"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] Diagnostics were unchanged for DocumentDiagnosticSource: /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs in VSCodeTestExplorerBugExample"
}

[Trace - 2:59:16 PM] Received response 'textDocument/diagnostic - (1902)' in 2ms.
Result: {
"kind": "unchanged",
"resultId": "PublicDocumentPullDiagnosticsHandler(category: enc):4"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] Microsoft.CodeAnalysis.LanguageServer.Handler.Diagnostics.Public.PublicDocumentPullDiagnosticsHandler finished getting diagnostics"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] [01:59:16.518][End]textDocument/diagnostic"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] Processing 1 documents"
}

[Trace - 2:59:16 PM] Received response 'textDocument/diagnostic - (1908)' in 2ms.
Result: {
"kind": "unchanged",
"resultId": "PublicDocumentPullDiagnosticsHandler(category: DocumentCompilerSemantic):42"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] [01:59:16.518][End]textDocument/diagnostic"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] [01:59:16.518][Start]textDocument/diagnostic"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] PublicDocumentPullDiagnosticsHandler(category: DocumentAnalyzerSemantic) started getting diagnostics"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] previousResults.Length=1"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] [01:59:16.518][Start]textDocument/diagnostic"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] Diagnostics were unchanged for DocumentDiagnosticSource: /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs in VSCodeTestExplorerBugExample"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] PublicDocumentPullDiagnosticsHandler(category: XamlDiagnostics) started getting diagnostics"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] Microsoft.CodeAnalysis.LanguageServer.Handler.Diagnostics.Public.PublicDocumentPullDiagnosticsHandler finished getting diagnostics"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] Processing 1 documents"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] previousResults.Length=0"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] Diagnostics were unchanged for DocumentDiagnosticSource: /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs in VSCodeTestExplorerBugExample"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] Microsoft.CodeAnalysis.LanguageServer.Handler.Diagnostics.Public.PublicDocumentPullDiagnosticsHandler finished getting diagnostics"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] Processing 0 documents"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] Microsoft.CodeAnalysis.LanguageServer.Handler.Diagnostics.Public.PublicDocumentPullDiagnosticsHandler finished getting diagnostics"
}

[Trace - 2:59:16 PM] Received response 'textDocument/diagnostic - (1907)' in 3ms.
Result: {
"kind": "unchanged",
"resultId": "PublicDocumentPullDiagnosticsHandler(category: DocumentAnalyzerSemantic):20"
}

[Trace - 2:59:16 PM] Received response 'textDocument/diagnostic - (1904)' in 3ms.
No result returned.

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] [01:59:16.518][End]textDocument/diagnostic"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] [01:59:16.518][Start]textDocument/diagnostic"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] [01:59:16.518][End]textDocument/diagnostic"
}

[Trace - 2:59:16 PM] Received response 'textDocument/diagnostic - (1903)' in 3ms.
Result: {
"kind": "unchanged",
"resultId": "PublicDocumentPullDiagnosticsHandler(category: syntax):27"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] PublicDocumentPullDiagnosticsHandler(category: NonLocal) started getting diagnostics"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] previousResults.Length=0"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] [01:59:16.518][Start]textDocument/diagnostic"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] PublicDocumentPullDiagnosticsHandler(category: DocumentAnalyzerSyntax) started getting diagnostics"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] previousResults.Length=1"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] [01:59:16.518][End]textDocument/diagnostic"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] Processing 0 documents"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] Microsoft.CodeAnalysis.LanguageServer.Handler.Diagnostics.Public.PublicDocumentPullDiagnosticsHandler finished getting diagnostics"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] Processing 1 documents"
}

[Trace - 2:59:16 PM] Received response 'textDocument/diagnostic - (1906)' in 3ms.
No result returned.

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] [01:59:16.518][End]textDocument/diagnostic"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] Diagnostics were unchanged for DocumentDiagnosticSource: /home/maarten/code/sandbox/VSCodeTestExplorerBugExample/VSCodeTestExplorerBugExample.cs in VSCodeTestExplorerBugExample"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] Microsoft.CodeAnalysis.LanguageServer.Handler.Diagnostics.Public.PublicDocumentPullDiagnosticsHandler finished getting diagnostics"
}

[Trace - 2:59:16 PM] Received response 'textDocument/diagnostic - (1905)' in 3ms.
Result: {
"kind": "unchanged",
"resultId": "PublicDocumentPullDiagnosticsHandler(category: DocumentAnalyzerSyntax):7"
}

[Trace - 2:59:16 PM] Received notification 'window/logMessage'.
Params: {
"type": 5,
"message": "[LanguageServerHost] [01:59:16.518][End]textDocument/diagnostic"
}

</details>

## Environment information ##

**VSCode version**: 1.95.3
**C# Extension**: 2.55.29
**Using OmniSharp**: false


<details><summary>Dotnet Information</summary>
.NET SDK:
 Version:           8.0.110
 Commit:            87a66bb3d1
 Workload version:  8.0.100-manifests.9143487c

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  22.04
 OS Platform: Linux
 RID:         ubuntu.22.04-x64
 Base Path:   /usr/lib/dotnet/sdk/8.0.110/

.NET workloads installed:
 Workload version: 8.0.100-manifests.9143487c
There are no installed workloads to display.

Host:
  Version:      8.0.10
  Architecture: x64
  Commit:       81cabf2857

.NET SDKs installed:
  8.0.110 [/usr/lib/dotnet/sdk]

.NET runtimes installed:
  Microsoft.AspNetCore.App 8.0.10 [/usr/lib/dotnet/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 8.0.10 [/usr/lib/dotnet/shared/Microsoft.NETCore.App]

Other architectures found:
  None

Environment variables:
  Not set

global.json file:
  Not found

Learn more:
  https://aka.ms/dotnet/info

Download .NET:
  https://aka.ms/dotnet/download
</details>
<details><summary>Visual Studio Code Extensions</summary>

|Extension|Author|Version|Folder Name|
|---|---|---|---|
|cmake|twxs|0.0.17|twxs.cmake-0.0.17|
|cmake-tools|ms-vscode|1.19.52|ms-vscode.cmake-tools-1.19.52|
|copilot|GitHub|1.245.0|github.copilot-1.245.0|
|copilot-chat|GitHub|0.22.4|github.copilot-chat-0.22.4|
|cpptools|ms-vscode|1.22.11|ms-vscode.cpptools-1.22.11-linux-x64|
|cpptools-extension-pack|ms-vscode|1.3.0|ms-vscode.cpptools-extension-pack-1.3.0|
|cpptools-themes|ms-vscode|2.0.0|ms-vscode.cpptools-themes-2.0.0|
|csdevkit|ms-dotnettools|1.11.14|ms-dotnettools.csdevkit-1.11.14-linux-x64|
|csharp|ms-dotnettools|2.55.29|ms-dotnettools.csharp-2.55.29-linux-x64|
|debugpy|ms-python|2024.12.0|ms-python.debugpy-2024.12.0-linux-x64|
|git-graph|mhutchie|1.30.0|mhutchie.git-graph-1.30.0|
|gitlens|eamodio|16.0.2|eamodio.gitlens-16.0.2|
|powershell|ms-vscode|2024.4.0|ms-vscode.powershell-2024.4.0|
|python|ms-python|2024.20.0|ms-python.python-2024.20.0-linux-x64|
|vscode-catch2-test-adapter|matepek|4.12.2|matepek.vscode-catch2-test-adapter-4.12.2|
|vscode-clangd|llvm-vs-code-extensions|0.1.31|llvm-vs-code-extensions.vscode-clangd-0.1.31|
|vscode-docker|ms-azuretools|1.29.3|ms-azuretools.vscode-docker-1.29.3|
|vscode-dotnet-runtime|ms-dotnettools|2.2.3|ms-dotnettools.vscode-dotnet-runtime-2.2.3|
|vscode-pylance|ms-python|2024.11.2|ms-python.vscode-pylance-2024.11.2|;

</details>


Extension version: 2.55.29
VS Code version: Code 1.95.3 (f1a4fb101478ce6ec82fe9627c43efbf9e98c813, 2024-11-13T14:50:04.152Z)
OS version: Windows_NT x64 10.0.22631
Modes:
Remote OS version: Linux x64 5.15.146.1-microsoft-standard-WSL2

<details>
<summary>System Info</summary>

|Item|Value|
|---|---|
|CPUs|13th Gen Intel(R) Core(TM) i7-13700H (20 x 2918)|
|GPU Status|2d_canvas: enabled<br>canvas_oop_rasterization: enabled_on<br>direct_rendering_display_compositor: disabled_off_ok<br>gpu_compositing: enabled<br>multiple_raster_threads: enabled_on<br>opengl: enabled_on<br>rasterization: enabled<br>raw_draw: disabled_off_ok<br>skia_graphite: disabled_off<br>video_decode: enabled<br>video_encode: enabled<br>vulkan: disabled_off<br>webgl: enabled<br>webgl2: enabled<br>webgpu: enabled<br>webnn: disabled_off|
|Load (avg)|undefined|
|Memory (System)|31.69GB (8.80GB free)|
|Process Argv|--crash-reporter-id 308f4b56-6b9c-42ef-8d4f-fb34cc0ea82b|
|Screen Reader|no|
|VM|0%|

|Item|Value|
|---|---|
|Remote|WSL: Ubuntu-22.04|
|OS|Linux x64 5.15.146.1-microsoft-standard-WSL2|
|CPUs|13th Gen Intel(R) Core(TM) i7-13700H (20 x 0)|
|Memory (System)|15.47GB (7.64GB free)|
|VM|0%|
</details><details>
<summary>A/B Experiments</summary>

vsliv368:30146709
vspor879:30202332
vspor708:30202333
vspor363:30204092
vswsl492cf:30256860
vscod805:30301674
binariesv615:30325510
vsaa593:30376534
py29gd2263:31024239
c4g48928:30535728
azure-dev_surveyone:30548225
a9j8j154:30646983
962ge761:30959799
pythonnoceb:30805159
asynctok:30898717
pythonmypyd1:30879173
h48ei257:31000450
pythontbext0:30879054
cppperfnew:31000557
dsvsc020:30976470
pythonait:31006305
dsvsc021:30996838
0ee40948:31013168
dvdeprecation:31068756
dwnewjupyter:31046869
2f103344:31071589
nativerepl1:31139838
pythonrstrctxt:31112756
cf971741:31144450
iacca1:31171482
notype1:31157159
5fd0e150:31155592
dwcopilot:31170013
stablechunks:31184530


</details>

<!-- generated by issue reporter -->
@JoeRobich JoeRobich mentioned this issue Nov 20, 2024
Closed
@webreidi webreidi added bug Something isn't working area-test Test discovery, execution, debugging labels Nov 22, 2024
@ocallesp
Copy link
Member

Hi @JoeRobich, does the example code you provided serve as a good reproduction of the issue? This has been reported in the following GitHub issues: #1490 and #996

@ocallesp ocallesp self-assigned this Nov 26, 2024
@ocallesp
Copy link
Member

@JoeRobich I verified that this sample code reproes the issue.

Working on this issue this week.

@sorama-maarten
Copy link

sorama-maarten commented Nov 27, 2024 •
edited
Loading

Hi @ocallesp, better to tag me since I'm the original poster of the problem, I just posted it to the wrong project :)

Glad to hear that you've been able to reproduce the issue this way! If you have any questions, please let me know! Also, my apologies for the duplicates, I tried searching but guess I called it differently from the other people

Let me also add: if you remove one of the tests, i.e. there are 9 tests in the file in total, then all tests show up. This might help point you in the right direction.

@ocallesp
Copy link
Member

ocallesp commented Dec 3, 2024

Fixed.

It will available in the next release

@ocallesp ocallesp closed this as completed Dec 3, 2024
@sorama-maarten
Copy link

Hi @ocallesp, great news! Could you share what the issue was precisely? I'm curious :) If that's not possible, then that's fine of course. Thanks for fixing this!

@ocallesp
Copy link
Member

ocallesp commented Jan 9, 2025

@sorama-maarten, we identified a bug in the VS Code API that occurs when it’s invoked multiple times on the same file. This issue was encountered when processing failed test results. We resolved it by implementing a caching mechanism to prevent repeated calls from triggering the bug.

@ocallesp ocallesp mentioned this issue Jan 9, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ocallesp ocallesp

Labels
area-test Test discovery, execution, debugging bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@ocallesp @JoeRobich @webreidi @sorama-maarten