Apply structured assertion messages (RFC 012) to Assert.Fail / Assert.Inconclusive

[Evangelink]

Continues the rollout of RFC 012 — Structured Assertion Messages by migrating Assert.Fail and Assert.Inconclusive to the new formats. Follows the patterns used by the prior assertion migrations (#8170, #8187, #8210, #8214, #8258, #8260, #8265).

Output format

Assert.Fail("Order processing should not reach this branch")

Before:

Assert.Fail failed. Order processing should not reach this branch

After:

Assertion failed.
Order processing should not reach this branch

Assert.Inconclusive("Database server not available for integration tests.")

Before:

Assert.Inconclusive failed. Database server not available for integration tests.

After:

Assert.Inconclusive. Database server not available for integration tests.

Notes

• Assert.Fail now uses StructuredAssertionMessage with an empty summary, so the message is just Assertion failed. plus the optional user message on the next line.
• Assert.Inconclusive now formats messages as Assert.Inconclusive. <message> (or just Assert.Inconclusive.), intentionally dropping failed per RFC 012. This is the documented breaking change to AssertInconclusiveException.Message and it still does not use the universal Assertion failed. prefix.
• Added the new InconclusivePrefix resource and regenerated the corresponding XLF files via UpdateXlf.
• Updated affected unit and integration test expectations that asserted on the legacy single-line formats.

Validation

• .\build.cmd -c Debug -projects src\TestFramework\TestFramework\TestFramework.csproj — passed.
• .\.dotnet\dotnet.exe msbuild src\TestFramework\TestFramework\TestFramework.csproj /t:UpdateXlf /p:Configuration=Debug — passed.
• .\.dotnet\dotnet.exe test test\UnitTests\TestFramework.UnitTests\TestFramework.UnitTests.csproj -c Debug --no-build — 3935/3935 passed.
• .\.dotnet\dotnet.exe build test\UnitTests\MSTestAdapter.PlatformServices.UnitTests\MSTestAdapter.PlatformServices.UnitTests.csproj -c Debug — passed.
• artifacts\bin\MSTestAdapter.PlatformServices.UnitTests\Debug\<tfm>\MSTestAdapter.PlatformServices.UnitTests.exe (net462, net48, net8.0, net8.0-windows10.0.18362.0, net9.0) — 3954/3954 passed.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

[Copilot]

Pull request overview

Migrates MSTest Assert.Fail and Assert.Inconclusive message formatting to align with RFC 012 structured assertion messages, updating localized resources and adjusting unit/integration test expectations accordingly.

Changes:

• Assert.Fail now throws via StructuredAssertionMessage (empty summary) producing Assertion failed. + optional user message on the next line.
• Assert.Inconclusive now formats as Assert.Inconclusive. (optionally followed by the user message) using a new FrameworkMessages.InconclusivePrefix resource.
• Updated unit tests, adapter platform-services tests, and an acceptance test expectation to match the new message formats.

Show a summary per file

File	Description
test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.ScopeTests.cs	Updates scope behavior test expectation for the new Assert.Fail message layout.
test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.InconclusiveTests.cs	Updates inconclusive regression test substring expectation to the new prefix.
test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.FailInconclusive.cs	Adds new focused unit tests covering Fail/Inconclusive message formatting scenarios.
test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Execution/TestMethodInfoTests.cs	Updates adapter unit test expected exception messages for new Fail/Inconclusive formats.
test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Execution/TestExecutionManagerTests.cs	Updates expected failure output message for Assert.Fail.
test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Execution/TestClassInfoTests.cs	Updates expected initialization/cleanup failure messages for new formats.
test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Execution/TestAssemblyInfoTests.cs	Updates expected initialization/cleanup failure messages for new formats.
test/IntegrationTests/MSTest.Acceptance.IntegrationTests/RetryTests.cs	Updates acceptance test output expectation to match the new multi-line Assert.Fail message.
src/TestFramework/TestFramework/Resources/FrameworkMessages.resx	Adds InconclusivePrefix resource entry.
src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.*.xlf	Regenerates localized XLF entries to include InconclusivePrefix.
src/TestFramework/TestFramework/Assertions/Assert.Inconclusive.cs	Implements new Assert.Inconclusive formatting using InconclusivePrefix.
src/TestFramework/TestFramework/Assertions/Assert.Fail.cs	Switches Assert.Fail to StructuredAssertionMessage-based formatting.

Copilot's findings

• Files reviewed: 24/24 changed files
• Comments generated: 2

[Evangelink]

@copilot address review comments

[Copilot]

@copilot address review comments

Addressed in e24a4c5: the retry acceptance test now uses Environment.NewLine, and the duplicate inconclusive { regression test was removed so the existing coverage remains authoritative.

[Evangelink]

@copilot address review comments and fix build issues

[Copilot]

Copilot's findings

Comments suppressed due to low confidence (3)

test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Execution/TestMethodInfoTests.cs:1141

• This expected message hard-codes CRLF ("\r\n") inside the Assert.Fail message. Structured assertion messages use Environment.NewLine, so the test becomes OS-dependent. Prefer composing the expected string with Environment.NewLine (or normalize both strings) to keep the test portable.

        string expectedErrorMessage = string.Format(
            CultureInfo.CurrentCulture,
            Resource.UTA_CleanupMethodThrows,
            typeof(DummyTestClass).FullName,
            _testClassInfo.TestCleanupMethod!.Name,
            "Microsoft.VisualStudio.TestTools.UnitTesting.AssertFailedException: Assertion failed.\r\nTest failed");

test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Execution/TestClassInfoTests.cs:485

• The test hard-codes CRLF ("\r\n") when checking the embedded assertion message. Structured assertion messages use Environment.NewLine, so this check can fail on non-Windows. Use Environment.NewLine (or a newline-normalization helper) when building the expected substring.

        classCleanupException.Should().NotBeNull();
        classCleanupException.Message.StartsWith("Class Cleanup method DummyTestClass.ClassCleanupMethod failed.", StringComparison.Ordinal).Should().BeTrue();
        classCleanupException.Message.Contains("Error Message: Assertion failed.\r\nTest Failure.").Should().BeTrue();
        classCleanupException.Message.Should().Contain(

test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Execution/TestAssemblyInfoTests.cs:268

• This StartsWith expectation hard-codes CRLF ("\r\n") in the embedded Assert.Fail message. Structured assertion messages use Environment.NewLine, so the test is platform-dependent. Build the expected prefix using Environment.NewLine (or normalize newlines) to keep it portable.

        _testAssemblyInfo.AssemblyCleanupMethod = typeof(DummyTestClass).GetMethod("AssemblyCleanupMethod")!;
        string? actualErrorMessage = (await _testAssemblyInfo.ExecuteAssemblyCleanupAsync(GetTestContext()))?.Message;
        actualErrorMessage!.StartsWith(
            "Assembly Cleanup method DummyTestClass.AssemblyCleanupMethod failed. Error Message: Assertion failed.\r\nTest Failure..", StringComparison.Ordinal).Should().BeTrue($"Value: {actualErrorMessage}");
    }

• Files reviewed: 24/24 changed files
• Comments generated: 3

[Copilot]

@copilot address review comments and fix build issues

Addressed in 024ea6a: fixed the build-breaking blank line in AssertTests.FailInconclusive, updated the remaining acceptance expectations for the new multiline Assert.Fail output (RetryTests, SoftAssertionTests, GenericTestMethodTests), and revalidated the affected projects locally. The remaining local acceptance run failures were transient feed download errors while restoring generated test assets, not code failures.

[Evangelink]

@copilot address review comments

[Copilot]

@copilot address review comments

Addressed in fe7b2c6: the remaining MSTestAdapter.PlatformServices expectations now use Environment.NewLine instead of hard-coded CRLF in TestMethodInfoTests, TestClassInfoTests, and TestAssemblyInfoTests. I also attempted local validation twice, but both restore/build runs were blocked by transient dotnet-public package download failures (AwesomeAssertions / StrongNamer).

[Copilot]

Copilot's findings

Comments suppressed due to low confidence (1)

src/TestFramework/TestFramework/Assertions/Assert.AreAllDistinct.cs:125

• Same as the generic overload above: switching to comparer.GetType().Name changes the assertion’s evidence output (namespace is dropped; generic type names may include backticks). If this is accidental/unrelated, consider reverting; if intentional, please call it out alongside the other message-format changes.

    public static void AreAllDistinct([NotNull] IEnumerable? collection, [NotNull] IEqualityComparer? comparer, string? message = "", [CallerArgumentExpression(nameof(collection))] string collectionExpression = "")
    {
        CheckParameterNotNull(collection, "Assert.AreAllDistinct", "collection");
        CheckParameterNotNull(comparer, "Assert.AreAllDistinct", "comparer");
        AreAllDistinctImpl(collection.Cast<object?>(), new NonGenericEqualityComparerAdapter(comparer), comparerTypeName: comparer.GetType().Name, message, collectionExpression);
    }

• Files reviewed: 28/28 changed files
• Comments generated: 2

[Evangelink]

@copilot resolve the merge conflicts in this pull request

[Evangelink]

@copilot address review comments

[Copilot]

@copilot resolve the merge conflicts in this pull request

Resolved in 2474c28: merged main into this branch and fixed the STAThreadingTests conflict by keeping the newer ignored no-runsettings STA scenario from main. The conflicted acceptance test project also builds successfully locally.

[Copilot]

@copilot address review comments

Addressed in 1a434a4: Assert.Inconclusive now reuses the shared prefix/message formatter from Assert, removing the duplicated newline/whitespace handling without changing behavior. I also rebuilt src/TestFramework/TestFramework/TestFramework.csproj locally; the remaining test invocation in this sandbox is still blocked by transient dotnet-public package download failures and the missing net48 targeting pack.

[Copilot]

Copilot's findings

• Files reviewed: 28/28 changed files
• Comments generated: 4