RFC 012: align AreNotEqual/MatchesRegex summaries with catalog and close discussion phase#8375
Merged
Merged
Conversation
… complete - Change AreNotEqual summary to `Expected values to differ.` (was `Expected values to not be equal.`) to match the RFC 012 catalog and the existing `to differ` wording used by AreNotEqual (string/delta) and AreSequenceEqual. - Change MatchesRegex / DoesNotMatchRegex summaries to `...the specified pattern.` (was `...the specified regular expression.`) to match the RFC 012 catalog and the `pattern` parameter name used in the evidence block. - Update related unit tests and regenerate .xlf files for all locales (UpdateXlf). - Check the `Under discussion` box on RFC 012 since the discussion phase is closed and all catalog entries have been implemented (verified all 1154 TestFramework unit tests pass). `Shipped` will be checked once a release containing all RFC 012 work is published. Co-authored-by: Copilot <[email protected]>
Contributor
There was a problem hiding this comment.
Pull request overview
Aligns a few remaining RFC 012 structured-assertion summary strings (generic AreNotEqual and regex matchers) with the message catalog, and updates resources/tests accordingly. This is part of the ongoing effort to keep MSTest assertion failures consistent and grep-friendly across APIs and locales.
Changes:
- Updated
AreNotEqual(generic) summary to “Expected values to differ.” - Updated
MatchesRegex/DoesNotMatchRegexsummaries to use “pattern” instead of “regular expression.” - Regenerated localized XLF resources and updated unit test expectations + RFC 012 status checklist.
Show a summary per file
| File | Description |
|---|---|
| test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.MatchesRegex.cs | Updates expected structured-message summaries for regex assertions. |
| test/UnitTests/TestFramework.UnitTests/Assertions/AssertTests.AreEqualTests.cs | Updates expected structured-message summary for generic AreNotEqual (including interpolated-string handler test). |
| src/TestFramework/TestFramework/Resources/FrameworkMessages.resx | Updates three summary resource values to match RFC 012 catalog wording. |
| src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.cs.xlf | Regenerated localized resource wrapper for updated summaries (cs). |
| src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.de.xlf | Regenerated localized resource wrapper for updated summaries (de). |
| src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.es.xlf | Regenerated localized resource wrapper for updated summaries (es). |
| src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.fr.xlf | Regenerated localized resource wrapper for updated summaries (fr). |
| src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.it.xlf | Regenerated localized resource wrapper for updated summaries (it). |
| src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.ja.xlf | Regenerated localized resource wrapper for updated summaries (ja). |
| src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.ko.xlf | Regenerated localized resource wrapper for updated summaries (ko). |
| src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.pl.xlf | Regenerated localized resource wrapper for updated summaries (pl). |
| src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.pt-BR.xlf | Regenerated localized resource wrapper for updated summaries (pt-BR). |
| src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.ru.xlf | Regenerated localized resource wrapper for updated summaries (ru). |
| src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.tr.xlf | Regenerated localized resource wrapper for updated summaries (tr). |
| src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.zh-Hans.xlf | Regenerated localized resource wrapper for updated summaries (zh-Hans). |
| src/TestFramework/TestFramework/Resources/xlf/FrameworkMessages.zh-Hant.xlf | Regenerated localized resource wrapper for updated summaries (zh-Hant). |
| docs/RFCs/012-Structured-Assertion-Messages.md | Updates RFC 012 status checklist to mark “Under discussion” consistent with repo convention. |
Copilot's findings
- Files reviewed: 17/17 changed files
- Comments generated: 0
Evangelink
commented
May 20, 2026
Member
Author
Evangelink
left a comment
Evangelink
left a comment
There was a problem hiding this comment.
Review: RFC 012 — Align AreNotEqual/MatchesRegex/DoesNotMatchRegex summaries
All 21 dimensions checked. No blocking or major issues found.
Verdict Table
| # | Dimension | Severity | Finding |
|---|---|---|---|
| 1 | Algorithmic Correctness | MAJOR | ✅ N/A — string-only change, no logic altered |
| 2 | Threading & Concurrency | BLOCKING | ✅ N/A |
| 3 | Security & IPC Contract Safety | BLOCKING | ✅ N/A |
| 4 | Public API & Binary Compatibility | BLOCKING | ✅ No new public API surface. AssertFailedException.Message wording changes are intentional per RFC 012. No PublicAPI.Unshipped.txt changes required. No init accessors introduced. |
| 5 | Performance & Allocations | MAJOR | ✅ N/A |
| 6 | Cross-TFM Compatibility | MAJOR | ✅ N/A — .resx/.xlf changes have no TFM dependency |
| 7 | Resource & IDisposable Management | MAJOR | ✅ N/A |
| 8 | Defensive Coding at Boundaries | MAJOR | ✅ N/A |
| 9 | Localization & Resources | MAJOR | ✅ .resx updated as canonical source. All 13 .xlf locales regenerated via dotnet msbuild /t:UpdateXlf (confirmed: source strings updated, target state reset to "new" and set to match source — correct auto-generated behaviour for strings not yet translated). No manual .xlf edits. |
| 10 | Test Isolation | MAJOR | ✅ N/A |
| 11 | Assertion Quality | MAJOR | ✅ All four test cases updated to match new strings. Tests use FluentAssertions .Should().Be()/.WithMessage() as required. |
| 12 | Flakiness Patterns | BLOCKING | ✅ N/A |
| 13 | Test Completeness & Coverage | MAJOR | ✅ All affected test cases updated; no new code paths introduced |
| 14 | Data-Driven Test Coverage | MODERATE | ✅ N/A |
| 15 | Code Structure & Simplification | MODERATE | ✅ N/A |
| 16 | Naming & Conventions | NIT | ✅ N/A |
| 17 | Documentation Accuracy | NIT | ✅ RFC 012 status checkbox updated correctly. Pre-existing XML doc inconsistencies in Assert.Matches.cs (e.g., "MatchesRegex a regular expression" as verb) are out of scope for this PR. |
| 18 | Analyzer & Code Fix Quality | MAJOR | ✅ N/A — no src/Analyzers/ changes |
| 19 | IPC Wire Compatibility | MAJOR | ✅ N/A — no serialization changes |
| 20 | Build Infrastructure & Dependencies | MODERATE | ✅ N/A |
| 21 | Scope & PR Discipline | MODERATE | ✅ Single concern (aligning 3 summary strings to RFC 012 catalog + closing discussion status). DoesNotMatchRegex change is correctly included despite not being in the PR title. |
Summary
The PR is clean and well-executed:
- The three
.resxstring changes (AreNotEqualFailedSummary,MatchesRegexFailedSummary,DoesNotMatchRegexFailedSummary) correctly align with the RFC 012 catalog wording. - XLF regeneration is confirmed auto-generated: all 13 locales have
state="new"with source and target set to the new English string, which is the correct output ofUpdateXlffor strings awaiting translation. - The
AreNotEqualStringsCaseInsensitiveFailedSummaryalready used"differ"phrasing, so the newAreNotEqualFailedSummaryis now consistent across the family. - RFC status reflects the real state (discussion closed, implementation ongoing, not yet shipped).
No changes requested.
Generated by Expert Code Review (on open) for issue #8375 · ● 6M
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Following up on the RFC 012 implementation push, this PR:
Aligns three
Assert.*failure summaries with the RFC 012 catalog that previously diverged:Assert.AreNotEqual(generic):Expected values to not be equal.→Expected values to differ.to differwording already used byAreNotEqual(string/delta) andAreSequenceEqual.Assert.MatchesRegex:Expected string to match the specified regular expression.→Expected string to match the specified pattern.Assert.DoesNotMatchRegex:Expected string to not match the specified regular expression.→Expected string to not match the specified pattern.patternparameter name (andpattern:evidence label) used by these APIs.Regenerates the XLF resources for all 13 locales via
dotnet msbuild /t:UpdateXlf.Updates unit tests to match the new strings.
Updates RFC 012 status to check
Under discussion(the discussion phase is closed and the entire catalog is implemented), matching the convention used by RFC 011.Shippedwill be checked once a release containing all RFC 012 work is published.Catalog audit
I walked the entire RFC 012 Assertion Message Catalog and verified every entry is implemented:
AreEqual/AreNotEqual(generic, delta, string, string+ignoreCase/culture)AreSame/AreNotSame(incl. null sides + value-type guard)IsTrue/IsFalse(incl. null condition)IsNull/IsNotNullIsInstanceOfType,IsNotInstanceOfType,IsExactInstanceOfType,IsNotExactInstanceOfType(incl. null)Throws,ThrowsExactly,ThrowsAsync,ThrowsExactlyAsync(incl.messageBuilderoverloads)Contains,DoesNotContain,StartsWith,DoesNotStartWith,EndsWith,DoesNotEndWithMatchesRegex,DoesNotMatchRegexContains(item / predicate)Beyond the catalog, the following structured-message APIs have also been added/migrated since the RFC was approved:
AreEquivalent/AreNotEquivalent,AreSequenceEqual/AreNotSequenceEqual,HasCount/IsEmpty/IsNotEmpty,ContainsAll/DoesNotContainAll,IsGreaterThan/IsLessThan/IsGreaterThanOrEqualTo/IsLessThanOrEqualTo/IsPositive/IsNegative/IsInRange,AreAllNotNull/AreAllDistinct/AreAllOfType, andFail/Inconclusive.Validation
dotnet build test/UnitTests/TestFramework.UnitTests/TestFramework.UnitTests.csproj -f net9.0✅Microsoft.VisualStudio.TestPlatform.TestFramework.UnitTests.exe✅ (1154 / 1154 tests passing onnet9.0)