Skip to content

Unify reflection helpers#7344

Merged
Evangelink merged 31 commits into
mainfrom
dev/amauryleve/reflection
May 14, 2026
Merged

Unify reflection helpers#7344
Evangelink merged 31 commits into
mainfrom
dev/amauryleve/reflection

Conversation

@Evangelink
Copy link
Copy Markdown
Member

@Evangelink Evangelink commented Feb 4, 2026

Summary

Unify the reflection helper infrastructure by consolidating ReflectHelper logic into ReflectionOperations (the IReflectionOperations implementation). This eliminates a layer of indirection and makes the reflection surface area consistent across the adapter and platform services.

Note: ReflectHelper still exists in this PR — it has been partially migrated. Instance methods like IsAttributeDefined, GetFirstAttributeOrDefault, GetSingleAttributeOrDefault, and GetAttributes<T> still live on ReflectHelper and delegate to the centralized ReflectionOperations cache. The remaining call sites (e.g. TypeValidator, TestMethodValidator) will be migrated in a follow-up.

Changes

Consolidation into IReflectionOperations / ReflectionOperations

  • Extended IReflectionOperations with higher-level generic methods previously only available on ReflectHelper:
    • IsAttributeDefined<TAttribute>
    • GetFirstAttributeOrDefault<TAttribute>
    • GetSingleAttributeOrDefault<TAttribute>
    • GetAttributes<TAttributeType>
    • GetCustomAttributesCached
    • IsMethodDeclaredInSameAssemblyAsType
  • Moved the attribute cache (ConcurrentDictionary<ICustomAttributeProvider, Attribute[]>) from ReflectHelper into ReflectionOperations.
  • Made ReflectionOperations inherit from MarshalByRefObject (needed for .NET Framework app-domain isolation).

New helper: ReflectionHelper (extension methods)

  • Created ReflectionHelper in MSTestAdapter.PlatformServices with extension methods on IReflectionOperations for:
    • GetTestCategories — collects TestCategoryBaseAttribute from method, type, and assembly levels.
    • GetTestPropertiesAsTraits — extracts TestPropertyAttribute values as Trait[].

ReflectHelper simplification (partial migration)

  • ReflectHelper.GetCustomAttributesCached now delegates to PlatformServiceProvider.Instance.ReflectionOperations.GetCustomAttributesCached, so discovery and execution share a single attribute cache (previously each had its own copy).
  • Static helper methods (GetTestCategories, GetTestPropertiesAsTraits, GetCustomAttributesCached, ClearCache) are now thin wrappers that forward to the ReflectionOperations cache.
  • Deleted ReflectionOperationsNetFrameworkAttributeHelpers.cs — the .NET Framework-specific reflection-only-load logic is no longer needed.
  • Deleted the ReflectHelperTests unit test file (tests moved to ReflectionOperationsTests).

Call-site migrations

  • Updated all call sites that used ReflectHelper.Instance to go through PlatformServiceProvider.Instance.ReflectionOperations instead:
    • TestAssemblySettingsProvider, TestMethodInfo, TestMethodRunner, AttributeHelpers, TestDeployment, DeploymentItemUtility, DeploymentUtilityBase.
  • TypeEnumerator.GetTestFromMethod now routes GetTestCategories, GetTestPropertiesAsTraits, and GetCustomAttributesCached calls through PlatformServiceProvider.Instance.ReflectionOperations (+ extension methods) instead of static ReflectHelper methods, keeping the DI surface consistent.

Test infrastructure

  • Introduced MockableReflectionOperations — a wrapper that delegates non-generic IReflectionOperations methods to a Moq mock, and implements the new generic methods by filtering mock results. This solves the problem of Moq not supporting generic methods with type constraints.
  • Updated TestablePlatformServiceProvider.SetupMockReflectionOperations() to use the new wrapper.
  • Migrated and expanded tests from ReflectHelperTests into ReflectionOperationsTests (attribute caching, IsAttributeDefined, GetTestCategories, etc.).
  • Updated ReflectionUtilityTests (integration tests) to use Assembly.LoadFrom instead of the removed Assembly.ReflectionOnlyLoadFrom.
  • Added new test cases for the GetAttributes<T> generic method on both unit and integration test levels.
  • Added GetTestPropertiesAsTraits null-ReflectedType test, GetAttributes<T> happy-path tests, and MockableReflectionOperations null-guard test.

@Evangelink Evangelink marked this pull request as ready for review February 6, 2026 13:39
@Evangelink Evangelink enabled auto-merge February 6, 2026 13:39
Comment thread src/Adapter/MSTestAdapter.PlatformServices/Services/ReflectionOperations.cs Outdated
# Conflicts:
#	src/Adapter/MSTestAdapter.PlatformServices/Services/ReflectionOperations.cs
#	test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Execution/TestPropertyAttributeTests.cs
#	test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Execution/TypeCacheTests.cs
#	test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Helpers/ReflectHelperTests.cs
…tion

# Conflicts:
#	src/Adapter/MSTestAdapter.PlatformServices/Services/ReflectionOperations.cs
#	src/Adapter/MSTestAdapter.PlatformServices/Services/TestDeployment.cs
#	src/Adapter/MSTestAdapter.PlatformServices/Utilities/DeploymentItemUtility.cs
#	src/Adapter/MSTestAdapter.PlatformServices/Utilities/DeploymentUtilityBase.cs
#	src/Adapter/MSTestAdapter.PlatformServices/Utilities/ReflectionOperationsNetFrameworkAttributeHelpers.cs
#	test/IntegrationTests/PlatformServices.Desktop.IntegrationTests/ReflectionUtilityTests.cs
#	test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Services/DesktopTestDeploymentTests.cs
#	test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Services/TestDeploymentTests.cs
#	test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Utilities/DeploymentItemUtilityTests.cs
#	test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Utilities/DeploymentUtilityTests.cs
#	test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Utilities/ReflectionUtilityTests.cs
Copy link
Copy Markdown
Member

@Youssef1313 Youssef1313 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess we are good now that everything uses the caching ReflectHelper on current main?

(Except that we need to investigate if it's possible to simplify the implementation of reflection operations under .NET Framework)

@Evangelink
Copy link
Copy Markdown
Member Author

@copilot resolve the merge conflicts in this pull request

…tion

# Conflicts:
#	src/Adapter/MSTestAdapter.PlatformServices/Discovery/AssemblyEnumerator.cs
#	src/Adapter/MSTestAdapter.PlatformServices/Discovery/TypeEnumerator.cs
#	src/Adapter/MSTestAdapter.PlatformServices/Execution/TypeCache.cs
#	src/Adapter/MSTestAdapter.PlatformServices/Helpers/ReflectHelper.cs
#	test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Discovery/TestMethodValidatorTests.cs
#	test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Discovery/TypeEnumeratorTests.cs
#	test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Execution/TypeCacheTests.cs
#	test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Execution/UnitTestRunnerTests.cs

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Copilot AI review requested due to automatic review settings April 17, 2026 18:40
@Evangelink Evangelink review requested due to automatic review settings April 17, 2026 18:40
Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 17, 2026

@copilot resolve the merge conflicts in this pull request

Resolved the merge conflicts and pushed the merge commit: 19c164645.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • k4kvsblobprodcus344.vsblob.vsassets.io
    • Triggering command: /home/REDACTED/work/testfx/testfx/.dotnet/dotnet /home/REDACTED/work/testfx/testfx/.dotnet/dotnet msbuild /m /nologo /clp:Summary /v:minimal /nr:true /warnaserror /p:TreatWarningsAsErrors=true /p:ContinuousIntegrationBuild=false /home/REDACTED/work/testfx/testfx/artifacts/toolset/restore.proj /t:__WriteToolsetLocation /clp:ErrorsOnly;NoSummary /p:__ToolsetLocationOutputFile=/home/REDACTED/work/testfx/testfx/artifacts/toolset/11.0.0-beta.26211.1.txt (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

@Evangelink
Copy link
Copy Markdown
Member Author

@copilot fix the following issues:

@azure-pipelines
azure-pipelines
/ microsoft.testfx (Build Linux Debug)
src/Adapter/MSTestAdapter.PlatformServices/Execution/UnitTestRunner.cs#L68

src/Adapter/MSTestAdapter.PlatformServices/Execution/UnitTestRunner.cs(68,36): error CS1503: (NETCORE_ENGINEERING_TELEMETRY=Build) Argument 1: cannot convert from 'Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.Interface.IReflectionOperations' to 'Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Helpers.ReflectHelper'
Check failure on line 146 in src/Adapter/MSTestAdapter.PlatformServices/Discovery/AssemblyEnumerator.cs

@azure-pipelines
azure-pipelines
/ microsoft.testfx (Build Linux Debug)
src/Adapter/MSTestAdapter.PlatformServices/Discovery/AssemblyEnumerator.cs#L146

src/Adapter/MSTestAdapter.PlatformServices/Discovery/AssemblyEnumerator.cs(146,47): error CS1503: (NETCORE_ENGINEERING_TELEMETRY=Build) Argument 1: cannot convert from 'Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Helpers.ReflectHelper' to 'Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.Interface.IReflectionOperations'

Copy link
Copy Markdown
Member Author

@Evangelink Evangelink left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Workflow: Expert Code Reviewer 🧠
Date: 2026-05-12
Repository: microsoft/testfx

Key Findings

[Performance] Dual attribute caches — memory regression for large test suites
ReflectionOperations now has its own _attributeCache (ConcurrentDictionary<ICustomAttributeProvider, Attribute[]>), but ReflectHelper retains an identical independent cache. Since the discovery path (TypeEnumerator, TypeCache, AssemblyEnumerator) still routes through ReflectHelper._attributeCache while the execution path (TestMethodInfo) now routes through ReflectionOperations._attributeCache, the same MethodInfo ends up cached in both dictionaries for every test that is discovered and then run. This doubles the attribute cache memory footprint.

[Performance] ReflectionHelper.GetTestCategories LINQ overhead vs allocation-free ReflectHelper equivalent
The new extension method allocates three iterator state machines plus Concat/SelectMany enumerators per call. The existing ReflectHelper.GetTestCategories was specifically optimised to use direct array loops with a lazily-allocated List<string>. When the migration is eventually completed (TypeEnumerator migrates from _reflectHelper to IReflectionOperations), this will become a measurable per-test-method regression in test discovery.

[Correctness] MockableReflectionOperations.GetSingleAttributeOrDefault throws a different exception message than production code
LINQ .SingleOrDefault() emits the BCL default message on sequences with more than one element, while ReflectionOperations.GetSingleAttributeOrDefault uses Resource.DuplicateAttributeError. Tests that exercise the duplicate-attribute error path via the mock will not validate the real user-facing message.

Positive Observations

  • The prior round's findings are all properly addressed: TestAssemblySettingsProvider now uses the cached GetSingleAttributeOrDefault / IsAttributeDefined APIs; GetCustomAttributesCached has a correct null-guard before the type check; IsAttributeDefined<T> accepts ICustomAttributeProvider rather than MemberInfo.
  • MockableReflectionOperations is a clean solution to Moq's inability to mock generic methods with type constraints — the design is sound.
  • [DoNotParallelize] on ReflectionOperationsTests correctly guards against races on the global PlatformServiceProvider.Instance.
  • The allocation-free GetTestPropertiesAsTraits implementation in ReflectionHelper matches the optimised ReflectHelper version — good consistency there.

Recommendations

  1. Either complete the TypeEnumerator/TypeCache migration in this PR or add an explicit task to remove ReflectHelper._attributeCache once those callers are migrated — the dual-cache situation is easy to forget.
  2. Align ReflectionHelper.GetTestCategories with the allocation-free loop pattern in ReflectHelper.GetTestCategories before the migration is completed.
  3. Make MockableReflectionOperations.GetSingleAttributeOrDefault emit the same Resource.DuplicateAttributeError message as the production implementation.

Generated by Expert Code Reviewer 🧠

🧠 Reviewed by Expert Code Reviewer 🧠

Comment thread src/Adapter/MSTestAdapter.PlatformServices/Helpers/ReflectionHelper.cs Outdated
- Rename SetCustomAttribute to AddCustomAttribute to reflect additive semantics
- Add GetTestPropertiesAsTraits unit tests (method, class, multi-level, empty)
- Remove fragile HaveCount assertions from integration tests (filter-based instead)
- Unify attribute cache: ReflectHelper delegates to ReflectionOperations cache
- Replace LINQ allocations in GetTestCategories with direct iteration
- Fix MockableReflectionOperations.GetSingleAttributeOrDefault error message
- Make ReflectHelper methods static where they no longer access instance data
Copilot AI review requested due to automatic review settings May 12, 2026 12:08
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 31/31 changed files
  • Comments generated: 3

Comment thread src/Adapter/MSTestAdapter.PlatformServices/Discovery/TypeEnumerator.cs Outdated
Comment thread src/Adapter/MSTestAdapter.PlatformServices/Helpers/ReflectHelper.cs
Copy link
Copy Markdown
Member Author

@Evangelink Evangelink left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Workflow: Test Expert Reviewer 🧪
Date: 2026-05-12
Repository: microsoft/testfx
PR: #7344 — Unify reflection helpers


Key Findings

The test changes in this PR are well-structured overall. The major improvements are:

  • Assertion quality uplift: SequenceEqual(...).Should().BeTrue().Should().Equal(...) throughout ReflectionOperationsTests gives much better failure messages.
  • Isolation correctly handled: [DoNotParallelize] added with an explanatory comment, PlatformServiceProvider.Instance set/restored in constructor/Dispose.
  • MockableReflectionOperations is a clever bridge for the Moq generic-method limitation; the design is well-documented.
  • New tests for GetTestCategories, GetTestPropertiesAsTraits, GetFirstAttributeOrDefault, GetSingleAttributeOrDefault, null guards, and caching are all solid.
  • Integration tests correctly switch from ReflectionOnlyLoadFrom to LoadFrom, and fragility against compiler-generated attributes is handled by filtering in GetAttributeValuePairs while preserving content correctness through .Should().Equal().

Three minor gaps were flagged as inline comments:

  1. [Coverage] GetAttributes<TAttributeType> — only has a null-guard unit test; happy-path unit tests (returns matching, empty when not matching) are missing (integration tests cover it, but unit-level feedback is faster).
  2. [Coverage] GetTestPropertiesAsTraits — no test exercises the null-ReflectedType branch in the production helper.
  3. [Coverage] MockableReflectionOperations.GetCustomAttributesCached — passes null causes NullReferenceException from .GetType() in the default switch arm, inconsistent with ReflectionOperations throwing ArgumentNullException.

None of these are blocking — no isolation bugs, false positives, or flakiness patterns found.


Generated by Test Expert Reviewer 🧪

🧪 Test quality reviewed by Test Expert Reviewer 🧪

Copy link
Copy Markdown
Member Author

@Evangelink Evangelink left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

Workflow: Expert Code Reviewer 🧠
Date: 2026-05-12
Repository: microsoft/testfx


Key Findings

All issues identified in prior review rounds have been correctly resolved:

  • Null check ordering in GetCustomAttributesCachedArgumentNullException is now thrown before the type-guard ArgumentException.
  • catch (Exception) swallowing ArgumentException — The type guard at the entry of GetCustomAttributesCached ensures unsupported types never reach the inner GetOrAdd factory, so the catch no longer masks programming errors.
  • IsAttributeDefined<TAttribute> API consistency — Now takes ICustomAttributeProvider (not MemberInfo), matching all sibling methods.
  • TestAssemblySettingsProvider bypassing cache — Now uses GetSingleAttributeOrDefault<ParallelizeAttribute> and IsAttributeDefined<DoNotParallelizeAttribute> through the unified cache.
  • GetRetryAttribute iterator allocation — Now iterates GetCustomAttributesCached(MethodInfo) directly with no state machine.
  • GetSingleAttributeOrDefault error message in mockMockableReflectionOperations mirrors Resource.DuplicateAttributeError exactly.

Observations

  • ReflectionHelper extension methods (GetTestCategories, GetTestPropertiesAsTraits) are exercised in ReflectionOperationsTests; TypeEnumerator still calls ReflectHelper.GetTestCategories which delegates through the shared cache — both paths are correct, and the extension methods are ready for the planned TypeEnumerator migration step.
  • Assembly-level static helpers in ReflectHelper (GetParallelizeAttribute, HasDiscoverInternalsAttribute, etc.) still call GetCustomAttributes directly — this is pre-existing behavior for once-per-session lookups and not a regression introduced by this PR.
  • Single unified _attributeCache in ReflectionOperations is correctly shared across discovery (via ReflectHelper static delegation) and execution (via direct PlatformServiceProvider.Instance.ReflectionOperations access).

Overall Assessment

The PR successfully unifies the reflection helper infrastructure. The architecture is sound, all previously flagged issues are resolved, caching is correctly centralized in ReflectionOperations._attributeCache, and the MarshalByRefObject pattern for .NET Framework AppDomain isolation is properly carried over. No new issues found.


Reviewed by Expert Code Reviewer 🧠

🧠 Reviewed by Expert Code Reviewer 🧠

Copilot AI review requested due to automatic review settings May 12, 2026 14:30
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 31/31 changed files
  • Comments generated: 2

@Evangelink
Copy link
Copy Markdown
Member Author

@copilot resolve the merge conflicts in this pull request

Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 13, 2026 16:11
@Evangelink Evangelink review requested due to automatic review settings May 13, 2026 16:11
Copy link
Copy Markdown
Contributor

Copilot AI commented May 13, 2026

@copilot resolve the merge conflicts in this pull request

Done in 90d15e1. I merged origin/main into this branch and resolved conflicts in:

  • src/Adapter/MSTestAdapter.PlatformServices/Execution/TestMethodInfo.cs
  • src/Adapter/MSTestAdapter.PlatformServices/Helpers/AttributeHelpers.cs

Copilot AI review requested due to automatic review settings May 13, 2026 16:51
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 30/30 changed files
  • Comments generated: 2

The PR's recent introduction of `MockableReflectionOperations` (which
implements the new generic interface methods by filtering results from
`mock.Object.GetCustomAttributes(Assembly, typeof(Attribute))`) plus a
couple of test-only mistakes left the Windows CI runs red on .NET FX 4.8,
net8.0 and net9.0. This commit addresses both the failures and the two
remaining unresolved Copilot review comments.

Test fixes:

- `TestAssemblySettingsProviderTests` (4 tests): the existing setups use
  `Setup(ro => ro.GetCustomAttributes(It.IsAny<Assembly>(), typeof(ParallelizeAttribute)))`
  but `MockableReflectionOperations` always passes `typeof(Attribute)`
  (matching production's `NotCachedReflectionAccessor` for assemblies),
  so those setups never matched and the tests saw a default
  `Workers = -1`. Switch the setups to `typeof(Attribute)`.

- `ReflectionOperationsTests` (4 `GetTestPropertiesAsTraits*` tests):
  switched from `Should().Equal(new[] { new Trait(...) })` to
  `Should().BeEquivalentTo(new[] { new Trait(...) }, options => options.WithStrictOrdering())`.
  `Trait` does not override `Equals`, so the previous `Equal()` call
  performed reference comparison and always failed even when values
  matched.

Review comment #2 (`TestableExtendedTestMethod` declared public in the
global namespace): the type is already in the file-scoped
`MSTestAdapter.PlatformServices.UnitTests.Services` namespace (the
reviewer's claim about the global namespace is not accurate with the
file-scoped namespace declaration), but it does not need to be public —
made it `internal`.

Review comment #1 (`AttributeMockingHelper` treating class providers as
`TypeInfo` rather than `Type`): investigated and confirmed the premise
is incorrect. On all supported runtimes (.NET Framework 4.5+ and
.NET Core 1.0+), `RuntimeType` derives from `TypeInfo`, so
`typeof(Foo) is TypeInfo` is `true` and the class-level branch in
`AttributeMockingHelper.GetCustomAttributesNotCached` is reachable.
Verified locally with a small repro on net8.0. No change needed.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 14, 2026 17:13
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 31/31 changed files
  • Comments generated: 7

Comment thread src/Adapter/MSTestAdapter.PlatformServices/Helpers/ReflectHelper.cs
@Evangelink Evangelink disabled auto-merge May 14, 2026 18:13
@Evangelink Evangelink merged commit e2209d7 into main May 14, 2026
47 checks passed
@Evangelink Evangelink deleted the dev/amauryleve/reflection branch May 14, 2026 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants