Unify reflection helpers#7344
Conversation
# Conflicts: # src/Adapter/MSTestAdapter.PlatformServices/Services/ReflectionOperations.cs # test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Execution/TestPropertyAttributeTests.cs # test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Execution/TypeCacheTests.cs # test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Helpers/ReflectHelperTests.cs
…tion # Conflicts: # src/Adapter/MSTestAdapter.PlatformServices/Services/ReflectionOperations.cs # src/Adapter/MSTestAdapter.PlatformServices/Services/TestDeployment.cs # src/Adapter/MSTestAdapter.PlatformServices/Utilities/DeploymentItemUtility.cs # src/Adapter/MSTestAdapter.PlatformServices/Utilities/DeploymentUtilityBase.cs # src/Adapter/MSTestAdapter.PlatformServices/Utilities/ReflectionOperationsNetFrameworkAttributeHelpers.cs # test/IntegrationTests/PlatformServices.Desktop.IntegrationTests/ReflectionUtilityTests.cs # test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Services/DesktopTestDeploymentTests.cs # test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Services/TestDeploymentTests.cs # test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Utilities/DeploymentItemUtilityTests.cs # test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Utilities/DeploymentUtilityTests.cs # test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Utilities/ReflectionUtilityTests.cs
Youssef1313
left a comment
There was a problem hiding this comment.
I guess we are good now that everything uses the caching ReflectHelper on current main?
(Except that we need to investigate if it's possible to simplify the implementation of reflection operations under .NET Framework)
|
@copilot resolve the merge conflicts in this pull request |
…tion # Conflicts: # src/Adapter/MSTestAdapter.PlatformServices/Discovery/AssemblyEnumerator.cs # src/Adapter/MSTestAdapter.PlatformServices/Discovery/TypeEnumerator.cs # src/Adapter/MSTestAdapter.PlatformServices/Execution/TypeCache.cs # src/Adapter/MSTestAdapter.PlatformServices/Helpers/ReflectHelper.cs # test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Discovery/TestMethodValidatorTests.cs # test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Discovery/TypeEnumeratorTests.cs # test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Execution/TypeCacheTests.cs # test/UnitTests/MSTestAdapter.PlatformServices.UnitTests/Execution/UnitTestRunnerTests.cs Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Resolved the merge conflicts and pushed the merge commit: Warning Firewall rules blocked me from connecting to one or more addresses (expand for details)I tried to connect to the following addresses, but was blocked by firewall rules:
If you need me to access, download, or install something from one of these locations, you can either:
|
|
@copilot fix the following issues: @azure-pipelines src/Adapter/MSTestAdapter.PlatformServices/Execution/UnitTestRunner.cs(68,36): error CS1503: (NETCORE_ENGINEERING_TELEMETRY=Build) Argument 1: cannot convert from 'Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.Interface.IReflectionOperations' to 'Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Helpers.ReflectHelper' @azure-pipelines src/Adapter/MSTestAdapter.PlatformServices/Discovery/AssemblyEnumerator.cs(146,47): error CS1503: (NETCORE_ENGINEERING_TELEMETRY=Build) Argument 1: cannot convert from 'Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.Helpers.ReflectHelper' to 'Microsoft.VisualStudio.TestPlatform.MSTestAdapter.PlatformServices.Interface.IReflectionOperations' |
Evangelink
left a comment
There was a problem hiding this comment.
Summary
Workflow: Expert Code Reviewer 🧠
Date: 2026-05-12
Repository: microsoft/testfx
Key Findings
[Performance] Dual attribute caches — memory regression for large test suites
ReflectionOperations now has its own _attributeCache (ConcurrentDictionary<ICustomAttributeProvider, Attribute[]>), but ReflectHelper retains an identical independent cache. Since the discovery path (TypeEnumerator, TypeCache, AssemblyEnumerator) still routes through ReflectHelper._attributeCache while the execution path (TestMethodInfo) now routes through ReflectionOperations._attributeCache, the same MethodInfo ends up cached in both dictionaries for every test that is discovered and then run. This doubles the attribute cache memory footprint.
[Performance] ReflectionHelper.GetTestCategories LINQ overhead vs allocation-free ReflectHelper equivalent
The new extension method allocates three iterator state machines plus Concat/SelectMany enumerators per call. The existing ReflectHelper.GetTestCategories was specifically optimised to use direct array loops with a lazily-allocated List<string>. When the migration is eventually completed (TypeEnumerator migrates from _reflectHelper to IReflectionOperations), this will become a measurable per-test-method regression in test discovery.
[Correctness] MockableReflectionOperations.GetSingleAttributeOrDefault throws a different exception message than production code
LINQ .SingleOrDefault() emits the BCL default message on sequences with more than one element, while ReflectionOperations.GetSingleAttributeOrDefault uses Resource.DuplicateAttributeError. Tests that exercise the duplicate-attribute error path via the mock will not validate the real user-facing message.
Positive Observations
- The prior round's findings are all properly addressed:
TestAssemblySettingsProvidernow uses the cachedGetSingleAttributeOrDefault/IsAttributeDefinedAPIs;GetCustomAttributesCachedhas a correct null-guard before the type check;IsAttributeDefined<T>acceptsICustomAttributeProviderrather thanMemberInfo. MockableReflectionOperationsis a clean solution to Moq's inability to mock generic methods with type constraints — the design is sound.[DoNotParallelize]onReflectionOperationsTestscorrectly guards against races on the globalPlatformServiceProvider.Instance.- The allocation-free
GetTestPropertiesAsTraitsimplementation inReflectionHelpermatches the optimisedReflectHelperversion — good consistency there.
Recommendations
- Either complete the
TypeEnumerator/TypeCachemigration in this PR or add an explicit task to removeReflectHelper._attributeCacheonce those callers are migrated — the dual-cache situation is easy to forget. - Align
ReflectionHelper.GetTestCategorieswith the allocation-free loop pattern inReflectHelper.GetTestCategoriesbefore the migration is completed. - Make
MockableReflectionOperations.GetSingleAttributeOrDefaultemit the sameResource.DuplicateAttributeErrormessage as the production implementation.
Generated by Expert Code Reviewer 🧠
🧠 Reviewed by Expert Code Reviewer 🧠
- Rename SetCustomAttribute to AddCustomAttribute to reflect additive semantics - Add GetTestPropertiesAsTraits unit tests (method, class, multi-level, empty) - Remove fragile HaveCount assertions from integration tests (filter-based instead) - Unify attribute cache: ReflectHelper delegates to ReflectionOperations cache - Replace LINQ allocations in GetTestCategories with direct iteration - Fix MockableReflectionOperations.GetSingleAttributeOrDefault error message - Make ReflectHelper methods static where they no longer access instance data
Evangelink
left a comment
There was a problem hiding this comment.
Summary
Workflow: Test Expert Reviewer 🧪
Date: 2026-05-12
Repository: microsoft/testfx
PR: #7344 — Unify reflection helpers
Key Findings
The test changes in this PR are well-structured overall. The major improvements are:
- Assertion quality uplift:
SequenceEqual(...).Should().BeTrue()→.Should().Equal(...)throughoutReflectionOperationsTestsgives much better failure messages. - Isolation correctly handled:
[DoNotParallelize]added with an explanatory comment,PlatformServiceProvider.Instanceset/restored in constructor/Dispose. MockableReflectionOperationsis a clever bridge for the Moq generic-method limitation; the design is well-documented.- New tests for
GetTestCategories,GetTestPropertiesAsTraits,GetFirstAttributeOrDefault,GetSingleAttributeOrDefault, null guards, and caching are all solid. - Integration tests correctly switch from
ReflectionOnlyLoadFromtoLoadFrom, and fragility against compiler-generated attributes is handled by filtering inGetAttributeValuePairswhile preserving content correctness through.Should().Equal().
Three minor gaps were flagged as inline comments:
- [Coverage]
GetAttributes<TAttributeType>— only has a null-guard unit test; happy-path unit tests (returns matching, empty when not matching) are missing (integration tests cover it, but unit-level feedback is faster). - [Coverage]
GetTestPropertiesAsTraits— no test exercises the null-ReflectedTypebranch in the production helper. - [Coverage]
MockableReflectionOperations.GetCustomAttributesCached— passesnullcausesNullReferenceExceptionfrom.GetType()in the default switch arm, inconsistent withReflectionOperationsthrowingArgumentNullException.
None of these are blocking — no isolation bugs, false positives, or flakiness patterns found.
Generated by Test Expert Reviewer 🧪
🧪 Test quality reviewed by Test Expert Reviewer 🧪
Evangelink
left a comment
There was a problem hiding this comment.
Summary
Workflow: Expert Code Reviewer 🧠
Date: 2026-05-12
Repository: microsoft/testfx
Key Findings
All issues identified in prior review rounds have been correctly resolved:
- ✅ Null check ordering in
GetCustomAttributesCached—ArgumentNullExceptionis now thrown before the type-guardArgumentException. - ✅
catch (Exception)swallowingArgumentException— The type guard at the entry ofGetCustomAttributesCachedensures unsupported types never reach the innerGetOrAddfactory, so the catch no longer masks programming errors. - ✅
IsAttributeDefined<TAttribute>API consistency — Now takesICustomAttributeProvider(notMemberInfo), matching all sibling methods. - ✅
TestAssemblySettingsProviderbypassing cache — Now usesGetSingleAttributeOrDefault<ParallelizeAttribute>andIsAttributeDefined<DoNotParallelizeAttribute>through the unified cache. - ✅
GetRetryAttributeiterator allocation — Now iteratesGetCustomAttributesCached(MethodInfo)directly with no state machine. - ✅
GetSingleAttributeOrDefaulterror message in mock —MockableReflectionOperationsmirrorsResource.DuplicateAttributeErrorexactly.
Observations
ReflectionHelperextension methods (GetTestCategories,GetTestPropertiesAsTraits) are exercised inReflectionOperationsTests;TypeEnumeratorstill callsReflectHelper.GetTestCategorieswhich delegates through the shared cache — both paths are correct, and the extension methods are ready for the plannedTypeEnumeratormigration step.- Assembly-level static helpers in
ReflectHelper(GetParallelizeAttribute,HasDiscoverInternalsAttribute, etc.) still callGetCustomAttributesdirectly — this is pre-existing behavior for once-per-session lookups and not a regression introduced by this PR. - Single unified
_attributeCacheinReflectionOperationsis correctly shared across discovery (viaReflectHelperstatic delegation) and execution (via directPlatformServiceProvider.Instance.ReflectionOperationsaccess).
Overall Assessment
The PR successfully unifies the reflection helper infrastructure. The architecture is sound, all previously flagged issues are resolved, caching is correctly centralized in ReflectionOperations._attributeCache, and the MarshalByRefObject pattern for .NET Framework AppDomain isolation is properly carried over. No new issues found.
Reviewed by Expert Code Reviewer 🧠
🧠 Reviewed by Expert Code Reviewer 🧠
|
@copilot resolve the merge conflicts in this pull request |
Co-authored-by: Evangelink <11340282+Evangelink@users.noreply.github.com>
Done in
|
The PR's recent introduction of `MockableReflectionOperations` (which
implements the new generic interface methods by filtering results from
`mock.Object.GetCustomAttributes(Assembly, typeof(Attribute))`) plus a
couple of test-only mistakes left the Windows CI runs red on .NET FX 4.8,
net8.0 and net9.0. This commit addresses both the failures and the two
remaining unresolved Copilot review comments.
Test fixes:
- `TestAssemblySettingsProviderTests` (4 tests): the existing setups use
`Setup(ro => ro.GetCustomAttributes(It.IsAny<Assembly>(), typeof(ParallelizeAttribute)))`
but `MockableReflectionOperations` always passes `typeof(Attribute)`
(matching production's `NotCachedReflectionAccessor` for assemblies),
so those setups never matched and the tests saw a default
`Workers = -1`. Switch the setups to `typeof(Attribute)`.
- `ReflectionOperationsTests` (4 `GetTestPropertiesAsTraits*` tests):
switched from `Should().Equal(new[] { new Trait(...) })` to
`Should().BeEquivalentTo(new[] { new Trait(...) }, options => options.WithStrictOrdering())`.
`Trait` does not override `Equals`, so the previous `Equal()` call
performed reference comparison and always failed even when values
matched.
Review comment #2 (`TestableExtendedTestMethod` declared public in the
global namespace): the type is already in the file-scoped
`MSTestAdapter.PlatformServices.UnitTests.Services` namespace (the
reviewer's claim about the global namespace is not accurate with the
file-scoped namespace declaration), but it does not need to be public —
made it `internal`.
Review comment #1 (`AttributeMockingHelper` treating class providers as
`TypeInfo` rather than `Type`): investigated and confirmed the premise
is incorrect. On all supported runtimes (.NET Framework 4.5+ and
.NET Core 1.0+), `RuntimeType` derives from `TypeInfo`, so
`typeof(Foo) is TypeInfo` is `true` and the class-level branch in
`AttributeMockingHelper.GetCustomAttributesNotCached` is reachable.
Verified locally with a small repro on net8.0. No change needed.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Summary
Unify the reflection helper infrastructure by consolidating
ReflectHelperlogic intoReflectionOperations(theIReflectionOperationsimplementation). This eliminates a layer of indirection and makes the reflection surface area consistent across the adapter and platform services.Changes
Consolidation into
IReflectionOperations/ReflectionOperationsIReflectionOperationswith higher-level generic methods previously only available onReflectHelper:IsAttributeDefined<TAttribute>GetFirstAttributeOrDefault<TAttribute>GetSingleAttributeOrDefault<TAttribute>GetAttributes<TAttributeType>GetCustomAttributesCachedIsMethodDeclaredInSameAssemblyAsTypeConcurrentDictionary<ICustomAttributeProvider, Attribute[]>) fromReflectHelperintoReflectionOperations.ReflectionOperationsinherit fromMarshalByRefObject(needed for .NET Framework app-domain isolation).New helper:
ReflectionHelper(extension methods)ReflectionHelperinMSTestAdapter.PlatformServiceswith extension methods onIReflectionOperationsfor:GetTestCategories— collectsTestCategoryBaseAttributefrom method, type, and assembly levels.GetTestPropertiesAsTraits— extractsTestPropertyAttributevalues asTrait[].ReflectHelpersimplification (partial migration)ReflectHelper.GetCustomAttributesCachednow delegates toPlatformServiceProvider.Instance.ReflectionOperations.GetCustomAttributesCached, so discovery and execution share a single attribute cache (previously each had its own copy).GetTestCategories,GetTestPropertiesAsTraits,GetCustomAttributesCached,ClearCache) are now thin wrappers that forward to theReflectionOperationscache.ReflectionOperationsNetFrameworkAttributeHelpers.cs— the .NET Framework-specific reflection-only-load logic is no longer needed.ReflectHelperTestsunit test file (tests moved toReflectionOperationsTests).Call-site migrations
ReflectHelper.Instanceto go throughPlatformServiceProvider.Instance.ReflectionOperationsinstead:TestAssemblySettingsProvider,TestMethodInfo,TestMethodRunner,AttributeHelpers,TestDeployment,DeploymentItemUtility,DeploymentUtilityBase.TypeEnumerator.GetTestFromMethodnow routesGetTestCategories,GetTestPropertiesAsTraits, andGetCustomAttributesCachedcalls throughPlatformServiceProvider.Instance.ReflectionOperations(+ extension methods) instead of staticReflectHelpermethods, keeping the DI surface consistent.Test infrastructure
MockableReflectionOperations— a wrapper that delegates non-genericIReflectionOperationsmethods to a Moq mock, and implements the new generic methods by filtering mock results. This solves the problem of Moq not supporting generic methods with type constraints.TestablePlatformServiceProvider.SetupMockReflectionOperations()to use the new wrapper.ReflectHelperTestsintoReflectionOperationsTests(attribute caching,IsAttributeDefined,GetTestCategories, etc.).ReflectionUtilityTests(integration tests) to useAssembly.LoadFrominstead of the removedAssembly.ReflectionOnlyLoadFrom.GetAttributes<T>generic method on both unit and integration test levels.GetTestPropertiesAsTraitsnull-ReflectedTypetest,GetAttributes<T>happy-path tests, andMockableReflectionOperationsnull-guard test.