test: extract the shared test infrastructure project with an emulator partition budget by xperiandri · Pull Request #39 · fsprojects/FSharp.Azure.Cosmos

xperiandri · 2026-10-02T23:03:47Z

Proposed Changes

Phase 0 step 3 of ADR 0001 (#34): a shared test infrastructure project, so the test projects that follow (the SQL model, the quotation translator, the Hedgehog MSTest adapter) reuse one emulator setup. Stacked on #37 (net10.0); GitHub retargets it to main when #37 merges.

tests/Cosmos.Tests.Infrastructure (library, IsTestProject=false, IsPackable=false, references MSTest.TestFramework only): Assert/CosmosAssert helpers, FakeFeedIterator/FakeFeedResponse, the emulator base classes and the emulator category, moved from tests/Cosmos.Tests with their namespaces, so test names and categories are unchanged.
Partition budget: one process-wide budget (COSMOS_EMULATOR_PARTITION_COUNT, default 25, the Windows emulator's default partition count). A fixture takes all its permits atomically before creating its database and returns exactly what it took in finally; database and container creation is serialised. With it the suite passes at the default 32 workers, where the local emulator answered 500/503 on container creation before (also on main).
Stable database names fsac-test-<hash of class and test name>_<test name>[_<row hash>], at most 80 characters, with an explicit set of replaced characters.
Leftover sweep in [<AssemblyInitialize>]/[<AssemblyCleanup>]: deletes only fsac-test- databases not modified for an hour (COSMOS_TEST_LEFTOVER_AGE_MINUTES), so concurrent runs on one emulator keep each other's live databases; a test that meets its own database from an aborted run recreates it empty.
Endpoint and key from COSMOS_EMULATOR_ENDPOINT/COSMOS_EMULATOR_KEY, defaulting to the documented emulator values; Gateway mode and the local-only certificate check are kept.
Build: the project is in both .slnx and .slnf; testsGlob narrowed to tests/**/*.Tests.??proj so dotnet test runs only test applications; Clean, Fantomas and the build still cover the helper project; the coverage filter excludes it; tests/Directory.Build.props references it from every test project.
.github/copilot-instructions.md: the project in the solution tree and the environment variables.

Types of changes

Bugfix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)

Test infrastructure only; the package is unchanged.

Checklist

Build and tests pass locally
I have added tests that prove my fix is effective or that my feature works (if appropriate)
I have added necessary documentation (if appropriate)

Further comments

Debug and Release builds: the same 19 nullness warnings as the base (5 moved with Assert.fs); Fantomas clean.
Tests: 154/154 (119 existing + 35 new unit tests for the identifier, the budget and the sweep) in Debug and Release at the default parallelism on the local emulator, after rebasing onto chore!: target net10.0 #37; build.cmd DotnetTest runs only FSharp.Azure.Cosmos.Tests.fsproj; no fsac-test- databases are left after a run.
A probe showed each container costs one emulator partition and databases cost none.
The budget and the sweep are per process: two test processes on one emulator must share its capacity through COSMOS_EMULATOR_PARTITION_COUNT.

🤖 Generated with Claude Code

Copilot

Copilot review overview

🟡 Changes recommended

Distinct data-row value types can currently produce identical database identifiers and interfere with parallel tests.

Review effort: Balanced
Findings: 1

Open (1)

Prevent row ID collisions from ambiguous value encoding · New

What changed in this PR

Extracts reusable Cosmos emulator test infrastructure and adds resource budgeting for parallel tests.

Changes:

Adds shared fixtures, assertions, fakes, database naming, partition budgeting, and leftover cleanup.
Adds unit coverage for identifiers, budgeting, and sweep rules.
Updates solution, build, package, and test-project wiring.

File	Description
`.github/copilot-instructions.md`	Documents infrastructure and emulator settings.
`Directory.Packages.props`	Adds the MSTest framework version.
`FSharp.Azure.Cosmos.slnf`	Includes the infrastructure project.
`FSharp.Azure.Cosmos.slnx`	Includes the infrastructure project.
`build/build.fs`	Separates test applications from helper projects.
`tests/Directory.Build.props`	References shared infrastructure from test projects.
`tests/Cosmos.Tests.Infrastructure/Assert.fs`	Moves and documents F# assertions.
`tests/Cosmos.Tests.Infrastructure/CosmosAssert.fs`	Moves and documents Cosmos assertions.
`tests/Cosmos.Tests.Infrastructure/DatabaseIdentifier.fs`	Generates stable test database identifiers.
`tests/Cosmos.Tests.Infrastructure/Emulator.fs`	Configures clients and leftover cleanup.
`tests/Cosmos.Tests.Infrastructure/FakeFeed.fs`	Extracts reusable feed fakes.
`tests/Cosmos.Tests.Infrastructure/FSharp.Azure.Cosmos.Tests.Infrastructure.fsproj`	Defines the shared helper library.
`tests/Cosmos.Tests.Infrastructure/IntegrationInfrastructure.fs`	Implements emulator-backed fixture lifecycle.
`tests/Cosmos.Tests.Infrastructure/PartitionBudget.fs`	Adds atomic partition permit management.
`tests/Cosmos.Tests.Infrastructure/TestCategories.fs`	Provides the shared emulator category.
`tests/Cosmos.Tests/DatabaseIdentifierTests.fs`	Tests identifier generation.
`tests/Cosmos.Tests/FSharp.Azure.Cosmos.Tests.fsproj`	Updates test compilation inputs.
`tests/Cosmos.Tests/IntegrationInfrastructure.fs`	Removes infrastructure moved to the library.
`tests/Cosmos.Tests/IterationExtensionsUnitTests.fs`	Uses the extracted feed fakes.
`tests/Cosmos.Tests/LeftoverSweepTests.fs`	Tests leftover age decisions.
`tests/Cosmos.Tests/PartitionBudgetTests.fs`	Tests partition-budget concurrency.
`tests/Cosmos.Tests/ReadExtensionsTests.fs`	Routes hierarchical container creation through the fixture.
`tests/Cosmos.Tests/TestAssembly.fs`	Adds assembly-level leftover sweeps.
`tests/Cosmos.Tests/TestCategories.fs`	Updates project-specific categories.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

xperiandri · 2026-10-02T23:08:22Z

        else
            message

+    /// <summary>Returns the resource of an HTTP 200 or 201 <paramref name="response"/>; fails the test with <paramref name="message"/> on any other status.</summary>


Make multiline in this file

Suggested change

/// <summary>Returns the resource of an HTTP 200 or 201 <paramref name="response"/>; fails the test with <paramref name="message"/> on any other status.</summary>

/// <summary>

/// Returns the resource of an HTTP 200 or 201 <paramref name="response"/>; fails the test with <paramref name="message"/> on any other status.

/// </summary>

Done in this file and in every other file of the pull request: no one-line <summary> with tags is left (3864ff5, 1875924, 27763ce), wrapped at 120 characters.

🤖 Generated with Claude Code

xperiandri · 2026-10-02T23:10:02Z

+    /// <param name="test">
+    /// <list type="bullet">
+    /// <item><description><c>ClassName</c> – the fully qualified name of the test class.</description></item>
+    /// <item><description><c>TestName</c> – the test method name; <c>ValueNone</c> in a class-level context such as


I asked to reference types and members using <see!!!!!!!!!!!!

Fixed here and across the whole pull request: every reference to a type, member, field or record field in the doc comments is now a <see cref> (parameters <paramref>), <c> is left only for literal values, and all 203 crefs resolve. The <c>ClassName</c> list is gone: the parts are now documented fields of the TestIdentity record.

🤖 Generated with Claude Code

xperiandri · 2026-10-02T23:10:23Z

+            struct {|
+                ClassName : string
+                TestName : string voption
+                TestData : (obj | null) array | null


Suggested change

TestData : (obj | null) array | null

TestData : (objnull) array | null

Done: TestData : objnull array | null (1875924), a nullable array of nullable objects, as TestContext.TestData declares it. No obj | null is left in the pull request.

🤖 Generated with Claude Code

xperiandri · 2026-10-02T23:12:31Z

+/// test databases left behind.
+/// </summary>
+[<RequireQualifiedAccess>]
+module Emulator =


Declare as root module

Done: Emulator.fs now declares module FSharp.Azure.Cosmos.Tests.Integration.Emulator at the top, after its docs and [<RequireQualifiedAccess>] (1875924).

🤖 Generated with Claude Code

xperiandri · 2026-10-02T23:16:03Z

+            DisplayName = null
+        |}
+
+    static let rowIdentifier (testData : (obj | null) array) =


use objnull alias

Suggested change

static let rowIdentifier (testData : (obj | null) array) =

static let rowIdentifier (testData : objnull array) =

Done: rowIdentifier (testData : objnull array) (1875924).

🤖 Generated with Claude Code

xperiandri · 2026-10-02T23:16:36Z

+        DatabaseIdentifier.create struct {|
+            ClassName = "Tests.Class"
+            TestName = ValueSome "Data-driven test"
+            TestData = testData
+            DisplayName = null
+        |}


Why this struct is anonymous?

It should not have been: anonymous records are for return types. The four parts are now the named [<Struct>] TestIdentity record, each field documented on the field, and DatabaseIdentifier.create takes it (1875924).

🤖 Generated with Claude Code

xperiandri · 2026-10-02T23:25:35Z

    <PackageVersion Include="Microsoft.NET.Test.Sdk" Version="18.5.1" />
    <PackageVersion Include="Microsoft.SourceLink.GitHub" Version="10.0.401" />
    <PackageVersion Include="MSTest" Version="4.2.3" />
+    <!-- Keep at the MSTest version: the helper libraries under tests/ reference the framework package alone -->


Not necessary

Suggested change

Removed (3864ff5). The MSTest.TestFramework version entry itself stays: without it restoring the infrastructure project fails with NU1010.

🤖 Generated with Claude Code

Move the code that later test projects will share out of tests/Cosmos.Tests into the library tests/Cosmos.Tests.Infrastructure (ADR 0001, section 5 and 22): the fixture and base classes (IntegrationInfrastructure.fs), the Assert and CosmosAssert helpers, the FakeFeedIterator/FakeFeedResponse fakes and the "Cosmos DB Emulator" category, which the emulator base class now carries as an attribute class instead of a string. The types keep their namespaces, so the tests compile unchanged; test-specific code (TestItem, OperationTestBase, scenarios, the per-operation categories) stays in the test project. The library sets IsTestProject=false and IsPackable=false explicitly and references MSTest.TestFramework only, never the MSTest metapackage. Central package management requires a PackageVersion for every PackageReference, so Directory.Packages.props lists MSTest.TestFramework next to MSTest; without it the restore of the library fails with NU1010. tests/Directory.Build.props references the library from every test project except itself. It is added to both FSharp.Azure.Cosmos.slnx and .slnf, testsGlob is narrowed to tests/**/*.Tests.??proj so `dotnet test` and `dotnet watch test` run only test applications, Clean still covers every project under tests/, and the coverage report excludes the infrastructure assembly. Every public member of the library is documented. The documentation refers to types, members and union cases through <see cref>, FSharp.Core union cases by the documentation IDs FSharp.Core.xml documents them under (such as T:Microsoft.FSharp.Core.FSharpValueOption`1.ValueNone), the assertion families through one example pair each, and keeps <c> for literal values. The F# compiler copies crefs verbatim and never resolves them, so they were checked against the built assemblies. Every <summary> tag sits on a line of its own, with the text wrapped at 120 characters without breaking inside a tag. In build.fs, the doc comments of testsGlob and of the ==>! and ?=>! operators follow the same rules. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Replace the database identifier of the integration fixtures with the algorithm of ADR 0001 section 22.3, in the DatabaseIdentifier module. The identity of a test is the TestIdentity struct record: the fully qualified class name; the test name, read from TestContext.Properties["TestName"] and ValueNone in class-level contexts, where TestContext.TestName throws; the data row, a nullable array of nullable objects as TestContext.TestData declares it; and the display name of the row. Each field is documented on the field itself. An identifier is the common prefix fsac-test-, a short hash of the class name and the test name together, separated by a line break, which occurs in neither, and the readable test name, in which the characters of an explicit, platform-independent set (the / \ ? # Cosmos DB forbids, the other characters Windows forbids in file names, % and whitespace) become _ and which is cut to fit 80 characters. The readable name alone does not tell tests apart: equally named methods of two classes share it, and so do names such as "a/b", "a b" and "a_b" or names that differ only beyond the cut, and parallel tests that share a database delete it under each other. The hash does tell them apart, so a cut name needs no hash of its own. A class-level identifier hashes the class name alone and has its own separator. A data-driven test gets a hash of its row appended. The row is encoded unambiguously before it is hashed: every value is written as the name of its runtime type followed by its text, each with its length in front; the items of a sequence are encoded the same way and framed as a whole, null is an empty type name, and the display name follows the values. So 1 as int and 1L as int64 get different databases. The type name comes from Type.ToString rather than Type.FullName, whose generic arguments carry assembly versions that would change the identifier with the runtime. What remains ambiguous, two values of one type whose text is equal and two types of one full name from different assemblies, is documented along with why it is acceptable. Every hash is SHA-256 over the UTF-16 code units of its text in little-endian order: not String.GetHashCode, which is randomised per process, and not UTF-8, which replaces every lone surrogate by U+FFFD. The endpoint and key come from COSMOS_EMULATOR_ENDPOINT and COSMOS_EMULATOR_KEY when set, with the documented emulator values as defaults; the client keeps Gateway mode and the HTTP client that accepts the self-signed certificate from local hosts only. They live in Emulator, a top-level module. An [<AssemblyInitialize>] and an [<AssemblyCleanup>] hook sweep leftover databases: those with the prefix that stayed unmodified for the leftover age, read from _ts through DatabaseProperties.LastModified. The prefix cannot tell test processes apart, but the age can: a test keeps its database only while it runs, so the sweep keeps the live databases of another test process on the same emulator, such as another clone, a CI agent or a parallel branch. The age is 60 minutes by default, overridden in whole minutes by COSMOS_TEST_LEFTOVER_AGE_MINUTES, where 0 sweeps every test database, for a run that has the emulator to itself. A database modified after the sweep began or reported without _ts is kept, and the kept ones are counted in the TestContext. A malformed variable skips the sweep instead of guessing, and so does an emulator that does not answer, so emulator-free test runs are not slowed down. Failures are written to the TestContext and never fail the run. A test still deletes its own database in its cleanup, whatever its age. Since the sweep keeps young databases and identifiers are stable, a rerun of a test whose earlier run was aborted, or whose cleanup failed, meets the old database with its containers and items, and a scenario seed then fails with a conflict on an item the earlier run created (reproduced with ReadOperationIntegrationTests against a planted database). The fixture therefore recreates its database when CreateDatabaseIfNotExistsAsync reports that it existed already and writes that to the TestContext, so every test starts with an empty database and the old containers free their partitions. A normal run, where the database is new, makes no extra request. Emulator-free unit tests cover the identifier, with expected values computed independently in PowerShell from the documented format, and the decision of the sweep, Emulator.isLeftover, a pure function over the minimum age, the moment of the sweep and the last modification, together with the conversion of _ts into an instant. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Keep the integration fixtures within the limits of the local emulator, which answers HTTP 500 and 503 on container creation at the default parallelism (ADR 0001 section 22.4). A process-wide PartitionBudget, sized by COSMOS_EMULATOR_PARTITION_COUNT (default 25, the Windows emulator's default), hands out one permit per container. A fixture declares how many containers it creates (ContainerCount, default 1) and takes all their permits at once before it creates its database: only one caller at a time is between its first and its last permit, so fixtures can no longer each hold part of what they need and wait for each other. A cancelled acquisition gives back what it took, and the fixture releases exactly the number it acquired, in a finally block. Creating a container beyond ContainerCount fails, and the hierarchical container of the read extension tests is now created through the fixture. The partition count is read by a function rather than a value, so a malformed variable fails only the fixtures that need the budget. A second semaphore serialises database and container creation, the recreation of a database an earlier run left behind included, with the wait placed before the try. Cleanup is best-effort: a failed deletion is written to the TestContext and left to the next run of the same test, which recreates the database, or to the leftover sweep once it is old enough, instead of failing the test, and a fixture whose initialization failed half-way is cleaned up by [<TestCleanup>], which MSTest also runs after a failed [<TestInitialize>]. Setup, seeding and cleanup pass the TestContext cancellation token; cleanup reads it at cleanup time, when MSTest has replaced the token source of a timed-out test. The documentation names PartitionBudget, the fixture and the sweep through <see cref>, and that of the hierarchical container names PartitionKeyBuilder.AddNoneType for the level its items lack. Emulator-free unit tests cover the budget, including a scenario in which taking the permits one at a time deadlocks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…riables The shared test infrastructure project was missing from the solution tree of the agent instructions, and the environment variables it reads were only discoverable from its source. Agents and contributors running the tests against another emulator, a smaller emulator or next to another test process need to know COSMOS_EMULATOR_ENDPOINT, COSMOS_EMULATOR_KEY, COSMOS_EMULATOR_PARTITION_COUNT with its default of 25 and COSMOS_TEST_LEFTOVER_AGE_MINUTES with its default of 60, and that a new test project gets the infrastructure through tests/Directory.Build.props rather than a ProjectReference of its own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot

Copilot review overview

🟢 Approval recommended

The extraction, emulator safeguards, build integration, and associated tests are internally consistent with no unresolved defects found.

Review effort: Balanced
Findings: None

Resolved since last review (1)

Prevent row ID collisions from ambiguous value encoding

#39 moved the emulator fixture into tests/Cosmos.Tests.Infrastructure and added the partition budget, the database identifier and the leftover sweep, so the ADR's paths into tests/Cosmos.Tests and its list of what the repository still lacked were out of date. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI balanced review requested due to automatic review settings October 2, 2026 23:03

Copilot started reviewing on behalf of xperiandri October 2, 2026 23:04 View session

Copilot AI reviewed Oct 2, 2026

View reviewed changes

Comment thread tests/Cosmos.Tests.Infrastructure/DatabaseIdentifier.fs Outdated

xperiandri mentioned this pull request Oct 2, 2026

test: probe IcedTasks computation expressions in Debug and Release #40

Open

3 of 6 tasks

Base automatically changed from chore/net10 to main October 2, 2026 23:21

xperiandri commented Oct 2, 2026

View reviewed changes

xperiandri force-pushed the test/infrastructure-project branch from b496eae to 7141b99 Compare October 2, 2026 23:26

xperiandri and others added 4 commits October 3, 2026 02:32

xperiandri force-pushed the test/infrastructure-project branch from 7141b99 to a7b9109 Compare October 3, 2026 00:47

xperiandri requested a balanced review from Copilot October 3, 2026 00:48

Copilot started reviewing on behalf of xperiandri October 3, 2026 00:48 View session

xperiandri mentioned this pull request Oct 3, 2026

feat: use valueTask for the feed iterator's MoveNextAsync #41

Open

3 of 6 tasks

Copilot AI reviewed Oct 3, 2026

View reviewed changes

xperiandri merged commit d1b040a into main Oct 3, 2026
8 checks passed

xperiandri deleted the test/infrastructure-project branch October 3, 2026 01:05

	TestData : (obj \| null) array \| null
	TestData : (objnull) array \| null

	static let rowIdentifier (testData : (obj \| null) array) =
	static let rowIdentifier (testData : objnull array) =

Conversation

xperiandri commented Oct 2, 2026

Proposed Changes

Types of changes

Checklist

Further comments

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟢 Approval recommended

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants