[devmotion]

Prototype of a redesign of the tag machinery, opened as a draft to discuss the design. Fixes #714, #801 and #845.

Motivation

Nested Duals represent a tensor product D_A ⊗ D_B of dual-number algebras, one factor per tag. Preventing perturbation confusion requires only

• R1: distinct live perturbations have distinct tags, and
• R2: values and partials are only ever accessed by tag (coefficient maps of the D_T factor; identity and zero if T is absent).

An order of tags is not needed mathematically. It only fixes a canonical storage order, so that binary operations just compare the outermost tags, each element has a single type, and seeding is plain wrapping. But since operations rely on it, the order must be the same in all code exchanging Duals, including precompiled code. tagcount violates this (#714, #801).

Changes

1. value(T, d), partials(T, d), partials(T, d, i), valtype(T, D) descend through layers with other tags and return identity/zero if T is absent, independent of the order (R2).
2. Accessors without a tag are deprecated; tags must be types.
3. tagcount is replaced by a structural total order: a generated function returns a constant (rank, key...) ID per tag, and ≺ compares IDs (folded at compile time). The rank strictly increases under containment, so Tag{F,V} is greater than all tags in F and V. DualMismatchError is removed. The ID itself could be improved; what matters is that it is unique per tag, totally ordered and independent of the session, and that ≺ is folded to a constant Bool at compile time, as tagcount comparisons were.
4. The constructor requires tags to be strictly decreasing inwards, so the tags of a nested Dual are unique (R1 within a number). Hence HessianConfig uses a separate gradient tag (hessian: the two nested dual layers share a tag, so results depend on the code path and chunk size #845), and untagged Duals and f = nothing configs use Tag{Nothing,V}, so nesting them yields distinct tags by containment. Similar to the strict order, and for efficiency reasons, the constructor also requires a concrete value type (e.g. Dual{T,Real} is disallowed), so the type of a Dual determines all its tags. Containment does not guarantee that the tag of a call is greater than the tags of its input (e.g. an abstract input type hides them), so seeding keeps layers of the input with greater tags outside. Elements of abstract input arrays are seeded with their own concrete types, in work buffers with element type Real.

Results (ForwardDiff 1.4.6 vs. this PR)

	1.4.6	This PR	Changes
Siskind & Pearlmutter 2005, #83, jacobian of gradient, #238, #267, constant inner derivative, nesting not ordered by containment, nested calls with abstract input types	correct	correct
#714: nested derivative in precompiled code	Cannot determine ordering of Dual tags	correct	3
#801: packages precompiling tags in opposite orders	order depends on load order, confusion	same order, correct	3
#845: hessian, hessian!, DiffResults, chunk sizes, SVector	disagree with jacobian of gradient or throw	agree	4
Hand-built Dual with the same tag or a greater tag inside	silently accepted	error	4
Nested untagged Duals	Dual{Nothing,Dual{Nothing}}, confusion	distinct tags	4
#313, #443: calls with the same function and input types	silently wrong	silently wrong
#320: function and config deserialized separately	InvalidTagException, correct with Val(false)	same
4th-order nested derivative / gradient(rosen, 100) / hessian(rosen, 100)	24 ns / 7.8 µs / 1.93 ms	24 ns / 7.8 µs / 1.93 ms

Comparison with other PRs

• Fix #714: Make TAGCOUNT always greater than the last instantiated Tag's count #724 keeps tagcount and raises TAGCOUNT when a Tag is constructed at runtime. It does not cover tags that are only constructed as types, and the order still depends on the session.
• Add tagdepth fast path to tag comparison #807 (closed) and Order nested Dual tags by provable containment (supersedes #807) #820 order tags by depth or containment and fall back to tagcount. They still assume that the order matches the nesting, which containment does not guarantee: the review of Order nested Dual tags by provable containment (supersedes #807) #820 shows a Dual buffer in the differentiated function that makes correct code throw, and that ≺ is no longer transitive. Here containment only determines the rank in a total order, and extraction does not rely on the order matching the nesting (R2). The counterexamples from the reviews of Add tagdepth fast path to tag comparison #807 and Order nested Dual tags by provable containment (supersedes #807) #820 are correct on this branch.
• Add SmallTag type for more compact Dual types #748 (reverted) identified tags by hash, which is unsound since hashes of distinct types can collide. Here tags are still compared as types.
• Proposal: Tagging API #815 proposes an API for alternative tag systems. This PR changes the default tag system instead; restricting tags to Tag (see below) would conflict with it.

Breaking changes

• Untagged Duals and f = nothing configs have tag Tag{Nothing,V} instead of Nothing.
• Nesting a Dual in a Dual with the same or a greater tag throws an error.
• Duals with a non-concrete value type, such as Dual{T,Real}, throw an error. Packages allocating their own Dual buffers for abstract input arrays are affected.
• HessianConfig is parametrized on the type of its gradient config; work buffers for abstract input arrays have element type Real; DualMismatchError is removed; non-type tags throw an error.

Not addressed

R1 across numbers: Tag{F,V} is a static proxy for a fresh tag per call, so calls with the same function and input types share a tag.

• Perturbation Confusion II: The Tag Wars #313 (live collision): an inner call uses the tag of an enclosing call. Tracking the active tags in a ScopedValue would detect this on entry and could throw a descriptive error (or retag the inner call).
• Perturbation Confusion issue #443 (stale collision): a Dual leaks from a finished call through mutable state (a Core.Box) into a later call with the same tag. The tag is no longer active, and types cannot distinguish the stale Dual from a fresh one. Detecting this would require walking the state reachable from f and x (incomplete, e.g. globals) or a per-call identity stored in each Dual (too costly).

Another possible follow-up is restricting tags to Tag (no custom ≺). The same analysis applies to ReverseDiff#45; a small shared interface (active tags plus descending through wrappers of other packages) could let both packages interoperate.

🤖 Generated with Claude Code

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.54%. Comparing base (a3c0f4f) to head (8fdf29f).

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #853      +/-   ##
==========================================
+ Coverage   91.23%   93.54%   +2.31%     
==========================================
  Files          11       12       +1     
  Lines        1072     1146      +74     
==========================================
+ Hits          978     1072      +94     
+ Misses         94       74      -20     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[longemen3000]

The new structure switches the need for tagging API away from the tag type itself and into tagid, making it easier for structs with captured variables to provide their inner dual state if available.

For example, #313 could be resolved by the owner of the mutable struct by defining an appropiate tagid method depending on the whoami field.

#815 conflicts with this PR, because what needs to be exposes as an API changes. so merging this PR should close #815.

[Technici4n]

Generally I like it, but isn't the following part sufficient to solve the tagcount problems?

tagcount is replaced by a structural total order: a generated function returns a constant (rank, key...) ID per tag, and ≺ compares IDs (folded at compile time). The rank strictly increases under containment, so Tag{F,V} is greater than all tags in F and V. DualMismatchError is removed. The ID itself could be improved; what matters is that it is unique per tag, totally ordered and independent of the session, and that ≺ is folded to a constant Bool at compile time, as tagcount comparisons were.

Of course the other points might be useful, but this above seems to be the key aspect.

[devmotion]

I'd argue it's not.

In theory, you only need R1 and R2. In practice, you need something that is a safe and efficient implementation of both conditions. With tagless partial or value extraction it's unsafe (point 1). Without strictly monotonically ordered nested duals any operations on duals will be inefficient (point 4). An order that often does not put the currently active level of differentiation on the top level of the duals will be inefficient as well (point 3).

But the exact definition of the order is arguably secondary to the proposal. I'm still not convinced the proposed definition is a good choice, it's actually the only point I'm still unsure about. It still feels too much of a hack to me.

[Technici4n]

I'm still not convinced the proposed definition is a good choice, it's actually the only point I'm still unsure about.

Which part exactly?

To be fair I only have a limited understanding of which changes this PR is proposing. I only meant to comment that the general idea looked reasonable to me at a glance. ;)

[devmotion]

Sorry, that was a bit unclear. I was referring to the choice of rank and key in

a generated function returns a constant (rank, key...) ID per tag