Repository navigation
Plan: mutation rules, stage 1 (#323) - #417
berkgeveci wants to merge 4 commits into
Conversation
Structure sealed at the buffer once built, value-derived caches on revisions, sent arrays read-only to their sender; the prototype (experiment/mutation-rules) passes the whole suite with structure sealed, costs about 18 ns per written array a call, keeps zero-copy export, and found Set flags going stale when a set's ids are written.
| - **Implicit arrays** have no buffer; sealing one seals the stored arrays it | ||
| holds. | ||
|
|
||
| Constructors keep the caller's arrays, so **the caller's arrays are sealed |
There was a problem hiding this comment.
In situ, a solver often keeps its mesh in its own Tack arrays and wraps them as a topology for analysis. With the seal on the buffer, the solver’s own kernels can no longer write that memory, for example to remesh or refine in place between steps. Copying every step costs GPU memory, which in situ can’t spare. What’s the intended pattern? Perhaps a new topology per step over memory the solver still owns, or a seal that applies only to buffers Tack allocated.
There was a problem hiding this comment.
Right, and it was worse than one step: the seal is one-way and holds the memory, so a solver's buffer would stay sealed even after the topology was dropped. Revised D2 (539d964, 265d62f) to use the per-array flag instead, which marks who holds the memory rather than the memory itself:
Array.read_only(), public: a view over the same memory that cannot write, with no copy.- Giving an array to a constructor hands it over, as D5's sending does: that object becomes read-only to its caller, so a stray
connectivity.fill(0)is still refused at the write. - In situ, the solver passes
conn.read_only()and keepsconn, the only writable handle, to remesh in place. Everything Tack gives out from the topology is read-only, so analysis code cannot write the simulation's mesh by mistake. - The topology records the revision of the arrays it was given, and its structural caches check it on use, one check for the topology; when the owner has written, the derived structure is dropped and rebuilt.
Only memory Tack allocates for derived structure is sealed, so a seal never reaches a caller's buffer. D4 also shows the same read_only() pattern for a simulation's field values, so an adaptor can protect them from analysis code while the solver keeps writing.
|
|
||
| ### D4. Every cache derived from values tracks their revision | ||
|
|
||
| Values stay writable, so a cache derived from them must notice a write: an |
There was a problem hiding this comment.
Revisions only see writes Tack makes. A solver writing positions or fields through DLPack or wrap_ptr, as a Catalyst adaptor does every step, moves nothing, so a cached locator still serves() moved points. Could stage 1 add an explicit “these values changed” call (like VTK’s Modified()), with the in-situ docs saying to call it each step?
There was a problem hiding this comment.
Yes. Added to D4 in 539d964: Array.modified(), VTK's Modified(), which counts a write to the array's buffer as Tack's own writes are counted, so the locators, the other value caches and D2's topology check all notice. The in-situ docs will say to call it on each array written outside Tack, each step; D6's checks stay the safety net for a forgotten call.
| (`_writable`), not D1's buffer seal, which would lock the receiver too. Its | ||
| limit: views of the sent array made before sending can still write. | ||
|
|
||
| A sender that reuses one send buffer step after step now gets an error: a |
There was a problem hiding this comment.
Reusing one send buffer every step is the normal pattern for a solver’s halo exchange, so this breaks solver code that uses tack.communication, even if Tack itself doesn’t do it. Could exchange take an opt-out (the sender keeps writing at its own risk), or could the plan name the supported reuse pattern?
There was a problem hiding this comment.
Agreed. The hazard turns out to be narrower: across ranks exchange waits for every request before returning, so the receiver already has its own copy and reusing the send buffer is safe; only a same-rank receiver (and the threaded test world) gets the sender's memory. Revised D5 in 539d964: exchange(..., keep=True) keeps the sender's arrays writable and gives a same-rank receiver a copy. That is the supported pattern for reused halo buffers, and without it the error names keep=True.
| `DeviceBuffer` gets a one-way `sealed` flag: set once, never cleared, shared | ||
| by every array over the buffer. Host writes and kernel stores check it beside | ||
| `_writable` and refuse with a message that says what the array is and to copy | ||
| it. A copy is writable. DLPack exports of sealed memory are flagged read-only. |
There was a problem hiding this comment.
Do the main consumers accept read-only DLPack? It’s worth checking PyTorch and CuPy on a sealed connectivity before this lands, since some refuse read-only capsules.
There was a problem hiding this comment.
Checked PyTorch 2.15 against a read-only Tack array: it asks for DLPack 1.0, gets the read-only flag, accepts the capsule and ignores the flag, so the tensor is writable and a write through it reached the Tack array. NumPy 2.4 honours the flag. So nothing refuses sealed memory; the risk is the other way, a consumer writing it. A legacy (pre-1.0) consumer gets it as writable, since that protocol cannot carry the flag. D1 now says the seal stops at DLPack, exports stay zero-copy, and a consumer's writes are outside D1, for D6. CuPy is listed as unchecked (it needs a CUDA machine).
|
Yes to D1 through D4 and D6. No outside users means the break needs no warning first. The open points are our own in-situ patterns: solver-owned meshes written between steps (D2), values written outside Tack’s view (D4) and reused send buffers (D5). Each needs a named pattern before the code hits. |
A caller's own structure is not sealed: the topology holds read-only views of it and drops its derived structure when the arrays' revision moves (D2). Array.modified() counts writes Tack cannot see (D4). exchange(keep=True) lets a sender reuse its buffers, copying for a same-rank receiver only (D5). The seal stops at DLPack: NumPy honours the read-only flag, PyTorch 2.15 ignores it (D1).
…flag Array.read_only() gives a view that cannot write. Giving an array to a constructor makes that object read-only to its caller, so a stray fill is still refused; in situ the solver passes a read-only view and keeps the only writable handle, and the topology notices its writes by revision. The same pattern protects a simulation's field values (D4).
A plan for #323, for review before the code. Comment on any line of
docs/design/plans/mutation-rules.md. I've said yes to every recommendation (D1–D6); a "yes", "no" or "other" per decision from each of you is what it needs. The "Open questions" section has three things the plan leaves open.Stacked on #416, which adds
docs/design/plans/and its index: the first commit here is #416's. Merge #416 first; this PR then shows its own commit alone.The plan in short
A dataset's structure (topology ids, space tables, and faces, edges and links derived from them) becomes read-only once built, for every array over that memory. Values stay writable, and every cache derived from them notices a write. An array sent through
tack.communicationbecomes read-only to its sender. Nothing is copied.Why:
tack.datasetkeeps about 40 derived caches, and none of those derived from structure checks for changes. Writing a connectivity after its faces were derived silently gives wrong answers.sealedflag on the memory (DeviceBuffer), checked on host writes and kernel stores.Value immutability, dataset generations and a reset are later stages. They wait on caches without reference cycles, a tested in-situ loop, and a rule for
update_ghosts.Prototype (in the plan's "Prototype" section)
A local branch,
experiment/mutation-rules:Setflags going stale when a set's ids are written; keyed on the ids' revision, they follow the write.mpiexec -n 3.Docs only.
mkdocs build --strictpasses.🤖 Generated with Claude Code