Skip to content

Commit ee0157d

Browse files
authored
feat(tables): add currency column type on a new column-type registry (#6106)
* feat(tables): add currency column type on a new column-type registry Adds a `currency` column type, and consolidates the per-type knowledge it would otherwise have been scattered across. **Currency.** Stores a plain number and carries an ISO 4217 `currencyCode` as display metadata. That split is what keeps it cheap: filtering, sorting, uniqueness and CSV export all reuse the numeric paths unchanged, changing a column's currency rewrites no rows, and the public row output stays a number rather than a locale-formatted string consumers would have to reparse. Input accepts the shapes an amount actually arrives in — `$1,234.56`, `1 234,56 €`, `(12.00)` — so pastes, CSV imports and tool writes land as numbers instead of being nulled. **The registry.** Adding this type initially required edits in ~40 places: 32 switch arms under `lib/table`, ~26 UI branches, two hand-maintained icon maps, and a coercion implementation duplicated four times. Every one of those failed silently when missed — a missing `jsonbCastForType` arm compares numbers as text; a missing compatibility arm blocks all conversions. `lib/table/column-types/` now holds one file per type carrying its label, icon, badge colour, storage cast, filter operators, coercion, validation, compatibility and formatting. `Record<ColumnType, …>` on both registries is the completeness gate: adding a type to the union is a compile error naming exactly the two files to fill in, and the interface then requires every field. The 32 switch arms are down to 3. Two duplicates collapse as a consequence: - The client no longer mirrors the server's select id-resolution. Those helpers lived in `validation.ts`, which imports drizzle, so anything reaching them became server-only and the grid hand-rolled its own copy. Extracting them to `select-options.ts` lets both sides share one implementation, so the optimistic cache can no longer disagree with what gets persisted. - The two icon maps become one registry read. It also fixes a live inconsistency it surfaced: currency got a numeric keypad in the grid's inline editor but a plain text field in the row modal. Behaviour-neutral by construction: all 1046 tests in the touched areas pass unchanged, with no test edits. * test(tables): guard the column-type registry's invariants Property tests for the registry itself rather than any one type: entries key by their own id, COLUMN_TYPES stays derived, an unknown type degrades to string instead of throwing, only opaque-id types restrict filter operators, only configuration-free types are CSV-inferable, and every type that can reject a draft has a message to show. Plus the metadata-ownership matrix, which pins the generic ownership check to the same answers the hardcoded per-type rules gave. These target the registry's silent-failure class — a wrong jsonbCast or a stray operator whitelist used to be invisible until a filter failed in SQL. Both are verified to fail under mutation. * fix(tables): read exponent-form amounts and reject bad currency PATCHes up front Two P1s from review. Scientific notation lost magnitude. `String()` emits exponent form past 1e21, so a stored amount round-trips through the editor as `1e+21` — and the sanitizer treated the `e` as decoration to strip, reading it back as 121. An untouched cell silently lost 19 orders of magnitude on its next edit. Exponent form is now taken at face value, but only when the string is wholly a numeric literal once symbols are removed, so `12 EUR` (whose `E` survives the strip) still parses through the separator path. A failed currency PATCH left a partial rename. `renameColumn` commits in its own transaction before the currency write, so a `currencyCode` the service would reject — an unsupported code, or any code on a non-currency column — errored only after the rename had stuck. Both are now caught before the first write, matching the guard the route already applies to unique-on-select for exactly this reason. * refactor(tables): finish the registry migration and drop the dead config Audit pass over every consumer, closing the gaps the first cut left. Functional gap: the copilot agent had no currency support at all — it could create a currency column with no code and could never re-denominate one. `add_column` and `update_column` now accept `currencyCode`, with the same up-front validation and the same code-only routing as the HTTP routes. Config that consumers were still restating, now read from the registry: - `supportsUnique` replaces the unique-on-select guard stated in three places (service, both column routes, the copilot tool). - `editor === 'toggle'` replaces seven `type === 'boolean'` checks in the grid and expanded popover, all of which meant the same thing. - `defaultMetadata` replaces the per-type stamping in `addTableColumn` and `updateColumnType`. - `sampleValue` replaces the per-type example values in the LLM prompt scaffolding. - `storesOpaqueIds` replaces the select filter in the find-row matcher. Dead config removed: `getTypeBadgeVariant` had zero callers (already dead on staging), and it was the only reader of `badgeVariant` — so the field, its union, and all seven values went with it. `inferFromCsv` was read by nothing but a comment; CSV inference is an ordered heuristic a boolean cannot express, so it is gone too and `InferredCsvColumnType` is no longer exported. Fixes a latent crash found on the way: unique-constraint checking normalized a cell keyed on its RUNTIME type but reconstructed it keyed on the column's DECLARED type, so a unique `date` column stored a bare `2024-01-01` and then threw `SyntaxError` parsing it back. Both directions now go through JSON unconditionally. Pre-existing, unrelated to currency. Adds the `/add-column-type` skill and a Tables section in CLAUDE.md/AGENTS.md pointing at it, so the next type is one file plus two registry entries. * fix(tables): run the column PATCH guards ahead of the rename, not after it Greptile was right and my previous reply was wrong. The guards were added in the right shape but the wrong place — below `renameColumn`, which is the first write and commits in its own transaction. A PATCH combining a rename with an invalid currency therefore still committed the rename and then returned 400, exactly the counterexample reported. Moved the column lookup and all three pre-flight guards above every write. This also closes the same latent hole for the pre-existing unique-on-select guard, which sat in the same position. Adds route tests that assert `renameColumn` was never called on each rejection path, and that a valid combined rename + currency change still targets the new name. Verified to fail against the previous ordering. * fix(tables): make the retype gate and the write path share one parser A simplify pass over the registry found two real defects and several places the abstraction was being worked around. Silent data loss on conversion. `isCompatibleWith` was hand-written per type and had already drifted from `coerce`, despite the interface promising they could not: `boolean` accepted '1'/'0'/0/1 in the gate but only 'true'/'false' in the write path, so converting a column holding "1" reported zero incompatible rows and then nulled every one of them. `date` drifted the other way. `isCompatibleWith` is now optional and defaults to `coerce(...).ok`, so the two are the same code; only `select` overrides, because its rules are about the column (cleared-vs-required, cardinality) not the value. `isColumnType` used `in`, which matches inherited keys — `isColumnType('toString')` was true and `columnTypeById('toString')` returned `Function.prototype.toString`, which the validator would then call `.validateDefinition()` on. Now `Object.hasOwn`. `defaultMetadata` only ran on the currency arm of a retype, so a future type would get its defaults on create but silently not on conversion. It now runs for every non-select target, carrying forward only metadata the TARGET type declares it owns — a currency→text conversion no longer strands a currencyCode. The index doc claimed the registry is kept out of the `@/lib/table` barrel so 44 server modules don't pull `@sim/emcn/icons`. That was false: `constants.ts` re-exported `COLUMN_TYPES` from the icon-carrying `registry.ts`, and the barrel re-exports `constants`. `COLUMN_TYPES` now lives in the icon-free `types.ts`; verified with an import tracer that both are icon-free again. Also: 5 no-op `validateDefinition`s and 4 duplicated formatters collapsed into registry defaults; `CURRENCY_OPTIONS` was an eager module-load IIFE costing ~8ms of ICU work on every table API route for a list only the config sidebar reads, now built on first call; and the skill's validation grep claimed 'should return nothing' when it returns 8 legitimate hits — it now explains how to tell a leak from a genuine special case. * fix(tables): reject a non-leading sign so dates don't parse as amounts Found by Cursor Bugbot. `parseCurrencyInput` dropped every `-` as decoration, so an ISO date's hyphens vanished and its digit groups joined: `2024-01-01` read as 20240101. With the gate now sharing the write path's parser, a date → currency conversion reported zero incompatible rows and silently turned every cell into a huge number. A sign is only meaningful at the front; an interior one means the string is not a single amount. Leading signs, accounting parentheses, symbols, ISO codes, grouping separators, and exponent form all still parse — covered by the existing cases plus new ones, verified to fail without the fix. * fix(tables): use getErrorMessage in the columns route test mock `check:utils` bans the inline `e instanceof Error ? e.message : fallback` form; the mock for `rootErrorMessage` used it. * fix(tables): rename the column last so a failed write leaves it untouched Greptile's remaining concern: the pre-flight guards read a schema snapshot, so a column-type change landing concurrently can still make a later write fail — and with the rename running first, that failure returned an error with the rename already committed. Guards cannot close that window; each write is its own locked transaction and only the write itself sees the authoritative state. Ordering can. The rename is the one write that is purely cosmetic, so it now runs last: a failed typed write leaves the column entirely untouched, and a failed rename leaves the typed change applied under the old name — the recoverable half. The typed writes target the column's current name, since no rename has happened yet. Tests cover both directions: a typed write rejected mid-flight must not rename, and a successful one must rename strictly after. Verified to fail under the previous ordering. * fix(tables): write back coerced values on every conversion Round 4 findings, all real. A conversion is allowed exactly when the target type's `coerce` accepts the value — and `coerce` frequently TRANSFORMS it. Only `select` and `currency` wrote the transformed value back, so a conversion to any other transforming type left the cell holding its old bytes under the new type. Converting a number column to `date` accepted epoch values, stored them unchanged, and then `(data->>'col')::timestamptz` failed on EVERY query against that column. I opened this myself by defaulting `isCompatibleWith` to `coerce(...).ok`. Fixed at the class rather than the instance: the compatibility scan now records whatever `coerce` produced whenever it differs from what is stored, and one generic write-back applies it. That subsumes the currency-specific migration entirely, so it and its helpers are gone. `select` keeps its own id↔name migrations, which are not coerce-expressible in the outbound direction. The post-conversion column definition is built once, before the scan, so the coercion reads the same metadata the stored value is later validated against. Exponent parsing was ambiguous when followed by text: `1e5 EUR` read as 15. An `e` with a digit on both sides is an exponent marker, so if the string is not a clean numeric literal it is refused rather than guessed — the digit on both sides is what keeps the `E` inside `12 EUR` parsing normally. A failed rename could still leave a typed change committed. The one rename failure a caller can cause — a name already taken — is now rejected up front, leaving only the concurrent-collision race, which no pre-flight check can close without spanning all writes in one transaction. * fix(tables): stop a blank cell blocking an optional type conversion Found by Cursor Bugbot. `''` is incompatible with every numeric type, and the compatibility scan counted it as a hard blocker regardless of whether the target was optional — so a text column with a single empty cell could not be converted to a number at all, and the error said 'to a required ...' either way. An unreadable-but-empty cell is not a conversion failure. The write path already turns an unreadable value into null on an optional column, so the conversion now does the same and records null for it. A required target still reports it, which the existing guard above already does with the message that actually fits. Also pins the two intentional divergences from the pre-registry behavior. A differential run of the registry against the pre-refactor implementations (55 values x 7 column shapes) found ZERO coercion differences and exactly two compatibility differences, both deliberate: boolean now rejects the '1'/'0' conversions the old gate accepted and then nulled, and date now accepts the epoch numbers its write path always accepted. Tests pin both so neither can be silently reverted or widened. * fix(tables): refuse conversions that would invent or destroy values Final adversarial scan found two data-corrupting conversions, both opened by defaulting the retype gate to the write path's parser. number → date destroyed every value. `date.coerce` reads a number as epoch milliseconds, which is right for one deliberate write and catastrophic applied to a whole column: 1, 5, 42 became three timestamps in January 1970, and a Unix-seconds column landed in 1970 rather than the year it meant. Irreversible. `date` now overrides the gate to reject numbers, restoring the pre-refactor behavior, and the contract states the rule the override obeys: a gate may be STRICTER than `coerce`, never looser. Stricter refuses a bulk conversion while single writes still work; looser is the direction that corrupts. string → currency invented values. The parser stripped every non-digit and joined what was left, so `01/02/2024` read as 1022024, `Room 101` as 101, and `0.1.2` as 12 — a column of SKUs or phone numbers converted with zero reported incompatibilities. What remains after removing symbols, spacing and an ISO code must now be only digits and separators, and grouping must be well-formed (a first group of 1-3 digits, the rest exactly 3). Every legitimate form still parses, including all the locale variants. Also generifies the last three metadata leaks: `buildConvertedColumn` strips and carries back by iterating the key list rather than naming keys (naming them meant a future type's metadata rode onto a target that rejects it, failing that column's validation on every later write), `normalizeColumn` forwards metadata through a shared `typeMetadataOf`, and `filterOperatorsFor` moved onto the definition — it was a per-type branch inside the registry's own accessor, the one thing the registry exists to forbid. Skill corrected: it claimed COLUMN_TYPES derives from the registry (backwards), promised exactly two compile errors (four once a type owns metadata), used a grep that missed half the real branches, and never mentioned `import.ts`'s second coercion path, whose silent default arm is the costliest miss available. Differential re-run vs the pre-refactor implementations: 0 coercion differences, 1 intentional compatibility difference (boolean no longer accepts the 0/1 conversions the old gate accepted and then nulled). * fix(tables): let the row modal accept formatted amounts again Found by Cursor Bugbot. I unified the row modal's input type with the grid's `inputMode` last round, but in the wrong direction: mapping `inputMode: 'decimal'` to `<input type="number">` made the modal reject $1,234.56, 1.234,56 and (12.00) — the exact formats `parseCurrencyInput` exists to accept, and which the grid's inline editor takes fine. A native number input and a numeric keypad are different things. Types whose parser accepts formatted text now say so, and get a text field with `inputMode='decimal'` — the shape the grid already uses. A plain number keeps the native input, its spinner, and its validation. * fix(tables): fold a rename into the write it accompanies Closes the last partial-update window, properly rather than by pre-checking around it. A rename is metadata-only — `renameColumn`'s own comment says so: rows, metadata, and workflow-group refs all key on the stable column id, so it is a pure schema write. Nothing forced it to be its own transaction. Running it separately is what created the window: whichever half committed first survived a failure in the other, and no pre-flight guard can close a concurrent collision because only the write itself sees authoritative state. The four column writes now accept an optional `newName` and apply it through one shared `applyPendingRename`, which validates the name shape and checks the collision against the very schema snapshot that write is landing in. A combined request rides the rename on whichever write runs last, so both halves commit together or neither does — a concurrent claim on the name now aborts the whole transaction instead of leaving the other change applied. The routes also address every write by the column's stable id rather than its name, so folding a rename into one write cannot break the next one's lookup. A rename with nothing to ride on still runs standalone. What remains partial is a type write followed by a failing constraints write — two independently locked transactions, pre-existing, and untouched by this PR. * fix(tables): migrate scalar cells when converting a column to select Found by Cursor Bugbot. `resolveSelectOptionId` stringifies a number or boolean before matching, so a `number` column whose values equal option NAMES passes the compatibility gate — but `migrateCellsToSelectIds` only rewrote JSONB `string` and `array` cells. Those cells stayed raw numbers inside a select column, where they render as nothing and fail option membership on the next write. `data->>key` yields the text form for every scalar, so the existing lookup already worked; the predicate was simply too narrow. Widened to cover `number` and `boolean`. The outbound migration is unchanged — cells leaving a select column are option ids, always strings or arrays. Pre-existing on staging (both the resolver's scalar handling and the migration SQL predate this branch), but it lives in a file this PR creates. Tests pin the resolver behavior the predicate depends on, so narrowing either one without the other now fails. * fix(tables): validate a retype's unique against the values it writes Validated the last partial-update seam with a focused investigation rather than assuming. The answer was split. `required` is already safe: `updateColumnType` runs the same `countEmptyCells` against the constraint the request is about to set, which is why that check exists. `unique` was not, and the reachable case commits the unrecoverable half. A text column holding "5" and "5.0", PATCHed with {type: number, unique: true}: the conversion succeeds and coerces both to 5, then the separate constraint write finds duplicates and 400s — with the column already numeric and "5.0" irreversibly rewritten. A pre-scan of the raw text finds nothing; the conversion is what manufactures the duplicate. The retype now carries `unique` and checks it after the write-back, against the values it just wrote. Constraint changes on a workflow-output column were the same shape — rejected by the constraint write, after a type change had committed. Now rejected in the route's pre-flight block, before any write. The duplicate scan is extracted and shared between both paths for the same reason `countEmptyCells` is: two copies of one rule is the drift that produced the original required-check bug. Deliberately NOT merging `updateColumnType` and `updateColumnConstraints`. They assert different lock levels (destructive vs schema-only) and only the retype needs the full row scan, so merging would either force a constraints-only toggle to materialize every row or reintroduce the branching it was meant to remove. With both reachable failures pre-validated, what remains at the seam is concurrent races no in-process check can close. * fix(tables): don't drop a rename when the write it rides on no-ops Found by Cursor Bugbot — a bug I introduced folding the rename in. `updateColumnCurrency` returns early when the code is unchanged, and that return sat ahead of the rename, so PATCH {name, currencyCode} with the column's current code answered 200 with the rename silently discarded. Both early returns now treat a pending rename as work: the currency path only no-ops when the code is unchanged AND no rename is riding along, and the retype path applies a rename-only write when the type is unchanged. `applyPendingRename` signals "nothing to do" by returning the same reference, which is what lets both detect it cleanly. Also extracts `persistColumns` — five sites were repeating the same schema-write-and-return. * fix(tables): make a combined column PATCH a single transaction Finishes the fold-in rather than pre-validating around the seam. A retype now APPLIES the constraints it already validates against — it checks empty cells for `required` and post-conversion duplicates for `unique`, so it was doing the work without persisting the result — and the route skips the separate constraint write when the type changed. A request combining a rename, a retype and constraint changes is now one locked transaction: no half of it can commit while another fails. The separate constraint write remains for requests that do not change type, which is the only case that still needs it. Deliberately still NOT merging the two service functions. They assert different lock levels (destructive vs schema-only) and only the retype needs the full row scan into memory, so a merged function would force a constraints-only toggle to materialize every row or reintroduce the branching it was meant to remove. Folding the payload in gets atomicity without either cost. * fix(tables): reject a flattened list as an amount; fold constraints into every typed write Two findings from round 11. Multi-select converted to nonsense amounts. `selectValueForConversion` flattens a multi cell to its comma-joined option names, and the parser read that as a formatted number: options 12 and 34 became 12.34, and 100 and 200 became 100200. No real amount puts whitespace after a separator, but a delimited list does — so a separator followed by whitespace is now refused. Every legitimate form still parses, including space-grouped locales. Combined options-or-currency + constraints could still commit partially. Those two writes now carry constraints the same way the retype does, through one shared `applyConstraints` that validates (workflow-output, empty cells for required, supportsUnique and duplicates for unique) and applies them. The separate constraint write now runs only when no typed write does. Three copies of those rules is the drift that produced the original required-check bug, so they live in one place. * fix(tables): validate constraints after the migrations that rewrite cells Self-caught while reviewing my own previous commit, which introduced both. `updateColumnOptions` ran the shared `applyConstraints` BEFORE its cell migrations. Those migrations rewrite stored values — a single<->multi toggle changes the shape, removing an option clears cells — so a `unique` scan read the pre-migration values, passed, and the rewrite could then produce the duplicates the scan was meant to prevent. Moved to after the migrations, which is where `updateColumnType` already had it. The same commit also left the options path running `required`'s empty-cell check twice: once in the shared helper and once in its original inline block, whose comment still described a separate constraint write that no longer runs. Removed the duplicate — one query, one rule, which is the whole point of the shared helper. Also routes the options path through `persistColumns` like the others. * fix(tables): stop inventing amounts from identifiers; fix the copilot retype Adversarial pass over the final state, seven real findings. Two destroyed data. The copilot `update_column` still used the two-transaction pattern the HTTP routes were fixed for: `unique` was never forwarded to the typed write, so a retype+unique committed the conversion and then failed the constraint — the same irrecoverable half. It now rides the typed write, and the separate constraint write only runs when no typed write did. And the parser's three-letter strip removed ANY three letters, not an ISO code: `SKU400` parsed as 400, `ABC1234` as 1234. Converting a column of part numbers to currency rewrote every cell with an invented value — while the comment two lines above claimed a SKU was exactly what it prevented. The rule is now that a letter touching a digit means identifier, not amount; a currency marker is always separated by a space or a symbol. That same change fixed a class the review surfaced: the pinned currencies could not parse their own conventional notation. `R$ 1.234,56`, `1 234,56 kr`, `1234,56 zł`, `CHF 1’234.56` and Indian lakh grouping (`₹12,34,567.89`) all work now — these are what Intl emits, so a paste from a spreadsheet was being rejected. `updateColumnConstraints` was a fourth copy of the constraint rules the shared helper exists to unify, and had already drifted: it hardcoded `type === 'select'` where the helper asks the registry, so a future type declaring `supportsUnique: false` would have been ignored on that path. It now uses the helper. `updateColumnType`'s unchanged-type early return silently discarded every field except the rename. Callers gate on the type changing, but from a read taken before the lock — so a concurrent change could land there with real work pending and answer success. It now throws. Also: `UpdateColumnCurrencyData` was missing `required`, which only compiled because the routes pass it through a spread; a missing column returns 404 instead of a 400 reading "of type undefined"; and the comments describing the old two-transaction architecture are gone. Verified NOT a bug: CSV export of a currency column writes the raw number, so export/import round-trips losslessly. * fix(tables): read the negative and RTL forms Intl actually emits An Intl sweep across 24 locales found two forms the parser rejected, both from an ordinary spreadsheet paste. `Intl` emits U+2212 MINUS SIGN rather than the ASCII hyphen for negatives in several locales, so `−12,50 kr` read as null instead of -12.5. And it wraps RTL-locale output in invisible bidi control marks, so `‏1,234.56 ‏₪` carried characters that are not part of the amount. Both are now normalized away. 24 locales x 6 amounts now round-trip, up from 99/100 when the sweep started — and the test generates them from `Intl` rather than listing them by hand, so a parser change cannot quietly regress a locale nobody remembered to write down. Locales that format with their own numeral systems (Arabic-Indic) are still rejected, and now say so in the docstring. That is a safe failure — null rather than a wrong value — and supporting them is a wider decision than this type, since it would also touch `number`, display, and sorting.
1 parent 7798e83 commit ee0157d

62 files changed

Lines changed: 4142 additions & 791 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 160 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,160 @@
1+
---
2+
name: add-column-type
3+
description: Add a new table column type to Sim — registry entry, icon, storage shape, coercion, and the behavioral hooks the grid and API read. Use when adding a value kind under `apps/sim/lib/table/column-types/`.
4+
argument-hint: <type-name>
5+
---
6+
7+
# Adding a Table Column Type
8+
9+
A column type is **one file** in `apps/sim/lib/table/column-types/` plus a registry entry. Everything that varies per type — label, icon, storage cast, coercion, validation, conversion compatibility, formatting, editor, filter operators — lives on that one object, so no consumer needs editing.
10+
11+
This was not always true: adding `currency` originally took ~40 edits across 32 `switch` arms and 26 UI branches, each of which failed **silently** when missed. The registry exists to make that impossible, so the rule is absolute: **if you find yourself adding a `case 'yourtype':` anywhere outside `column-types/`, the registry is missing a field. Add the field instead.**
12+
13+
## Hard Rule: the compiler tells you what to do
14+
15+
Do **not** hunt for places to edit. Add your type to the `ColumnType` union first and let `tsc` produce the list:
16+
17+
```bash
18+
cd apps/sim && bunx tsc --noEmit -p tsconfig.json
19+
```
20+
21+
You will get two errors, naming `column-types/registry.ts` and `column-types/registry.server.ts`. Register in both.
22+
23+
If your type owns metadata, adding its key to `TYPE_SPECIFIC_COLUMN_KEYS` produces two more legitimate errors — `FOREIGN_METADATA_VERB` in `validation.ts` (a `Record` over those keys) and the key's absence from `ColumnDefinition`. Those are the gate working, not sites to "fix".
24+
25+
Any error beyond those four is a site reading a hardcoded type list that should read the registry — fix that site, don't work around it.
26+
27+
## Directory Structure
28+
29+
```
30+
apps/sim/lib/table/column-types/
31+
├── types.ts # ColumnTypeDefinition — the contract you implement
32+
├── types.server.ts # ColumnTypeServerDefinition — cell migrations only
33+
├── registry.ts # Record<ColumnType, …> ← client-safe, the gate
34+
├── registry.server.ts # Record<ColumnType, …> ← adds migrations (drizzle)
35+
├── index.ts # barrel + accessors (columnTypeOf, columnTypeById, …)
36+
└── {type}.ts # one file per type — what you write
37+
```
38+
39+
## Step 1: Pick the storage shape
40+
41+
Decide what a cell literally holds in `user_table_rows.data` (JSONB). This drives almost everything else:
42+
43+
| Storage | `jsonbCast` | Notes |
44+
|---------|-------------|-------|
45+
| number | `'numeric'` | Filters/sorts compare numerically. `currency` does this. |
46+
| ISO string | `'timestamptz'` | `date` does this. |
47+
| string / bool / object | `null` | Text comparison is correct. |
48+
49+
**Prefer an existing primitive over a new shape.** `currency` stores a plain number and keeps its ISO code as *display metadata* — which is why filtering, sorting, uniqueness, and CSV export all reuse the numeric paths untouched, and why re-denominating a column rewrites zero rows.
50+
51+
## Step 2: Add the icon
52+
53+
Create `packages/emcn/src/icons/type-{name}.tsx`, copying the geometry conventions of its siblings exactly:
54+
55+
```tsx
56+
import type { SVGProps } from 'react'
57+
58+
/**
59+
* Type {name} icon component - {what the glyph is} for {name} columns
60+
* @param props - SVG properties including className, fill, etc.
61+
*/
62+
export function Type{Pascal}(props: SVGProps<SVGSVGElement>) {
63+
return (
64+
<svg
65+
width='24'
66+
height='24'
67+
viewBox='-1.75 -1.5 24 24'
68+
fill='none'
69+
stroke='currentColor'
70+
strokeWidth='1.55'
71+
strokeLinecap='round'
72+
strokeLinejoin='round'
73+
xmlns='http://www.w3.org/2000/svg'
74+
aria-hidden='true'
75+
{...props}
76+
>
77+
<path d='' />
78+
</svg>
79+
)
80+
}
81+
```
82+
83+
- `viewBox='-1.75 -1.5 24 24'` is the **`type-*` family** value, not the set-wide default. Match the family.
84+
- Center the glyph on the viewBox's optical center (**y = 10.5**, **x = 10.25**) — every sibling does, and a few tenths off is visible at `size-[14px]`.
85+
- Export alphabetically **by component name** in `packages/emcn/src/icons/index.ts`.
86+
87+
## Step 3: Write the type file
88+
89+
`apps/sim/lib/table/column-types/{name}.ts`. Copy the closest existing type and change what differs. Every field is required by the interface, so the compiler enumerates them for you — read the TSDoc in `types.ts` rather than guessing.
90+
91+
The three that are easy to get wrong:
92+
93+
- **`coerce`** is the *single* write-path implementation. The server runs it before persisting **and** the grid runs it to fill the optimistic cache. Accept every shape the value legitimately arrives in (paste, CSV, tool write), because rejecting means the cell is nulled.
94+
- **`isCompatibleWith`** gates type conversion and must read the value **exactly as `coerce` will**, or a conversion will pass its check and then null the cell.
95+
- **`ownedMetadata`** lists the `ColumnDefinition` keys your type owns. Anything you add must also be added to `TYPE_SPECIFIC_COLUMN_KEYS` in `types.ts` and given a phrase in `FOREIGN_METADATA_VERB` in `validation.ts` — both are `Record`-typed, so the compiler will tell you.
96+
97+
## Step 4: Register
98+
99+
Add the entry to `COLUMN_TYPE_REGISTRY` in `registry.ts` **and** `COLUMN_TYPE_SERVER_REGISTRY` in `registry.server.ts`.
100+
101+
`COLUMN_TYPES` is declared in `types.ts` (not derived from the registry — the registry is annotated `Record<ColumnType, …>` against it, which is the gate). `constants.ts` re-exports it, so `columnTypeSchema = z.enum(COLUMN_TYPES)` picks your type up with no edit. **Type-specific metadata does not** — see the next step.
102+
103+
## Step 5: Migrations (only if the stored bytes change)
104+
105+
If converting an existing column **to** your type must rewrite cells, add `migrateCellsTo` in `registry.server.ts`; if converting **away** must rewrite them, add `migrateCellsFrom`.
106+
107+
This is load-bearing, not cosmetic: filters and sorts apply `jsonbCast` to whatever is stored, so leaving a non-castable string behind makes **every query on that column fail** — not merely render oddly.
108+
109+
Prefer set-based SQL. When the transform genuinely needs JS (`currency`'s separator disambiguation), compute the values during the compatibility scan and pass them through `resolved`, then apply them in one batched statement.
110+
111+
## Naming Convention
112+
113+
- Type id: lowercase, singular — `currency`, not `Currency` or `currencies`
114+
- File: `column-types/{id}.ts`, export `const {id}ColumnType`
115+
- Icon: `type-{kebab}.tsx`, export `Type{Pascal}`
116+
117+
## Watch out
118+
119+
- **Import cycles.** `column-types/select.ts` imports `select-values.ts`, so `select-values.ts` must **not** import the registry — that closes a cycle and fails at module init. Inside a type's own helper module the string literal is the implementation, not a config leak.
120+
- **The client-safe boundary.** `registry.ts` and everything it imports must stay free of `@sim/db`, `drizzle-orm`, and `next/server` — the tables grid imports it directly. A React icon is fine (it's a component *reference*, never called server-side). Only `registry.server.ts` may touch drizzle.
121+
- **Don't re-export the registry from `@/lib/table`.** 44 server modules import that barrel; routing this through it pulls `@sim/emcn/icons` into all of them. Deep-import `@/lib/table/column-types`.
122+
- **`import.ts`'s `coerceValue` is a SECOND write path and is not opt-in.** Importing into a column of your type always hits it, and its `default` arm silently `String(value)`s — so a missing `case` stores text in a column whose `jsonbCast` is numeric, and then every filter and sort on that column errors in Postgres. Add a `case`, even though the switch compiles without one. (It is deliberately separate from the registry's `coerce`: an import wants an unparseable value to survive as its raw string so the row error can name it.)
123+
- **CSV inference** is an ordered heuristic in `import.ts`, deliberately not registry-driven. A new type is not inferred from a CSV unless you extend `inferColumnType` — usually you should not, since inference cannot supply configuration (an option set, a currency code).
124+
125+
## If your type owns metadata, read this
126+
127+
Registering the *type* is compiler-enforced. Registering its *metadata* is not, and that is where the remaining manual work lives. A key like `precision` has to be added in each of these, none of which will fail to compile if you forget:
128+
129+
| Where | What happens if you forget |
130+
|---|---|
131+
| `lib/table/types.ts` `ColumnDefinition` | (this one DOES fail — the ownership loop indexes it) |
132+
| `column-types/types.ts` `TYPE_SPECIFIC_COLUMN_KEYS` | it is never stripped on conversion, and poisons the target type |
133+
| `lib/api/contracts/tables.ts` — the schema slot in all three column schemas, plus `refineColumnOptions` | zod strips it at the boundary; silently never saved |
134+
| `columns/service.ts` `addTableColumn` param type | callers cannot pass it |
135+
| A metadata-only update path (`updateColumnCurrency` is the model) + a branch in both column routes + the copilot tool | changing it on an existing column is a silent 200 no-op |
136+
| `column-config-sidebar.tsx` | no UI to set it |
137+
| `table-grid.tsx` delete-column undo + `use-table-undo.ts` restore | undo silently resets it to the default |
138+
139+
`normalizeColumn`, `buildConvertedColumn`, and the undo snapshot read `TYPE_SPECIFIC_COLUMN_KEYS` generically, so those three are already zero-edit.
140+
141+
**Known gap:** the metadata-only update path is ~6 near-identical copies (service + 2 routes + copilot). A `metadataUpdate` descriptor on `ColumnTypeServerDefinition` would collapse them; until that exists, copy `currency`'s.
142+
143+
## Checklist Before Finishing
144+
145+
- [ ] Added to the `ColumnType` union in `column-types/types.ts`
146+
- [ ] `column-types/{id}.ts` created, every interface field filled in
147+
- [ ] Registered in **both** `registry.ts` and `registry.server.ts`
148+
- [ ] Icon added, centered on the family's optical center, exported alphabetically
149+
- [ ] `migrateCellsTo` / `migrateCellsFrom` added if the stored bytes change
150+
- [ ] New metadata keys added to `TYPE_SPECIFIC_COLUMN_KEYS` + `FOREIGN_METADATA_VERB`
151+
- [ ] Unit tests for `coerce` / `isCompatibleWith` round-trips, verified to fail without the code
152+
- [ ] Docs row added to `apps/docs/content/docs/en/tables/index.mdx`
153+
154+
## Final Validation (Required)
155+
156+
1. **`cd apps/sim && bunx tsc --noEmit -p tsconfig.json`** — must be clean. If any file *outside* `column-types/` errors, that file has a hardcoded type list; fix it to read the registry.
157+
2. **Grep for leaks**`grep -rnE "(===|!==) '{id}'|case '{id}':" apps/sim --include='*.ts' --include='*.tsx' | grep -v column-types/`. (All three forms: a plain `!==` and a `case` are how half of `currency`'s real branches are written.) Hits are expected; judge each. A hit is fine when it mounts a specific React component or encodes a genuinely one-off behavior (`json`'s mono textarea, `date`'s timezone-aware parsing). A hit is a **leak** when it restates something the registry could answer — an icon, a label, a colour, an operator set, a cast, a coercion. Leaks get a registry field, not a new branch.
158+
3. **Run the suite**`bunx vitest run lib/table 'app/workspace/[workspaceId]/tables' lib/api app/api/table app/api/v1 lib/copilot/tools/server/table`. Existing tests must pass **unchanged**; needing to edit one means you changed behavior for the other types.
159+
4. **`bun run lint:check`, `bun run check:api-validation`, `bun run check:client-boundary`** from the repo root.
160+
5. **Exercise it in the running app** on a table with one column of every type: create, edit inline / in the expanded popover / in the row modal, paste from a spreadsheet, filter, sort, convert to and from other types, export CSV, undo a column delete.

0 commit comments

Comments
 (0)