Skip to content

Convert all PUT bodies to require all fields#10675

Draft
david-crespo wants to merge 6 commits into
value-update-pocfrom
complete-put-conversion
Draft

Convert all PUT bodies to require all fields#10675
david-crespo wants to merge 6 commits into
value-update-pocfrom
complete-put-conversion

Conversation

@david-crespo

@david-crespo david-crespo commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Completes #10673, leaving the PoC as-is because it's easier to review that way.

The complete diff against main can be viewed here: main...complete-put-conversion

Here is the diff of the generated TS client against main, which is my preferred dense form for viewing schema diffs. It's really not a big diff!

--- a/2026060800.0.0-f1db6e/Api.ts
+++ b/2026062300.0.0-55f3d5/Api.ts
@@ -312,7 +312,7 @@
 /**
  * Updateable properties of an `AffinityGroup`
  */
-export type AffinityGroupUpdate = { description?: string | null; name?: Name | null };
+export type AffinityGroupUpdate = { description: string; name: Name };
 
 export type BgpMessageHistory = Record<string, unknown>;
 
@@ -643,7 +643,7 @@
 /**
  * Updateable properties of an `AntiAffinityGroup`
  */
-export type AntiAffinityGroupUpdate = { description?: string | null; name?: Name | null };
+export type AntiAffinityGroupUpdate = { description: string; name: Name };
 
 export type AuditLogEntryActor =
   | { kind: "user_builtin"; userBuiltinId: string }
@@ -2148,9 +2148,9 @@
 };
 
 /**
- * Update an external subnet
+ * Updateable properties of an `ExternalSubnet`
  */
-export type ExternalSubnetUpdate = { description?: string | null; name?: Name | null };
+export type ExternalSubnetUpdate = { description: string; name: Name };
 
 /**
  * The `FieldType` identifies the data type of a target or metric field.
@@ -2297,9 +2297,9 @@
 };
 
 /**
- * Updateable identity-related parameters
+ * Updateable properties of a `FloatingIp`
  */
-export type FloatingIpUpdate = { description?: string | null; name?: Name | null };
+export type FloatingIpUpdate = { description: string; name: Name };
 
 /**
  * View of a Group
@@ -2802,16 +2802,16 @@
  * Note that modifying IP addresses for an interface is not yet supported, a new interface must be created instead.
  */
 export type InstanceNetworkInterfaceUpdate = {
-  description?: string | null;
-  name?: Name | null;
+  description: string;
+  name: Name;
   /** Make a secondary interface the instance's primary interface.
 
 If applied to a secondary interface, that interface will become the primary on the next reboot of the instance. Note that this may have implications for routing between instances, as the new primary interface will be on a distinct subnet from the previous primary interface.
 
 Note that this can only be used to select a new primary interface for an instance. Requests to change the primary interface into a secondary will return an error. */
-  primary?: boolean;
+  primary: boolean;
   /** A set of additional networks that this interface may send and receive traffic on */
-  transitIps?: IpNet[];
+  transitIps: IpNet[];
 };
 
 /**
@@ -3098,9 +3098,9 @@
 };
 
 /**
- * Parameters for updating an IP Pool
+ * Updateable properties of an `IpPool`
  */
-export type IpPoolUpdate = { description?: string | null; name?: Name | null };
+export type IpPoolUpdate = { description: string; name: Name };
 
 /**
  * The utilization of IP addresses in a pool.
@@ -3775,7 +3775,7 @@
 /**
  * Updateable properties of a `Project`
  */
-export type ProjectUpdate = { description?: string | null; name?: Name | null };
+export type ProjectUpdate = { description: string; name: Name };
 
 /**
  * View of a Rack
@@ -3955,10 +3955,10 @@
  * Updateable properties of a `RouterRoute`
  */
 export type RouterRouteUpdate = {
-  description?: string | null;
+  description: string;
   /** Selects which traffic this routing rule will apply to. */
   destination: RouteDestination;
-  name?: Name | null;
+  name: Name;
   /** The location that matched packets should be forwarded to. */
   target: RouteTarget;
 };
@@ -4195,15 +4195,15 @@
 };
 
 /**
- * Updateable properties of a Silo's resource limits. If a value is omitted it will not be updated.
+ * Updateable properties of a Silo's resource limits.
  */
 export type SiloQuotasUpdate = {
   /** The amount of virtual CPUs available for running instances in the Silo */
-  cpus?: number | null;
+  cpus: number;
   /** The amount of RAM (in bytes) available for running instances in the Silo */
-  memory?: ByteCount | null;
+  memory: ByteCount;
   /** The amount of storage (in bytes) available for disks or snapshots */
-  storage?: ByteCount | null;
+  storage: ByteCount;
 };
 
 /**
@@ -4634,9 +4634,9 @@
 };
 
 /**
- * Update a subnet pool
+ * Updateable properties of a `SubnetPool`
  */
-export type SubnetPoolUpdate = { description?: string | null; name?: Name | null };
+export type SubnetPoolUpdate = { description: string; name: Name };
 
 /**
  * Utilization of addresses in a subnet pool.
@@ -4696,7 +4696,7 @@
 
 export type SupportBundleUpdate = {
   /** User comment for the support bundle */
-  userComment?: string | null;
+  userComment: string | null;
 };
 
 /**
@@ -4992,7 +4992,7 @@
  */
 export type SystemNetworkingSettingsUpdate = {
   /** Toggle the fleet-wide external jumbo-frames opt-in. */
-  externalJumboFramesOptInEnabled?: boolean;
+  externalJumboFramesOptInEnabled: boolean;
 };
 
 /**
@@ -5439,7 +5439,7 @@
 /**
  * Updated list of firewall rules. Will replace all existing rules.
  */
-export type VpcFirewallRuleUpdateParams = { rules?: VpcFirewallRuleUpdate[] };
+export type VpcFirewallRuleUpdateParams = { rules: VpcFirewallRuleUpdate[] };
 
 /**
  * Collection of a Vpc's firewall rules
@@ -5495,7 +5495,7 @@
 /**
  * Updateable properties of a `VpcRouter`
  */
-export type VpcRouterUpdate = { description?: string | null; name?: Name | null };
+export type VpcRouterUpdate = { description: string; name: Name };
 
 /**
  * A VPC subnet represents a logical grouping for instances that allows network traffic between them, within an IPv4 subnetwork or optionally an IPv6 subnetwork.
@@ -5556,15 +5556,15 @@
  */
 export type VpcSubnetUpdate = {
   /** An optional router, used to direct packets sent from hosts in this subnet to any destination address. */
-  customRouter?: NameOrId | null;
-  description?: string | null;
-  name?: Name | null;
+  customRouter: NameOrId | null;
+  description: string;
+  name: Name;
 };
 
 /**
  * Updateable properties of a `Vpc`
  */
-export type VpcUpdate = { description?: string | null; dnsName?: Name | null; name?: Name | null };
+export type VpcUpdate = { description: string; dnsName: Name; name: Name };
 
 /**
  * Create-time identity-related parameters
@@ -5608,10 +5608,10 @@
  * Parameters to update a webhook configuration.
  */
 export type WebhookReceiverUpdate = {
-  description?: string | null;
+  description: string;
   /** The URL that webhook notification requests should be sent to */
-  endpoint?: string | null;
-  name?: Name | null;
+  endpoint: string;
+  name: Name;
 };
 
 export type WebhookSecretCreate = {
@@ -7491,7 +7491,7 @@
    * Pulled from info.version in the OpenAPI schema. Sent in the
    * `api-version` header on all requests.
    */
-  apiVersion = "2026060800.0.0";
+  apiVersion = "2026062300.0.0";
 
   constructor({ host = "", baseParams = {}, token }: ApiConfig = {}) {
     this.host = host;

…is *Lax

The strict (required name+description) identity update type drops its `Strict`
suffix to become the canonical `IdentityMetadataUpdateParams`; the legacy
all-optional "PATCH-via-PUT" type in omicron_common is renamed to
`IdentityMetadataUpdateParamsLax`.
@david-crespo

Copy link
Copy Markdown
Contributor Author

Discussing with @ahl the possibility of going the other direction, making them all PATCH endpoints and making all fields optional. Looks like this would require a small amount of work in typify/progenitor and the go generator to distinguish between optional and nullable.

🤖 notes on PATCH vs. PUT and required client generator changes

PATCH vs strict PUT: three-state fields and SDK codegen support

Date: 2026-06-25

Question / goal

This is a spin-off from the strict-PUT-bodies work (PR #10673, see
2026-06-25-put-conversion-cost-and-recipe.md and the value-semantics plan).

A colleague suggested we could use PATCH endpoints instead of PUT, possibly
replacing all the PUTs. Under PATCH, every field is optional, and for a field
that is itself nullable (clearable) we'd need to distinguish three states:

  • field absent → leave unchanged
  • field present, null → clear
  • field present, value → set

The proposed Rust encoding is Option<Nullable<T>> (equivalently
Option<Option<T>>) — the classic "double option". This is exactly JSON Merge
Patch
(RFC 7396) semantics applied per field: omit = no-op, null = delete,
value = set. For plain non-nullable fields PATCH just needs Option<T> (absent =
no-op, value = set).

The gating question: can our generated SDK clients actually express the
absent-vs-null distinction on the wire? OpenAPI has no type-level "undefined vs
null" — the distinction lives only in whether the JSON key is present. So whether
the design is viable depends entirely on what each client generator emits for a
property that is optional (not in required) AND nullable.

We investigated the three official SDKs (sibling repos ~/oxide/oxide.ts,
~/oxide/oxide.go, ~/oxide/progenitor) plus typify.

Bottom line

SDK optional+nullable generates omit? explicit null? 3-state today?
oxide.ts field?: T | null works out of the box
Progenitor / oxide.rs (typify) Option<T> + skip_serializing_if ❌ collapses
oxide.go T + omitzero ❌ collapses

Only TypeScript supports the three-state PATCH model today. Progenitor and Go
both collapse absent-and-null
— the client can never put an explicit null on
the wire, so the "clear" state is unreachable. Progenitor matters most because it
also powers the oxide CLI and internal Rust service-to-service clients
in Nexus.

Both gaps are fixable in our own codegen, and both changes are small
(see below). So the PATCH design is viable, but it has a real prerequisite:
land double-option support in typify and a tristate wrapper in oxide.go before
any PATCH endpoint is usable from Rust or Go.

Contrast — the current strict-PUT approach needs only two states (always
present; value or null), i.e. required + nullable, which all three SDKs
already handle: field: T|null (TS), *T no-omit → nil marshals to null
(Go), Option<T> without skip_serializing_ifNone serializes to null
(Rust). Clearing-via-null already works everywhere under strict PUT.

The serde double-option gotcha (important, but doesn't block us)

Native serde does not distinguish missing-key from null when
deserializing Option<Option<T>>: a present null deserializes to the outer
None, same as absent. Getting Some(None) requires a custom
deserialize_with (e.g. serde_with::rust::double_option).

But this is deserialize-only:

  • Serialization works with stock serde. Option<Option<T>> +
    #[serde(default, skip_serializing_if = "Option::is_none")] serializes all
    three states correctly (confirmed empirically): None → key omitted,
    Some(None)null, Some(Some(v)) → value. skip_serializing_if tests
    only the outer option, so Some(None) is not skipped and the inner None
    emits null.
  • oxide.rs only serializes these types (they build request bodies; update
    responses return the full resource view, a different type). So the deserialize
    hazard never applies to the generated client → no custom helper needed there.
  • Nexus server-side does deserialize the PATCH body, so the hand-written
    Nexus types (Option<Nullable<T>>) would each need the double-option
    deserialize_with. That's a per-field annotation, manageable, but it's the one
    place the gotcha bites.

Detailed findings per toolchain

oxide.ts — works today

Two independent codegen decisions in
oxide.ts/oxide-openapi-gen-ts/src/schema/types.ts:

  • optional (types.ts:70): ? added iff name absent from required.
  • nullable (types.ts:79): | null appended iff sub-schema is nullable.

→ optional+nullable yields field?: T | null. Serialization at
oxide-api/src/Api.ts:7670 is JSON.stringify(snakeify(body), dateReplacer);
snakeify (oxide-api/src/util.ts:31-66) only renames keys — it does not
strip null or undefined. So undefined/omit → key absent, null
"key":null, value → value. All three distinguishable. No body-level
null-stripping anywhere (the isNotNull filter exists only for query strings).

Caveat: depends on the spec marking the field optional. In the current Nexus
spec the nullable update fields (InstanceUpdate.boot_disk etc.,
VpcSubnetUpdate.custom_router) are nullable but listed in required, so
they generate field: T | null (no ?) — the strict-PUT two-state shape. To get
three-state, Nexus must leave the field out of required.

Progenitor / typify — collapses today; fix is SMALL

Progenitor delegates all type generation to typify 0.7.0 (Cargo.lock:3013;
the local ~/oxide/typify clone is 0.5.0 and is NOT the right reference).
Progenitor's to_schema.rs:582 faithfully renders nullable as a ["T","null"]
type array; the collapse happens entirely inside typify:

  1. nullable → single Option<T> via convert_option (convert.rs:62,
    :2064).
  2. optional does not add a second Option: has_default
    (typify-impl/src/structs.rs:423, arm at :440:
    (Some(Option(_)), None) => Optional) returns Optional without re-wrapping
    because the type is already an Option.
  3. serde attr emitted (structs.rs:351-355):
    #[serde(default, skip_serializing_if = "::std::option::Option::is_none")].

Result: Option<T>, None→omit, Some(v)→value, no way to emit null.
There is no double_option setting anywhere in typify or progenitor.

Change needed (verdict: SMALL):

  • Add an opt-in bool to TypeSpaceSettings (typify-impl/src/lib.rs:298-310)
    with a fluent with_double_option(...) method (precedent:
    with_struct_builder at lib.rs:438); thread it through progenitor's settings.
  • In struct_property (structs.rs:121-184), in the Optional arm
    (~:157-167): when the setting is on AND the property is optional AND was
    nullable (resolved type is already Option) AND has no explicit default, call
    id_to_option once more to produce Option<Option<T>>.
  • No change to the serde-attr generatorgenerate_serde_attr's
    (Optional, Option(_)) arm already emits the correct
    default + skip_serializing_if for the outer option.
  • Gate precisely on "optional + nullable + no default"; detect nullability from
    the schema (not just "resolved type is Option") to avoid catching refs that
    happen to generate as Option. Non-nullable-optional and nullable-required
    fields both stay Option<T> (unaffected). Behind a default-off flag, existing
    consumers see zero change.

Why small: serialization works with stock serde derive (no custom helper for the
client), the attr codegen already handles the double-option type, and the logic
change is one conditional wrap in one function plus a settings bool. Real work is
tests/snapshots and getting the gate condition precise.

oxide.go — collapses today; fix is SMALL–MEDIUM

Generator decisions in oxide.go/internal/generate/types.go:

  • IsPointer() (:284-313): pointer only when f.Required && v.Nullable
    (:295) or a hardcoded exception (exceptions.go:24). Optional+nullable is
    not a pointer.
  • StructTag() (:247-277): optional+nullable defaults to omitzero/omitempty.

So optional+nullable → value type T with omitzero → {omit, value}, no
null
. (The only nullable path, required+nullable → *T no-omit, gives
{null, value} → no omit. Neither path yields all three.) This already exists in
create bodies: InstanceCreate.{auto_restart_policy,boot_disk,cpu_platform} are
nullable+optional and generate value+omitzero (oxide/types.go:6715,6730,6735)
— two states only.

Change needed (verdict: SMALL–MEDIUM): a Go generic tristate wrapper.

  • Define once a Nullable[T any] (present/null/value) with MarshalJSON
    (null-present → null, else marshal value), UnmarshalJSON (must set
    present=true and detect null), and IsZero() (true when absent), plus
    Absent/Null/Set constructors. Emitted as a static snippet alongside
    existing hand-written helpers.
  • IsPointer()/GoType() (types.go:284-313): add a branch for
    !f.Required && v.Nullable returning Nullable[<inner>].
  • StructTag() (types.go:247-277): same case yields omitzero.
  • Field template already renders {{.Name}} {{.GoType}} {{.StructTag}}
    (templates/type.go.tpl:13), so these two methods are the only seams.
  • Go version is fine: go.mod:3 targets go 1.25; omitzero (Go 1.24) honors a
    custom IsZero() and is already used 24× in the generated code.

Confirmed empirically: the wrapper + omitzero produces absent / null / value
distinctly. Tests to update: types_test.go:184 (TestTypeField_StructTag
"nullable") and :222+ (TestTypeField_IsPointer) — these pinpoint the exact
behavioral surface (good containment signal). Gated on !f.Required,
required+nullable is untouched.

Why not quite trivial: the generic wrapper needs careful Marshal/Unmarshal/IsZero

  • ergonomic constructors; YAML interaction needs checking (fields also carry
    yaml:"...,omitzero", and gopkg.in/yaml.v3 does not honor omitzero like
    encoding/json — tristate YAML round-trip needs its own verification); the oneOf
    omit path (types.go:1136-1140) needs a confirming look; and you'd decide
    whether to retrofit the three existing InstanceCreate fields (a deliberate,
    reviewable change to a shipped surface). Core edit is an afternoon; a focused day
    with tests + YAML is realistic.

Cross-repo rollout note

typify and progenitor are upstream Oxide crates; landing the Rust change means a
PR to each plus a coordinated version bump into omicron/oxide.rs — the
eng:cross-repo-upgrade skill covers this dance. The code is small; the cost is
process/coordination, not engineering.

Decision framing

PATCH is the more RESTfully honest model (these bodies never carried the whole
resource, so "PUT replaces the resource" was always a fiction) and the
single-field-update ergonomics are a real win for wide resources like
instance_update. The downsides:

  • Prerequisite codegen work in typify + oxide.go (both small, but cross-repo).
  • Three-state handling in every Nexus handler + datastore path, plus the
    double-option deserialize_with on server types.
  • PUT→PATCH is a method change (heavier versioning event than same-method body
    tightening), and it reverts the identity-requiredness strict-PUT just added.

Strict PUT, by contrast, sidesteps absent-vs-null entirely (two states, always
present) and is fully supported across all three SDKs right now.

Net: the colleague's proposal is sound and not expensive to unblock, but it is a
genuine fork — worth an explicit decision (or short RFD) rather than a mid-PR
pivot, and the typify/oxide.go codegen should be scoped first since they gate
Rust/Go/CLI usability.

Investigation method (for resuming)

Three general-purpose subagents, one per repo, each answering "can a caller
distinguish omit from explicit null on the wire?" with file:line evidence, then a
second round assessing the concrete change + difficulty. All verdicts above are
evidence-backed against the actual generator source (typify 0.7.0 from the cargo
registry, not the stale local clone).

@davepacheco

Copy link
Copy Markdown
Collaborator

See also: https://rfd.shared.oxide.computer/rfd/0004#_put_and_patch_behavior, #333.

(not at all saying we should do what RFD 4 says, but the discussion of RFCs 6902 vs 7396 still seems relevant)

@david-crespo

david-crespo commented Jun 25, 2026

Copy link
Copy Markdown
Contributor Author

Definitely relevant. My other PR will have some PUT vs. PATCH considerations. Whatever we prefer, we can codify the decision in a short RFD. I am now thinking PATCH makes more sense for most but not all of the current PUTs, and I have some criteria for deciding which is appropriate.

@sudomateo

Copy link
Copy Markdown
Contributor

In Go, the omitempty and omitzero distinction is a bit confusing and, quite frankly, annoying. Luckily Go 1.27 is making omitempty behave more logically with its json/v2 package. However, that still wouldn't give us the ability to express all the desired states.

We would have to switch from omitempty to omitzero so we could add the IsZero() bool method where needed. Then we'd get the ability to express proper types for all combinations of required and nullable.

required nullable Possible States Generated Type
true false value T
true true value, null *T
false false value, absent *T + omitzero
false true value, null, absent Optional[T] + omitzero

david-crespo added a commit to oxidecomputer/console that referenced this pull request Jun 26, 2026
`api-diff` is a lovely tool (if I may say so) for seeing the effect of
OpenAPI schema changes in Omicron on the generated TS client. Until this
PR it can only look at GitHub. This is useful for pointing it at a PR
number. However, while working on
oxidecomputer/omicron#10673 and
oxidecomputer/omicron#10675, which cause large
diffs in the schema, I found I wanted to be able to see changes before I
push them. This PR adds a `--local` flag that lets you run it against
local Omicron commits.

While I think I wrote the original version of this script by hand, it's
gone through a few iterations of full vibe-coding, so while I've skimmed
the code, I am really not especially concerned about the details. The
way this PR works is it adds a concept of `Source`, which is either
GitHub or a local repo, and each has its corresponding methods for
listing schema files and reading the schema contents, etc. Nice work
from the robot.

```ts
/**
 * A source of schema files. The remote source reads from GitHub; the local
 * source reads from the git repo in the current directory. Both expose the
 * same primitives so the diff logic doesn't care where schemas come from.
 */
type Source = {
  /** Resolve a ref (PR number, SHA, branch, jj revset, or undefined for the default) to a commit id */
  resolveCommit: (ref?: string | number) => Promise<string>
  /** List schema filenames under openapi/nexus at a commit, or null if the dir is absent */
  listSchemaNames: (commit: string) => Promise<string[] | null>
  /** Read the filename that nexus-latest.json points to at a commit */
  readLatestPointer: (commit: string) => Promise<string>
  /** Read spec content at a commit (the remote source follows gitstub references) */
  readSpec: (commit: string, specPath: string) => Promise<string>
}
```


This implementation is also `jj`-aware — if the repo in question is a jj
repo it will use jj revs instead of git, which is relevant because when
you have a colocated jj/git repo, by default (without a rev specified)
you probably want to look at `@`, but git's HEAD is at `@-`. If that
doesn't mean anything to you, ignore it.

<img width="788" height="507" alt="image"
src="https://github.com/user-attachments/assets/5c180f0e-5772-46a3-98b1-b8a72ac334c6"
/>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants