Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion doc/developer/generated/adapter/coord/sequencer.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
source: src/adapter/src/coord/sequencer.rs
revision: 6eeaca032b
revision: d0e1746c64
---

# adapter::coord::sequencer
Expand Down
2 changes: 1 addition & 1 deletion doc/developer/generated/adapter/coord/sequencer/_module.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
source: src/adapter/src/coord/sequencer.rs
revision: 6eeaca032b
revision: d0e1746c64
---

# adapter::coord::sequencer
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
source: src/adapter/src/coord/sequencer/inner/cluster.rs
revision: 6eeaca032b
revision: ac496d5de8
---

# adapter::coord::sequencer::inner::cluster
Expand All @@ -12,7 +12,7 @@ Cluster and replica creation and deletion are applied through the catalog implic
For `CREATE CLUSTER` (both managed and unmanaged variants), `scoped_overrides_create_op` is called with `ClusterEvalContext` and `ReplicaEvalContext` values built from plan data and pre-allocated replica IDs, and the resulting `Op::UpdateScopedSystemParameters` is folded into the same catalog transaction that creates the cluster. This makes the committed diff drive the replica-scoped controller push before `create_replica`, so the new cluster's optimizer and each new replica's render-frozen flags take effect before any dataflow renders, rather than falling back to environment-wide values until the next sync tick.
`CREATE CLUSTER` validates the `HYDRATION SIZE` of an `AUTO SCALING STRATEGY` against the allowed replica sizes for the session role, just as `SIZE` itself is validated. This prevents a typo from failing silently at burst-arm time and prevents a size-restricted role from bursting at a size it may not `CREATE` with.
When `ENABLE_CLUSTER_CONTROLLER` is on and the cluster is user-owned, a shape-changing managed-to-managed `ALTER CLUSTER` (one that changes `SIZE`, `AVAILABILITY ZONES`, or `INTROSPECTION`) reshapes into a durable `reconfiguration` record via `reshape_alter_cluster_managed` instead of going through the legacy 3-stage machine. The record carries the full target config shape, a deadline, and an `on_timeout` action (`ROLLBACK` by default). Non-shape fields (`workload_class`, `schedule`, `auto_scaling_strategy`, etc.) are applied to the realized config immediately. The controller converges the replica set onto the target and cuts the realized config over at hydration. `ClusterStage::AwaitReconfiguration` (carrying `AlterClusterAwaitReconfiguration`) is a foreground wait-shim that polls the durable record until it reaches a terminal status, then reports success or failure depending on whether the realized config reached the target; it is used when `ENABLE_BACKGROUND_ALTER_CLUSTER` is off. The legacy foreground `AlterClusterWaitForHydrated` readiness check additionally requires every pending replica to be online (per `mz_cluster_replica_statuses`) before declaring the reconfiguration ready to cut over, so a cluster hosting only single-replica sources (which are never placed on a pending replica until cut-over) does not declare readiness vacuously the moment the pending replicas are created.
When a reconfiguration is in flight, a subsequent shape-changing `ALTER` folds onto the in-flight target via `fold_reconfiguration_target`: dimensions the `ALTER` set explicitly take the new value, while dimensions left `Unchanged` keep the in-flight target's value (not the realized config's value, which would silently revert the in-flight transition). An `ALTER` back to the realized shape cancels the record. Changing the replication factor while a reconfiguration is in flight is refused with `AdapterError::AlterClusterReplicationFactorWhileReconfiguring`.
When a reconfiguration is in flight, a subsequent shape-changing `ALTER` folds onto the in-flight target via `fold_reconfiguration_target`: dimensions the `ALTER` set explicitly take the new value, while dimensions left `Unchanged` keep the in-flight target's value (not the realized config's value, which would silently revert the in-flight transition). The `deadline` and `on_timeout` are handled separately: when the folding `ALTER` carries an explicit `WAIT` clause, the clause's deadline and action overwrite the in-flight record's (allowing a later `ALTER` to steer the deadline without discarding hydration progress already made on the target); when the folding `ALTER` carries no `WAIT` clause, the in-flight record's `deadline` and `on_timeout` are preserved so that an unrelated shape-changing `ALTER` does not silently reset the user-set deadline. When there is no reconfiguration in flight and no `WAIT` clause, the system-default timeout and implicit `ROLLBACK` action apply. An `ALTER` back to the realized shape cancels the record. Changing the replication factor while a reconfiguration is in flight is refused with `AdapterError::AlterClusterReplicationFactorWhileReconfiguring`.
`ALTER CLUSTER` validates a newly set or changed `AUTO SCALING STRATEGY`'s `HYDRATION SIZE` like `SIZE` itself: it must name a real replica size the session role may use. Only a changed strategy is checked, so an existing policy does not block unrelated `ALTER`s if the size allow-list later shrinks. Additionally, if a reconfiguration is in flight, `HYDRATION SIZE` must differ from the in-flight reconfiguration's target `SIZE`: equality would end the reshape with a no-op burst shape and a stored statement that fails its own re-plan. Unmanaged clusters reject `AUTO SCALING STRATEGY` changes with an error, mirroring how other managed-only options are handled.
Converting a cluster to unmanaged while a hydration burst is in flight is refused with `AdapterError::AlterClusterUnmanagedWhileBursting`. The unmanaged variant carries no burst field, so converting would drop the record without a `Finished` audit event and strand the burst replica as an ordinary unmanaged replica nothing ever tears down.
`validate_reconfiguration_resource_limits` checks that the transient peak credit rate (realized plus target shapes running simultaneously) and per-cluster replica count fit the environment-wide budget before the record is written. `validate_reconfiguration_credit_peak` computes the peak by excluding the reconfiguring cluster's own replicas from the base consumption rate to avoid double-counting an already-materialized overlap.
2 changes: 1 addition & 1 deletion doc/developer/generated/adapter/flags.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
source: src/adapter/src/flags.rs
revision: 6c2b81feaf
revision: 1d07b38a9c
---

# adapter::flags
Expand Down
3 changes: 2 additions & 1 deletion doc/developer/generated/environmentd/http/_module.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
source: src/environmentd/src/http.rs
revision: 59b1f165b2
revision: 53e55ae9df
---

# environmentd::http
Expand All @@ -9,6 +9,7 @@ Implements `environmentd`'s embedded HTTP server using Axum.
Supports multiple authentication modes (`None`, `Password`, `Frontegg`, `Oidc`) via a layered middleware stack, manages session-based and per-request authentication (including WebSocket credential exchange), and routes requests across a rich set of endpoints: SQL execution (REST and WebSocket), Prometheus metrics scraping, memory/heap profiling, catalog and coordinator introspection, webhook ingestion, MCP (AI agent interface), audit event injection, console proxy, cluster replica proxying, and internal deployment management.
Key types include `HttpServer`, `HttpConfig`, `AuthedClient`, `Metrics`, and `InternalRouteConfig`.
`HttpConfig` carries `allowed_origin` (the `AllowOrigin` predicate for the CORS layer), `allowed_origin_list` (the raw `Vec<HeaderValue>` injected as an Axum `Extension` into the MCP router for server-side origin validation against DNS rebinding attacks), `frontegg_oauth_issuer_url` (the optional Frontegg workspace URL advertised as the authorization server in MCP OAuth discovery), `dyncfgs` (the process-wide `Arc<ConfigSet>` threaded into `WebhookState` for per-request dyncfg reads), `mcp_metrics` (a `McpMetrics` handle injected as an Axum `Extension` into the MCP router for Prometheus metrics tracking), `oauth_metadata_metrics` (an `OauthMetadataMetrics` handle for Prometheus metrics on OAuth metadata endpoints), `routes_enabled` (the `HttpRoutesEnabled` value that governs which route groups are active and which `AllowedRoles` each group enforces). Role policy lives per route group via `RouteGroup` values in `HttpRoutesEnabled`, not at the listener level. The MCP CORS layer restricts allowed methods to POST and allowed headers to `Authorization` and `Content-Type`.
When the listener authenticator is `None`, the MCP router applies `x-materialize-user` header authentication, matching the behavior of `base_router` and `ws_router` on unauthenticated listeners.
`WebhookState` carries `dyncfgs: Arc<ConfigSet>` so the webhook handler can read the `WEBHOOK_MAX_REQUEST_SIZE_BYTES` limit from the live config without a coordinator round-trip.
`AuthError` enumerates authentication failure modes; the `OidcFailed(String)` variant carries the sanitized `OidcError` display string, which is forwarded to the client so the console can surface OIDC sign-in failures on the login page.
`group_claim_for` (which resolves the group claim via a coordinator `Command::GetSystemVars` round-trip) is called only when the authenticator is `Frontegg`; for `None`, `Password`, and `OIDC` paths the call is skipped so liveness and readiness probes are not coupled to coordinator health.
Expand Down
2 changes: 1 addition & 1 deletion doc/developer/generated/expr/scalar/func/_module.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
source: src/expr/src/scalar/func.rs
revision: d08f8f74a0
revision: e5dab9cc27
---

# mz-expr::scalar::func
Expand Down
2 changes: 1 addition & 1 deletion doc/developer/generated/mysql-util/_crate.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
source: src/mysql-util/src/lib.rs
revision: e757b4d11b
revision: 1d07b38a9c
---

# mysql-util
Expand Down
4 changes: 2 additions & 2 deletions doc/developer/generated/mysql-util/tunnel.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
---
source: src/mysql-util/src/tunnel.rs
revision: 2a6ac3ab4c
revision: 1d07b38a9c
---

# mysql-util::tunnel

Implements MySQL connection establishment with support for three transport modes via `TunnelConfig`: direct TCP (with optional pre-resolved IPs), SSH tunnel (via `SshTunnelManager`), and AWS PrivateLink.
Defines `Config` (wraps `mysql_async::Opts`, tunnel, SSH and timeout config, and optional AWS config), `MySqlConn` (a `Conn` wrapper that keeps the SSH tunnel handle alive), and `TimeoutConfig` (snapshot lock/execution timeouts, TCP keepalive, and connect timeout).
Defines `Config` (wraps `mysql_async::Opts`, tunnel, SSH and timeout config, and optional AWS config), `MySqlConn` (a `Conn` wrapper that keeps the SSH tunnel handle alive), and `TimeoutConfig` (snapshot lock/execution/wait timeouts, TCP keepalive, and connect timeout).
The `Config::connect` method handles RDS IAM token injection, TLS hostname overriding through tunnels, and a configurable connection timeout.
4 changes: 2 additions & 2 deletions doc/developer/generated/mz-debug/internal_http_dumper.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
---
source: src/mz-debug/src/internal_http_dumper.rs
revision: dc4dcf22d7
revision: 8b438b66ce
---

# mz-debug::internal_http_dumper

Implements `HttpDumpClient` and top-level orchestration functions `dump_emulator_http_resources` and `dump_self_managed_http_resources` for collecting heap profiles, CPU profiles, and Prometheus metrics from running Materialize processes.
`HttpDumpClient` streams HTTP responses directly to disk and handles both HTTPS and HTTP with a fallback, as well as optional Basic authentication.
For self-managed (Kubernetes) environments, the module sets up per-service `kubectl` port-forwards to reach internal and external HTTP endpoints for each `environmentd` and `clusterd` instance; for emulator environments it connects directly to the container IP.
For self-managed (Kubernetes) environments, `dump_self_managed_http_resources` expands each cluster service into its backing pods via `find_service_pods`, then scrapes heap profiles and Prometheus metrics per pod using individual `kubectl port-forward` connections to each pod (not the service). This ensures that every process in a scaled multi-pod replica is reached. A failure on one pod is logged and skipped rather than aborting the dump. CPU profiles are captured per-pod in parallel afterwards. For emulator environments, `dump_emulator_http_resources` connects directly to the container IP.
Port selection logic is governed by `AuthMode`: unauthenticated deployments use the internal HTTP port (6878) for all data, while password-authenticated `environmentd` uses the external port (6877) for heap and CPU profiles.
CPU profiles are captured via a POST to each service's CPU profile endpoint after heap profiling completes; the server temporarily disables memory profiling during capture and restores it afterwards. The helper `dump_cpu_profile_and_verify_memory` probes the profiling mode endpoint to check CPU profiling support and verify memory profiling is active again after the capture.
7 changes: 4 additions & 3 deletions doc/developer/generated/mz-debug/kubectl_port_forwarder.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,11 @@
---
source: src/mz-debug/src/kubectl_port_forwarder.rs
revision: 047f99dfe6
revision: 8b438b66ce
---

# mz-debug::kubectl_port_forwarder

Provides `KubectlPortForwarder` and the resulting `PortForwardConnection` token for establishing `kubectl port-forward` sessions to Kubernetes services.
Provides `KubectlPortForwarder` and the resulting `PortForwardConnection` token for establishing `kubectl port-forward` sessions to Kubernetes resources.
`KubectlPortForwarder` targets either a `PortForwardTarget::Service` or a `PortForwardTarget::Pod`. Forwarding a service lets Kubernetes route to any backing pod (suitable when any replica will do); forwarding a pod reaches a specific process (required for profiling scaled multi-pod replicas).
`spawn_port_forward` launches a `kubectl` subprocess, parses its stdout to detect the assigned local address and port, and returns a `PortForwardConnection` whose drop kills the child process.
Also exposes `find_environmentd_service` and `find_cluster_services`, which query the Kubernetes API to locate the relevant services, and `create_pg_wire_port_forwarder`, which combines these into a ready-to-use forwarder targeting the `sql` port.
Also exposes `find_environmentd_service` and `find_cluster_services`, which query the Kubernetes API to locate the relevant services; `find_service_pods`, which lists the pod names backing a service by its label selector (returns an empty vec for an empty selector, and sorts pod names for deterministic output); and `create_pg_wire_port_forwarder`, which combines these into a ready-to-use forwarder targeting the `sql` port.
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
---
source: src/mz-deploy/src/project/compiler/typecheck/catalog.rs
revision: 74f18a3354
revision: a9e7b191b3
---

# mz-deploy::project::compiler::typecheck::catalog

Catalog-backed runtime typechecking.

Implements `CatalogRuntime`, an in-memory `SessionCatalog` built from `mz-sql` builtins, used to validate objects without a running Materialize container. Each typecheck run creates a fresh `CatalogRuntime`, populates it with the object's dependencies, then discards it, so state does not leak between validations.

When seeding builtin types during `CatalogRuntime` construction, each array type back-patches its element type's `array_id` field to point at itself. This mirrors the adapter's bootstrap logic and is required so that name resolution can follow `element_type.array_id` when resolving spelled array types such as `text[]`. The ordering of `BUILTINS::iter()` — element types before the array types that reference them — ensures the element entry is present before the back-patch runs.
2 changes: 1 addition & 1 deletion doc/developer/generated/ore/pool/_module.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
source: src/ore/src/pool.rs
revision: f2082d0163
revision: 4d6c73f6fa
---

# mz-ore::pool
Expand Down
3 changes: 2 additions & 1 deletion doc/developer/generated/pgrepr/value/_module.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
source: src/pgrepr/src/value.rs
revision: f147b1341a
revision: aef7da9d85
---

# mz-pgrepr::value
Expand All @@ -12,6 +12,7 @@ The main entry points are `Value::from_datum` (converting an `mz_repr::Datum` to
The helper function `values_from_row` converts a full `mz_repr::RowRef` into a `Vec<Option<Value>>` for use by the pgwire protocol layer.

Both text-format and binary-format decoding reject NUL (0x00) characters, matching PostgreSQL behavior. In the text path, `reject_nul(s)` is called once on the raw UTF-8 string before dispatching to any type-specific parser. In the binary path, `Text`, `BpChar`, and `VarChar` values go through `decode_binary_string`, which decodes the binary string and then calls `reject_nul`; `Name` values call `reject_nul` directly after decoding. Both helpers return `NulCharacterError` (from the `error` module) on failure.
Binary-format decoding of `Type::Time` reads the wire value as microseconds since midnight (an `i64`) and rejects any value outside `[0, 86_400_000_000)` with `"time out of range"`, rather than delegating to `NaiveTime::from_sql`, which silently wraps out-of-range values around midnight.

The child modules supply `ToSql`/`FromSql` implementations for types that require non-trivial encoding logic:

Expand Down
2 changes: 1 addition & 1 deletion doc/developer/generated/pgtz/timezone.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
source: src/pgtz/src/timezone.rs
revision: 4267863081
revision: e5dab9cc27
---

# mz-pgtz::timezone
Expand Down
2 changes: 1 addition & 1 deletion doc/developer/generated/proto/chrono.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
source: src/proto/src/chrono.rs
revision: a6d6c38fee
revision: e5dab9cc27
---

# mz-proto::chrono
Expand Down
3 changes: 2 additions & 1 deletion doc/developer/generated/sql/plan/hir.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
---
source: src/sql/src/plan/hir.rs
revision: fc2aaf02e7
revision: 0b5864836a
---

# mz-sql::plan::hir
Expand All @@ -9,5 +9,6 @@ Defines HIR (High-level Intermediate Representation), the plan IR produced by th
`HirRelationExpr` and `HirScalarExpr` mirror MIR but additionally allow correlated subqueries, lateral references, and column references with nonzero level (outer-relation columns).
Key types include `ColumnRef` (a leveled column reference), `JoinKind` (inner/left/right/full), `AggregateExpr`, `WindowExpr`, and `CoercibleScalarExpr` (a scalar that may still need type coercion).
`could_run_expensive_function` detects potentially expensive expressions by checking for unary, binary, variadic, and windowing function calls, `CallTable`/`Reduce` operators, and conservatively returns `true` on `RecursionLimitError`.
`relation_node_count` counts the number of `HirRelationExpr` nodes in an expression tree (including those reachable through scalar subqueries) as a structural size metric for comparing two expressions against each other; scalar nodes themselves are not counted.
`HirScalarExpr` implements `VisitChildren<HirRelationExpr>` to expose its immediate subquery bodies (`Exists`/`Select` arms) without descending further; this is the asymmetric counterpart of `VisitChildren<HirScalarExpr>` on `HirRelationExpr`, which traverses scalars into subqueries at any depth.
HIR is converted to MIR via `HirRelationExpr::lower()`, which is defined in `plan::lowering` as an `impl HirRelationExpr` method.
4 changes: 2 additions & 2 deletions doc/developer/generated/sql/plan/plan_utils.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
---
source: src/sql/src/plan/plan_utils.rs
revision: 844ad57e4b
revision: 42a4392d36
---

# mz-sql::plan::plan_utils

Miscellaneous planning helpers shared across the planner: `maybe_rename_columns` applies user-supplied column aliases to a `RelationDesc`; `JoinSide` and `GroupSizeHints` are small utility types used in join planning and query hints respectively.
Miscellaneous planning helpers shared across the planner: `maybe_rename_columns` applies user-supplied column aliases to a `RelationDesc`, rejecting a list longer than the relation's arity; `maybe_rename_columns_exact` wraps it with an additional check that requires the list length to match exactly when the list is non-empty (unless `unsafe_enable_incomplete_view_column_lists` is set, which is force-enabled during item re-planning to preserve backward compatibility for bootstrap); the private `column_count_mismatch` helper builds the shared error message for both. `JoinSide` and `GroupSizeHints` are small utility types used in join planning and query hints respectively.
Loading
Loading