Repository navigation
Pre 1.0 audit - #354
Merged
Merged
Pre 1.0 audit#354
Conversation
- Date, Timestamp and OffsetTimestamp parameters were bound with absolute Julian values, but Hyper's binary Bind expects PostgreSQL-epoch values (2000-01-01) and applies the Julian offset itself, so a bound 2024-01-15 read back as 8736-02-22. - Inserter/AsyncInserter::add_numeric ignored the value's own scale, so Numeric::new(12345, 1) into NUMERIC(10, 2) was stored as 123.45. The value is now rescaled to the column's scale; a lossy or overflowing rescale is an Error::Conversion. Both are covered by tests that failed against the real hyperd before the fix. Also fixes the varchar round-trip test and updates add_numeric docs.
…ence value_preview truncated at a byte offset, infer_string_type sliced by byte index, and query_stats logged &sql[..80]; a multi-byte character at the cut point aborted the request. Truncate by character / on a char boundary and only treat ASCII strings as ISO 8601 candidates.
release.yml and npm-build-publish.yml required a check named `clippy` that no longer exists. They now require all 13 current check names to be present and completed. Also corrects stale comments in ci.yml, release-please.yml, the Makefile, .cargo/config.toml, deny.toml and .cargo/audit.toml, fixes the platform list in scripts/assemble-npm.sh, and removes the dead nested hyperdb-api-node CI workflow.
No crate tarball or npm package carried the MIT/Apache-2.0 license text. Add LICENSE-MIT.txt and LICENSE-APACHE.txt to the eight crates (plus NOTICE in hyperdb-api-core for the code adapted from rust-postgres) and list them in the npm package.json `files` arrays.
Corrects stale rustdoc, READMEs, DEVELOPMENT.md files, error-variant names, transport lists and example text across the workspace; documents that query_timeout is recorded but not applied; normalizes #[must_use] placement on the connection builders.
…nterval sign - mcp: query_data/query_file built `DROP TABLE IF EXISTS "<temp>"` with the caller's table_name unescaped; embedded quotes now double. - core: get_table_labels/get_column_labels interpolated schema/table into string literals unescaped; use escape_literal. - core: AuthenticatedGrpcClient::has_table judged existence by Arrow stream byte length, so it was true for every table; count rows instead. - core: Interval Display dropped the sign of negative intervals under 1h. Each has a test that failed before the fix.
… and logs Debug on the OAuth config and token types printed client_secret and raw tokens; the debug-level response logging wrote full OAuth response bodies. Manual Debug impls now redact them and the logs record only byte counts.
A version such as ../victim made the installer remove_dir_all a path outside the install root. Versions are now validated and rejected with the new Error::InvalidVersion.
…ocess drop Drop removed any socket directory whose name started with hyper-, wiping a user's domain_socket_directory such as ~/hyper-data. Only the temp directory the process created itself is removed now.
ConnectionBuilder/AsyncConnectionBuilder::query_timeout was stored and never used; it is now applied as the session's query_timeout (hyperd ignores it as a startup parameter), with zero and gRPC rejected at build(). query_as!/query_scalar! accepted $N arguments and discarded them. QueryAs::new and QueryScalar::new now take &[&dyn ToSqlParam] and bind through fetch_*_as_params. The compile-time dry-run now prepares parameterized SQL instead of failing it.
QueryParameters::json_positional and json_named emitted bare values and a
JSON object, which hyperd rejects with 22023. hyperd requires an array of
typed entries ({"type","value"} plus "name" for named parameters). Both now
emit that format, inferring bool/bigint/numeric/float8/varchar from the JSON
value and rejecting arrays and objects. Doc examples that passed raw value
arrays through from_json_value, or used the keyword :name as a parameter, are
corrected.
hyperd rejects a single-column OR/IN/ANY/CASE filter that mixes a bound parameter with other values, at prepare time (XX000 "too few registered parameters"). Document the failing shapes, the affected APIs and the `IN (SELECT unnest(ARRAY[$1, $2]))` rewrite on Connection::query_params, the async and command_params variants, query_as!, and both READMEs.
…public API hyperdb_api::grpc no longer re-exports GrpcClient, GrpcClientSync or GrpcError (raw core clients carrying tonic/prost types); it exports GrpcColumnInfo instead. The escape hatches that returned hyperdb-api-core types are crate-private or removed: Connection::tcp_client, AsyncConnection::async_tcp_client and from_async_client, HyperProcess::connection_endpoint, ColumnDefinition::to_types_column_definition and the From<core ColumnDefinition> impl. Replacements: HyperProcess::connection_endpoint_string and AsyncConnection::copy_in_hyperbinary, which the Node inserter now uses instead of a hand-written COPY loop. The async pool wraps deadpool: Pool and PooledConnection are hyperdb-api types, ConnectionManager is crate-private, Pool::get returns hyperdb_api::Result (Timeout / Connection / underlying error) and Pool::status returns PoolStatus. BREAKING CHANGE: see hyperdb-api/CHANGELOG.md [Unreleased].
Follow-up to 48ea4e3. - grpc::TransferMode is now an owned enum (Sync, Async, Adaptive) rather than the prost type; GrpcColumnInfo fields are no longer public. - core: a cancelled COPY write no longer wedges the async connection. A write-in-flight flag marks the connection unhealthy after an interrupted write, no CopyFail is queued behind a partial frame, finish/cancel_copy refuse to run on it, and the post-cancel drain is bounded. - copy_in_hyperbinary requires exactly one HyperBinary header (InvalidOperation otherwise), uses the borrowed writer, and documents that it is not cancel-safe. - pool: a closed pool is Error::InvalidOperation; docs state that recycle failures are discarded and replaced; PoolStatus docs corrected. - HyperProcess absolutizes a custom domain socket directory so connection_endpoint_string is always routable. - Drop remaining deadpool/core mentions from user-facing docs and examples. Tests added: core cancel-safety unit tests, copy_in_hyperbinary header, server-error and gRPC cases, pool create-timeout, relative socket dir.
The core client's Io errors map to hyperdb_api::Error::Connection, and Error::Io is for direct SDK-boundary I/O (as error.rs already says), but about twenty `# Errors` sections promised Error::Io for network failures. Also correct DEVELOPMENT-client.md, which said pooling is not exposed.
SqlType, its struct variants (Numeric, Varchar, Char), the struct variants of hyperdb_api::Error and ColumnErrorKind::TypeMismatch are #[non_exhaustive], so a minor release can add a type or a field. Callers match with a wildcard, end field patterns with `..`, and construct via the existing constructors. Add SqlType::try_numeric. The Arrow decimal mapping built SqlType::Numeric by struct expression, which could yield an invalid NUMERIC (Decimal256 reaches precision 76); it now falls back to Text outside 1..=38.
…es work SqlIdentifier, format_table_name, TableDefinition::qualified_name (and the drop/staging SQL built on them) and derive(Table)'s CREATE_SQL left lowercase names bare, so tables or columns named order/user/table broke every COPY. Identifiers are now always double-quoted; generated SQL text changes but the resolved identifiers do not. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The rowset holds the connection lock while streaming; dropping its statement first closes it on the same connection and deadlocks. Return Rowset<'s> / AsyncRowset<'s> so the borrow checker rejects it, with a compile_fail doctest. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…_types Bump arrow and parquet 59 -> 60 (MSRV 1.88 verified) before 1.0 freezes the public API, re-export the three crates whose types appear in it, and state the policy: a major bump of a re-exported crate is a hyperdb-api major. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…ncel sender Replace the unused `client::tls` module (pub-field TlsConfig, async-only rustls_impl, never called) with the building blocks for PG-wire TLS on TCP: - TlsMode with the libpq sslmode names (Disable default, Prefer, Require, VerifyCa, VerifyFull), FromStr/Display, and ParseTlsModeError. - TlsConfig with private fields and a consuming builder (root_cert, client_cert, server_name); From<TlsMode>. - Crate-private TlsConnector: chain-only and signature-only verifiers, webpki fallback roots for VerifyFull, mTLS, IPv6 host handling. - negotiate_sync/negotiate_async: SSLRequest, one-byte reply read off the raw socket, Prefer falls back only on 'N'. The sync handshake is driven read_tls/write_tls at a time so the timeout is a deadline for the whole negotiation (complete_io keeps reading while a peer drips bytes). - send_cancel_sync/send_cancel_async: cancel over TLS, never plaintext, close_notify and a bounded drain. - frontend::ssl_request; client::Error::Tls, with from_io classifying wrapped rustls errors as Tls; mapped to hyperdb_api::Error::Tls. - Config::with_tls/tls; Config::from_str now returns client::Error and rejects the libpq ssl* keys. - hyperdb_api re-exports TlsConfig, TlsMode, ParseTlsModeError. Fake-server unit tests cover every mode, fallback, mTLS, timeouts and both cancel senders. The old tests/tls_tests.rs is removed. Clients and builders are wired up in the next change. BREAKING CHANGE: client::tls API replaced (TlsMode::VerifyCA renamed to VerifyCa, TlsConfig fields private, rustls_impl removed); client::Error gains a Tls variant; Config's FromStr error type is now client::Error.
…cel over TLS ConnectionBuilder::tls and AsyncConnectionBuilder::tls negotiate TLS with libpq sslmode semantics (Disable default, Prefer, Require, VerifyCa, VerifyFull; root cert, client cert, server-name override). Prefer falls back to plaintext only when the server declines TLS. A cancel for a TLS session goes over TLS. Over UDS / named pipes Prefer is plaintext and requiring modes fail with FeatureNotSupported; gRPC rejects tls(). Connection::is_tls and AsyncConnection::is_tls report the outcome. rustls session resumption is disabled: hyperd sets verify_peer without an OpenSSL session ID context and aborts every resumed handshake, which the cancel connection hit. BREAKING CHANGE: SyncStream::try_clone is removed and SyncStream / AsyncStream gain a Tls variant. Integration tests run against a real hyperd with ssl_force, using rcgen certificates: Require round trips (sync/async), VerifyCa/VerifyFull checks, Prefer fallback and no-downgrade, mTLS accept/reject, cancel, gRPC and IPC rejection.
PoolConfig and SyncPoolConfig gain a tls field and tls() builder, applied to every connection the pool opens. Both pools now open connections through AsyncConnectionBuilder / ConnectionBuilder unconditionally, so the async pool also picks up the builder's endpoint parsing (a malformed port is a Config error), IPC routing for authenticated pools, and its gRPC create-mode check. BREAKING CHANGE: PoolConfig and SyncPoolConfig have a new public field, so struct-literal construction no longer compiles.
…erProcess
Expose the Rust TLS support to the Node bindings:
- ConnectionBuilder.tls({ mode, rootCert, clientCert, clientKey,
serverName }) with libpq sslmode spellings, and Connection.isTls.
- ConnectionPool's `tls` option; only a TLS pool goes through
ConnectionBuilder, so a plain pool keeps Connection.connect's behavior.
- new HyperProcess(hyperPath, { transport, parameters }) to start a TCP
hyperd with ssl_key / ssl_certificate.
__test__/tls.mjs runs against an ssl_force hyperd with openssl-generated
certificates and is part of `npm test`; it fails on CI without openssl.
Both parsed every endpoint as host:port, so a Unix socket path failed with "failed to lookup address information". They now delegate to AsyncConnectionBuilder, like their sync counterparts: a malformed port is Error::Config instead of a fallback to 7483, and without_database also reaches gRPC. The now-unused connect_tcp_with_auth is removed from both transports. New test string_constructors_connect_over_a_unix_socket fails on the old code and passes now.
…nectable `transport: 'ipc'` was a no-op and the docs called IPC the default; HyperProcess actually defaults to TCP. 'tcp' and 'ipc' now both set the mode and anything else is rejected. `hyper.endpoint` uses connection_endpoint_string(), so over IPC it is the socket path every connect call accepts rather than the listen descriptor `<dir>/domain/<name>`. tls.mjs gains an IPC case: withoutDatabase and `prefer` connect in plaintext, `require` is rejected. README documents TLS for ConnectionBuilder, the pool and HyperProcess options.
README files describe the libpq sslmode support, the hyperdb-api README gains a TLS section, and DEVELOPMENT-client.md's TLS internals and tests match the implementation. The hyperdb-mcp attach comment records where a future TCP attach would take its TlsConfig.
- Core CHANGELOG lists the breaking reshape of the old client::tls surface, the new client::Error::Tls variant and is_tcp() on TLS streams. - `allow` is documented as unsupported, and parsing it hints at `prefer`. - The connection-string TLS-key error names Config::with_tls. - The builders' tls() docs say the TLS handshake is bounded by login_timeout, not the TCP connect. - Client/AsyncClient Debug redacts the cancel secret_key. - ConnectionPool treats `tls: null` as no TLS; it used to fail every acquire() in the napi conversion (red-checked).
A link-local host such as `[fe80::1%1]` connects in plaintext, but every TLS mode failed before connecting with "invalid TLS server name", because rustls cannot parse the `%zone` suffix. The zone names the local interface, not the server, and an IP SAN cannot hold one, so the derived server name is now the bare address: prefer/require/verify-ca connect, and verify-full checks the IP SAN `fe80::1`. An explicit server_name and a non-IPv6 host are unchanged. Both new unit tests fail with the zone kept and pass with it dropped.
… SQL tool query_data, query_file and export ran any statement, so `query_data(sql: "DROP TABLE persistent.public.t")` dropped a persistent table even under --read-only. They now refuse non-read-only SQL with SQL_ERROR, as query does; query_data/query_file also check the SQL after the table-name substitution. The classifier moves to sql_classify.rs and tokenizes like Hyper's lexer (non-nesting block comments, E-strings, string continuation, dollar-quote tags, Unicode whitespace). WITH and EXPLAIN take the kind of the statement they run, so `WITH ... DELETE` and `EXPLAIN (ANALYZE) DELETE` are no longer read-only; SELECT ... INTO is a write; more than one statement, unbalanced or over-deep parentheses are refused. export checks its SQL, has Hyper parse it alone, and puts it on its own lines inside COPY (...) TO, so it can neither pick its own target nor be broken by a trailing -- comment.
Add the building blocks that replace the unauthenticated loopback health port: a control endpoint only the current user can open (Unix socket 0600 / Windows named pipe with a user-SID DACL and first-instance flag) and a single-instance daemon lock. Not wired into the daemon yet.
…pipe Replace the unauthenticated loopback TCP health port with the per-user control endpoint, add a single-instance daemon lock, and require the state directory to be owned by the user and not group/other writable. Remove --port, HYPERDB_DAEMON_PORT, the port scan and the doctor live-from-scan path. Old clients against a new daemon fall back to local mode.
A client that finds a legacy-shaped discovery record verifies the daemon (identified PING, STATUS pid equals the record pid), stops it, and waits for its port to go quiet before spawning. A mismatched or dead record is removed without sending anything further; an open but unresponsive port keeps the record and the client uses local mode.
STOP now wakes the daemon immediately, and takeover and `daemon stop` wait for the old daemon to release its lock before continuing. A takeover that times out falls back to local mode instead of reusing the dying daemon. Daemon failures are now logged.
…dows lock The client now verifies the server behind the control socket or pipe belongs to the current user, the Windows daemon lock uses LockFileEx instead of an exclusive open, and the daemon checks the state directory before creating its log. Doctor and the CLI no longer create the lock file, and daemon stop names a recorded pre-1.0 daemon.
… deps Remove the linux-arm64-gnu and linux-x64-musl npm package stubs, which no workflow builds or publishes, and their references in the CI version guard, the artifact copier and the loader. Drop the unused root workspace dependencies parking_lot and regex.
Generate THIRD-PARTY-LICENSES.txt for the hyperdb-mcp binary and the hyperdb-api-node addon with cargo-about at publish time, include it in each platform package, and add a CI job that fails when a dependency uses a license outside the accepted list.
Tool signatures now read `tool(required, optional?=default) -> result` with a legend up front, and prose becomes terse fragments. The document shrinks from 23.9 KB to 21.5 KB while adding facts verified against the tool parameter structs: chart size clamps and format inference, export format_options and sql/table precedence, attach on_missing, copy_query temp_attach, save_query duplicate rejection, the watch_directory .ready sentinel protocol, and the kv_get / kv_pop / kv_set_many response shapes.
Windows clippy and test failed on items used only under #[cfg(unix)]: an AtomicUsize import, run_discovery_compatibility_child, scripted_daemon_noting_stop, wait_for_flag, and two `let mut targets` that only the unix symlink block mutates. Gate the helpers and rebind the target lists under cfg(unix) instead of mutating them.
…reset the connection A TLS 1.3 server sends session tickets the test client never reads. On Windows, dropping the socket with that data unread sends a reset (os error 10053), which discards the payload the server has not read yet. Send close_notify, shut down the write side and drain until the server hangs up.
…ng or panicking on Postgres constructs Hyper's builder inherited sea-query's MySQL-style backslash escaping for every path that did not go through the delegated prepare_value, so a Constant string like x\' OR 1=1 -- matched every row. Escaping, string and byte literals now follow Postgres rules (E'..', '\x..', NUL as \000). Also emit DISTINCT ON and NULLS FIRST/LAST, render PgOperator, PgFunc, TABLESAMPLE and AsEnum instead of panicking or dropping them, use COALESCE, and convert Value::Enum to a plain string. Fixes a panic on non-ASCII Char. Adds integration tests against the pinned hyperd. Documents the clauses that still render through the Postgres builder.
…connections, and avoid inserter drop hangs A late cancel request could land on the next query and fail it with 57014; the cancel now waits for the server to close the cancel connection. Dropping a MappedInserter while a result stream was live hung the thread. Abandoned streams drain up to a bounded ceiling before cancelling.
…p scratch tables Create the merge staging guard before the staging load, skip cleanup when a CTAS fails with 42P07 so another session's table is not dropped, keep the original error code across the rename swap, parse modes explicitly, and run catalog bootstrap inside catch_unwind.
…proc-macro server Its shared, lazily-expanding registry can be non-empty yet lack the struct under the cursor, producing false 'table not registered' diagnostics.
…talog bootstrap Clients that predate the stateless 2026-07-28 revision rejected the server. Also make the catalog-ready flag an AtomicBool so a panic cannot disable bootstrap, return an error instead of panicking in IngestMode::parse, and correct the CHANGELOG.
…laude Code falls back to initialize
…ata, write-type and API-summary docs
…ions and document per-transport builder settings query_timeout is already enforced via SET query_timeout (and rejected on gRPC). application_name was applied on TCP only; it is now sent on UDS/pipe too. login_timeout stays TCP-only and gRPC ignores user/password/application_name; the builder docs now say so.
…ead of buffering the whole COPY
…ng into integer columns
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Pre-1.0.0 audit: correctness, security and public-API fixes across the workspace, made while changes are still cheap (after 1.0.0 a
feat!:means 2.0.0). Several commits are breaking and are marked!.Breaking changes
grpcno longer re-exportsGrpcClient/GrpcClientSync/GrpcError;grpc::TransferModeis an owned enum).SqlTypeandErrorstruct variants arenon_exhaustive.hyperdb-api, core, derive).PreparedStatement::queryrowset is tied to the statement borrow.arrow,chronoandgeo_typesre-exported.tls(); cancel works over TLS. Also exposed in the Node bindings.exportandchartrefuse to overwrite by default and never replace a file they did not write.Fixes
has_table, gRPC JSON parameter format,copy_database(nowCREATE DATABASE … FROM), BYTES/TIME/INTERVAL rendering in MCP query results, Node integer columns reject NaN and out-of-range numbers instead of saturating.Debugand logs; path-likehyperdrelease versions are rejected; a caller-supplied socket directory is no longer deleted onHyperProcessdrop; the control endpoint is authenticated.query_timeoutis applied;application_nameis sent on Unix-socket and named-pipe connections;AsyncInserter::flushnow writes to the socket instead of holding the whole COPY in memory.server/discovergets method-not-found, so Claude Code falls back toinitialize.Other
AsyncInserter,NoDiscover,create_table_as,copy_database, gRPC close, Windows daemon/lock cases.CHANGELOG.md[Unreleased]sections updated.Known limits
login_timeoutis still honored on TCP only. Wiring it for Unix-socket and named-pipe connections needs a core change.AsyncInsertertests do not detect the flush memory-growth bug, which is why that fix rests on comparison with the sync path.Verification
cargo fmt, workspaceclippy --all-targets --all-features -D warnings,cargo doc -D warnings, andcargo testforhyperdb-apiandhyperdb-mcpwere run on macOS. Windows was exercised through CI.