Repository navigation
feat(driver): cached statement executors, insert lastrowid, and Cloud SQL connector retries - #930
Merged
Merged
Conversation
Expose a public registration surface for external adapters that implement single-call compiled cache-hit executors, and preserve `last_inserted_id` across both generic driver cache-hit fallback paths and the four MySQL adapter compiled executors. - Export `SyncCacheHitExecutor`, `AsyncCacheHitExecutor`, and `cache_hit_hooks_unchanged` from `sqlspec.driver`. - Add `cache_hit_executor`, `cache_hit_state`, `clear_cache_hit_executor`, and `resolve_last_inserted_id` on `CommonDriverAttributesMixin`, and typed `register_cache_hit_executor` methods on `SyncDriverAdapterBase` and `AsyncDriverAdapterBase`. - Preserve `last_inserted_id` on `DMLResult` for cached `INSERT` executions with `affected_rows > 0` in `_execute_cache_hit_with_cursor` and across `pymysql`, `aiomysql`, `asyncmy`, and `mysqlconnector` (`normalize_lastrowid` and `resolve_rowcount` also reject `bool` values). - Override `resolve_last_inserted_id` to return `None` in `PymssqlDriver` and `SybaseDriver` to match `dispatch_execute`. - Document the public cache-hit registration API and `SQLResult.create_cached_select` in reference docs. Closes #927 Closes #928
…r errors Update the Cloud SQL and AlloyDB asyncpg connector wrappers to accept arbitrary `asyncpg.create_pool` positional and keyword arguments, filter connector-managed and `None` options, and retry transient control-plane and transport failures with bounded jittered backoff. - Accept `*args, **kwargs` in `_AsyncpgCloudSqlConnector.__call__` and `_AsyncpgAlloydbConnector.__call__`, stripping connector-owned keys (`ssl`, `direct_tls`, `host`, `port`, `dsn`, credentials, instance selectors) and `None` values while preserving runtime options (`loop`, `connection_class`, `record_class`, `statement_cache_size`, `command_timeout`, `timeout`, `server_settings`). - Forward non-connector options from `AsyncpgConfig.create_connection()` to managed connector wrappers while keeping custom zero-arg `connect` callbacks unchanged. - Add `connector_max_retries`, `connector_retry_backoff_seconds`, and `connector_retry_max_backoff_seconds` to `AsyncpgDriverFeatures` with eager validation and shared default constants (`3`, `0.5`, `5.0`) in `sqlspec/adapters/asyncpg/core.py`. - Add compiled `is_transient_connector_error` and `calculate_connector_backoff_seconds` helpers in `sqlspec/adapters/asyncpg/core.py` using `sys.modules` lookups and cycle-/depth-bounded causal traversal to classify transient vs. permanent errors without eager optional imports. - Regenerate `docs/databases/postgres/asyncpg.md` configuration tables and update driver feature honesty tests. Closes #929
Bump package version from 0.72.2 to 0.73.0 and add the v0.73.0 release section to `docs/changelog.md`.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #930 +/- ##
==========================================
+ Coverage 84.65% 84.70% +0.05%
==========================================
Files 629 629
Lines 142422 142845 +423
Branches 30813 30912 +99
==========================================
+ Hits 120564 120999 +435
+ Misses 15222 15217 -5
+ Partials 6636 6629 -7
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
… retry settings Reduce the new public surface to one method and two protocols, and align the AsyncPG connector retry settings with the existing connection retry names. - Rename `SyncCacheHitExecutor` and `AsyncCacheHitExecutor` to `SyncCachedStatementExecutor` and `AsyncCachedStatementExecutor`, and `register_cache_hit_executor()` to `set_cached_statement_executor()`. - Passing `None` to `set_cached_statement_executor()` removes the executor, replacing `clear_cache_hit_executor()`. - Remove the `cache_hit_executor` and `cache_hit_state` properties and stop exporting `cache_hit_hooks_unchanged` from `sqlspec.driver`. - Leave the current executor and state unchanged when the `owner` check fails. - Rename `connector_max_retries`, `connector_retry_backoff_seconds`, and `connector_retry_max_backoff_seconds` to `connector_retry_count`, `connector_retry_delay`, and `connector_retry_max_delay`.
Retry transient Cloud SQL connector failures in the PyMySQL connection factory, matching the AsyncPG connector settings. - Add `connector_retry_count`, `connector_retry_delay`, and `connector_retry_max_delay` to `PyMysqlDriverFeatures`, validated when the config is built. - Add adapter-local `is_transient_connector_error` and `calculate_connector_backoff_seconds` helpers in `sqlspec/adapters/pymysql/core.py`; MySQL, authentication, and configuration errors are raised on the first attempt. - Write the AsyncPG and PyMySQL retry loops without an unreachable trailing raise. - Replace the AsyncPG pool test connection with a stub that sets no private asyncpg attributes, and cover status shapes, implicit exception context, missing optional modules, and uninitialized connectors. - Regenerate the `PyMysqlDriverFeatures` configuration table.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Let external adapters run cached statements through one compiled function, preserve
last_inserted_idacross cachedINSERTexecution paths, and update the AsyncPG Cloud SQL and AlloyDB connector wrappers to acceptasyncpg.create_poolconnection kwargs and retry transient failures with bounded exponential backoff.Key Changes
Driver (
sqlspec/driver/__init__.py,sqlspec/driver/_common.py,sqlspec/driver/_sync.py,sqlspec/driver/_async.py,sqlspec/core/result/_base.py):set_cached_statement_executor(executor, *, state=None, owner=None)onSyncDriverAdapterBaseandAsyncDriverAdapterBase, and export theSyncCachedStatementExecutorandAsyncCachedStatementExecutorprotocols fromsqlspec.driver. PassingNoneremoves the executor. Whenowneris given and a subclass overrides one of the watched driver methods, the call returnsFalseand leaves the current executor and state unchanged.resolve_last_inserted_id(cursor)onCommonDriverAttributesMixin.resolve_last_inserted_idin_CACHE_HIT_HOOKSand guardcache_hit_hooks_unchanged()against non-type arguments and non-subclasses.last_inserted_idonDMLResultin_execute_cache_hit_with_cursorfor cachedINSERToperations affecting at least one row across both the direct cursor anddispatch_executefallback paths.SQLResult.create_cached_select().MySQL, pymssql & Sybase adapters (
sqlspec/adapters/{pymysql,aiomysql,asyncmy,mysqlconnector}/core.py,sqlspec/adapters/{pymssql,sybase}/driver.py):boolvalues inresolve_rowcount()andnormalize_lastrowid()across all four MySQL adapters.normalize_lastrowid(cursor)intoDMLResultinsideexecute_cache_hit()(andexecute_cache_hit_async()) on successful cachedINSERTqueries withaffected_rows > 0.resolve_last_inserted_id()to returnNoneinPymssqlDriverandSybaseDriverto matchdispatch_execute().AsyncPG Cloud SQL & AlloyDB connectors (
sqlspec/adapters/asyncpg/config.py,sqlspec/adapters/asyncpg/core.py):_AsyncpgCloudSqlConnector.__call__and_AsyncpgAlloydbConnector.__call__to accept*args, **kwargsfromasyncpg.create_pool, stripping connector-managed keys (ssl,direct_tls,host,port,dsn, credentials, instance selectors) andNonevalues while forwarding runtime options such asloop,connection_class,record_class,statement_cache_size,command_timeout,timeout, andserver_settings.AsyncpgConfig.create_connection()to managed connector wrappers while keeping custom zero-argconnectcallbacks unchanged.connector_retry_count(3),connector_retry_delay(0.5), andconnector_retry_max_delay(5.0) toAsyncpgDriverFeatureswith eager validation in_validate_connector_config()and sharedDEFAULT_CONNECTOR_*constants incore.py.is_transient_connector_error()andcalculate_connector_backoff_seconds()helpers usingsys.moduleslookups (no eager optional imports) and cycle-/depth-bounded causal traversal so transient control-plane and transport errors retry with jittered backoff while permanent auth, permission, TLS/certificate, configuration, and PostgreSQL errors fail immediately.PyMySQL Cloud SQL connector (
sqlspec/adapters/pymysql/config.py,sqlspec/adapters/pymysql/core.py):connector_retry_count,connector_retry_delay, andconnector_retry_max_delaysettings, validated when the config is built.is_transient_connector_error()andcalculate_connector_backoff_seconds()helpers; MySQL, authentication, and configuration errors are raised on the first attempt.Documentation (
docs/databases/postgres/asyncpg.md,docs/databases/mysql.md,docs/reference/driver/,docs/reference/core/results.md,docs/changelog.md):AsyncpgDriverFeaturesandPyMysqlDriverFeaturesconfiguration tables.set_cached_statement_executor, the executor protocols,resolve_last_inserted_id, andSQLResult.create_cached_select.v0.73.0changelog section.Release (
pyproject.toml,uv.lock):0.72.2to0.73.0.Tests (
tests/unit/driver/test_query_cache.py,tests/unit/adapters/test_asyncpg/test_cloud_connectors.py,tests/unit/adapters/test_pymysql/test_cloud_sql_connector.py,tests/unit/adapters/test_{pymysql,aiomysql,asyncmy,mysqlconnector}/test_cache_hit_executor.py,tests/integration/adapters/_shared/_driver_type_system.py):None, observability routing, subclass override detection, generic sync/async cachedINSERTlast_inserted_idpaths, andpymysql/aiomysql/asyncmy/mysqlconnectorcache-hit row IDs.asyncpg.create_poolintegration tests for both Cloud SQL and AlloyDB wrappers, reserved kwarg filtering, retry recovery/exhaustion, fail-fast classification matrices, connector retry config validation, and subprocess import isolation.DRIVER_FEATURE_CONSUMED_KEYSforasyncpgandpymysql.Closes #927
Closes #928
Closes #929