Skip to content

Feature/sqlite compiletime#1180

Open
Donach wants to merge 4 commits into
wurstscript:masterfrom
Code-Fixxers:feature/sqlite-compiletime
Open

Feature/sqlite compiletime#1180
Donach wants to merge 4 commits into
wurstscript:masterfrom
Code-Fixxers:feature/sqlite-compiletime

Conversation

@Donach

@Donach Donach commented Apr 3, 2026

Copy link
Copy Markdown
Contributor

This PR adds compiletime support for sqlite operations. Not thoroughly tested, but simple stuff works:

Self-contained Examples:
SQLite.txt
SQLiteHelder.txt

@Frotty

Frotty commented Jun 6, 2026

Copy link
Copy Markdown
Member

Hi, as mentioned, will you do the standard library pendant? (provide api functions) else this isn't publicly usable unless users define @extern natives by themselves?

@Donach

Donach commented Jul 22, 2026

Copy link
Copy Markdown
Contributor Author

Updated this PR with the clean implementation (including handle validation, statement connection ownership tracking, auto-finalizing open statements when closing connections, and explicit interpreter error messages).

Also opened the WurstStdlib2 PR providing the native @extern bindings: wurstscript/WurstStdlib2#450

@Frotty

Frotty commented Jul 22, 2026

Copy link
Copy Markdown
Member

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1742a4fbd7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@Donach
Donach force-pushed the feature/sqlite-compiletime branch 3 times, most recently from d03161c to 5d16740 Compare July 22, 2026 22:23
@Donach
Donach force-pushed the feature/sqlite-compiletime branch from 5d16740 to 6e51eb3 Compare July 22, 2026 22:27
@Frotty

Frotty commented Jul 23, 2026

Copy link
Copy Markdown
Member

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6e51eb3786

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

ResultSet rs = sqliteResultSets.remove(statement.getVal());
if (rs != null) rs.close();
sqliteExecutedStatements.remove(statement.getVal());
stmt.clearParameters();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve SQLite bindings across reset

When a prepared statement with bound parameters is reset and stepped again without rebinding every parameter, this line clears all JDBC bindings. SQLite's reset operation leaves bindings in place, so common reuse patterns like binding a WHERE parameter once, reading a row, calling sqlite_reset(stmt), and stepping again silently query with NULL/no values or fail constraints instead of reusing the previous values. Please reset execution state without clearing bindings; expose a separate clear-bindings operation if needed.

Useful? React with 👍 / 👎.

Reborn Reviewer and others added 2 commits July 23, 2026 23:23
sqlite_reset previously called PreparedStatement.clearParameters(), wiping
all bound parameters. This diverges from SQLite's sqlite3_reset(), which
resets execution state but leaves bindings in place, and broke the common
reuse pattern of binding a parameter once, stepping, resetting, and stepping
again without rebinding.

sqlite_reset now only clears the execution state (open result set and the
executed marker). A separate sqlite_clear_bindings native maps to
sqlite3_clear_bindings() for callers that explicitly want to drop bindings.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Address adversarial review findings on the SQLite compiletime natives:

- sqlite_exec now runs every statement of a ';'-separated script instead of
  silently dropping all but the first (JDBC/xerial only executes the first).
  Splitting respects string literals, quoted identifiers and comments.
- sqlite_open rejects smuggled URI query parameters and explicitly disables
  extension loading, closing a load_extension() -> dlopen native-code vector
  reachable from a malicious compiletime dependency.
- sqlite_close now always closes the connection even if finalizing a
  statement throws, aggregating errors, so the connection can no longer leak.
- (re)binding a parameter invalidates prior execution state, so binding new
  values after a step (without an explicit sqlite_reset) re-executes instead
  of being silently ignored.
- Added sqlite_column_is_null so callers can distinguish SQL NULL from an
  empty string / zero value.
- closeAllSqliteResources no longer resets the handle counter, keeping
  handles monotonic so a stale handle can never alias a new resource.
- Documented the 1-based bind / 0-based column index convention and the
  32-bit truncation of sqlite_column_int.
- Removed dead, cleanup-escaping fallback branches in RunTests.

Tests: multi-statement exec, rebind-without-reset re-execution and NULL
detection compiletime tests, plus unit tests for the SQL statement splitter.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@Donach

Donach commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

Follow-up: binding preservation + hardening pass

Pushed two commits addressing the sqlite_reset review comment and an adversarial review of the whole SQLite native surface.

4bceaef92 — preserve bindings across sqlite_reset

sqlite_reset no longer calls PreparedStatement.clearParameters(). It now mirrors sqlite3_reset() — resets execution state (closes the open ResultSet, clears the executed marker) while leaving bound parameters in place, so the bind once → step → reset → step again reuse pattern works. Added sqlite_clear_bindings (maps to sqlite3_clear_bindings()) for callers that explicitly want to drop bindings.

4ba78432e — hardening pass

  • sqlite_exec ran only the first statement of a ;-separated script (JDBC/xerial silently drops the rest). Now splits and executes every statement, respecting string literals, quoted identifiers, and -- / /* */ comments.
  • sqlite_open extension-loading vector: rejects smuggled URI query params (e.g. ?enable_load_extension=true) and explicitly disables extension loading, closing a load_extension()dlopen native-code path reachable from a malicious compiletime dependency.
  • sqlite_close connection leak: the connection is now always closed even if finalizing a statement throws (errors are aggregated), instead of leaking a connection already removed from the handle map.
  • Rebind-without-reset silent no-op: (re)binding a parameter now invalidates prior execution state, so binding new values after a step re-executes on the next sqlite_step rather than being silently ignored.
  • NULL vs empty/zero: added sqlite_column_is_null (via ResultSet.wasNull()) so callers can distinguish a real SQL NULL.
  • Handle recycling: closeAllSqliteResources no longer resets the handle counter, keeping handles monotonic.
  • Documented the 1-based bind / 0-based column index convention and the 32-bit truncation of sqlite_column_int (inherent to WurstScript's 32-bit int).
  • Removed dead, cleanup-escaping fallback branches in RunTests.

Tests

  • Compiletime tests for reset-preserves-bindings, multi-statement exec, rebind-without-reset re-execution, and NULL detection.
  • Unit tests for the SQL statement splitter (semicolons inside string literals / quoted identifiers, comment stripping, empty-statement skipping).

All SQLite tests pass against sqlite-jdbc:3.46.1.3 on JDK 25.

Follow-up to the second adversarial review. Replace the hand-rolled SQL
statement splitter in sqlite_exec with a direct call to SQLite's native
sqlite3_exec (via the xerial driver's low-level DB handle). The splitter
could not parse trigger BEGIN...END bodies, CASE...END, or the [id] / `id`
identifier-quoting forms, so those scripts were shredded into invalid
fragments; delegating to SQLite's own parser handles all of them. The
splitter and its unit tests are removed; a native test now exercises a
multi-statement script containing a trigger and a ';'-bearing quoted
identifier.

Also from the review:
- sqlite_open now only rejects query parameters on the "file:" URI form,
  so a plain path legitimately containing '?' (e.g. on Linux) is accepted;
  enableLoadExtension(false) remains the defense-in-depth backstop for all
  paths (the extension-loading vector was verified closed either way).
- Documented that (re)binding a parameter closes any open result set.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants