sql: rewrite scid string literals to use scid() for index support - #9407
Open
vincenzopalazzo wants to merge 2 commits into
Open
sql: rewrite scid string literals to use scid() for index support#9407vincenzopalazzo wants to merge 2 commits into
vincenzopalazzo wants to merge 2 commits into
Conversation
Short channel IDs were stored as TEXT strings (e.g., "735095x480x1") in
SQLite, which prevented efficient use of indexes on SCID columns. Change
storage to INTEGER (the u64 encoding), using a custom "SCID" column type
so the result-reading code can detect these columns and format them back
as "NNNxNNNxNNN" strings for backward-compatible JSON output.
Add two new SQL functions:
- scid('NNNxNNNxNNN') -> integer: for efficient WHERE clause filtering
- fmt_scid(integer) -> 'NNNxNNNxNNN': for formatting in SQL expressions
Fixes ElementsProject#8941
When users query with WHERE in_channel='735095x480x1', the string
literal is compared directly against the integer SCID column, forcing
SQLite to perform a full table scan even when an index exists.
Automatically rewrite scid string literals (matching NNNxNNNxNNN format)
to use the scid() function before passing the query to SQLite, so
'735095x480x1' becomes scid('735095x480x1'). This allows SQLite to
use indexes on SCID columns transparently.
Queries already using scid() explicitly are detected and left unchanged.
Changelog-Fixed: sql plugin now automatically translates short_channel_id string literals to integers for efficient index usage.
Fixes: ElementsProject#8941
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Confirmed still relevant on master (2026-08-12):
plugins/sql.chas noscid()handling, scid string literals still cannot use indexes.Summary
WHERE in_channel='735095x480x1', the string literal was compared directly against the integer SCID column, forcing SQLite into a full table scan even with an index presentjson_sql()passes user queries directly tosqlite3_prepare_v2()without translating scid string literals to integersNNNxNNNxNNNformat to use thescid()function before query execution, so'735095x480x1'becomesscid('735095x480x1')transparentlyFixes #8941
Changelog-Fixed: sql plugin now automatically translates short_channel_id string literals to integers for efficient index usage.
Test plan
SELECT * FROM forwards WHERE in_channel='735095x480x1'now use indexesscid('...')explicitly are left unchanged