Skip to content

fix(memory): query prompt and seed metadata whose keys are not SQL identifiers - #3125

Open
Chen Yufeiyang (feiiiiii5) wants to merge 1 commit into
microsoft:mainfrom
feiiiiii5:fix/prompt-metadata-non-identifier-keys
Open

Chen Yufeiyang (feiiiiii5) wants to merge 1 commit into
microsoft:mainfrom
feiiiiii5:fix/prompt-metadata-non-identifier-keys

Conversation

@feiiiiii5

@feiiiiii5 Chen Yufeiyang (feiiiiii5) commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Fixes the free-form metadata keys that no backend can filter on.

Description

MemoryInterface documents prompt_metadata and seed metadata as "a free-form dictionary for tagging prompts with custom metadata". Both SQL backends spelled each key into the JSON path and reused it as the bind-parameter name:

json_conditions = " AND ".join(
    [f"JSON_EXTRACT(prompt_metadata, '$.{key}') = :{key}" for key in prompt_metadata]
)
condition = text(json_conditions).bindparams(**dict(prompt_metadata.items()))

Only [A-Za-z0-9_] survives that. Every other character raises before the query runs:

await memory.get_message_pieces_async(prompt_metadata={"attack.id": "v"})
# sqlalchemy.exc.ArgumentError: This text() construct doesn't define a
# bound parameter named 'attack.id'

So the filter cannot be expressed at all for a dotted, hyphenated or spaced key — and a dot was additionally read as a path step, meaning a key that did survive was looking up a nested object. The existing coverage uses only identifier-safe keys (finish_reason, status, incomplete_reason), which is why this went unnoticed.

The same two code paths serve seed metadata (_get_seed_metadata_conditions), so seed filtering has the same problem.

Changes

  • sqlite_memory.py: _get_message_pieces_prompt_metadata_conditions and _get_seed_metadata_conditions now bind the JSON path with the key quoted into it ('$."{key}"') and give each condition positional parameter names.
  • azure_sql_memory.py: the same in _get_metadata_conditions, which backs both prompt and seed metadata there.

This is how #3063 fixed the identical interpolation in the label conditions, so the three paths now agree.

JSON_EXTRACT(prompt_metadata, :pm_path_0) = :pm_value_0   --  pm_path_0 = '$."attack.id"'
JSON_VALUE(prompt_metadata, :md_path_0) = :md_value_0    --  Azure SQL

Binding the path also keeps keys and values out of the statement text entirely.

Tests

tests/unit/memory/memory_interface/test_interface_prompts.py queries four keys holding a dot, hyphen, space and colon, and one query naming three keys at once. Against this branch's parent they fail with the ArgumentError above. The plain-key cases the suite already covered still pass.

No documentation change: MemoryInterface already documents these dictionaries as free-form, and this PR makes that true rather than narrowing it.

Tests and Documentation

pytest tests/unit/memory/memory_interface/ tests/unit/memory/test_sqlite_memory.py \
       tests/unit/memory/test_azure_sql_memory.py -q
# 648 passed, 1 skipped
ruff check   # All checks passed!
ruff format --check   # 3 files already formatted

JupyText not run: no code samples or documentation changed.

Not addressed

A key containing a double quote cannot be expressed in a SQLite or T-SQL JSON path, so after this change it matches nothing rather than raising. That is the same residual #3063 left in the label path; closing it would need a different lookup strategy than JSON paths, which is more than this fix should take on.

🤖 Generated with Claude Code

`prompt_metadata` is documented as a free-form dictionary, but the SQLite
and Azure SQL backends spelled each key into the JSON path and reused it
as the bind-parameter name:

    JSON_EXTRACT(prompt_metadata, '$.{key}') = :{key}

Only `[A-Za-z0-9_]` survives that. Anything else raises before the query
runs, so the filter cannot be expressed at all:

    ArgumentError: This text() construct doesn't define a bound
    parameter named 'attack.id'

`get_message_pieces_async(prompt_metadata={"attack.id": "v"})` against
SQLiteMemory raises that today. A dot was also read as a path step, so a
key that happened to survive would have looked up a nested object.

Both backends now bind the path with the key quoted into it (`'$."{key}"'`)
and give each condition positional parameter names, matching how microsoft#3063
fixed the same interpolation in the label conditions:

    JSON_EXTRACT(prompt_metadata, :pm_path_0) = :pm_value_0

Binding the path also keeps the value out of the statement text entirely.

Seed metadata filtering goes through the same two code paths
(`_get_seed_metadata_conditions`), so it is fixed as well.

Regression tests query four keys holding a dot, hyphen, space and colon,
plus one query naming three keys at once. Against the parent commit they
fail with the ArgumentError above; the plain-key cases the suite already
covered still pass.

Not addressed: a key containing a double quote cannot be expressed in a
SQLite or T-SQL JSON path, so it matches nothing rather than raising.
That is the same residual microsoft#3063 left in the label path.

Validation at this head: tests/unit/memory/memory_interface/,
tests/unit/memory/test_sqlite_memory.py, tests/unit/memory/test_azure_sql_memory.py
-- 648 passed, 1 skipped; ruff check and format clean.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant