Skip to content

test: add Go native fuzz tests for schema generation#2792

Merged
tempusfrangit merged 2 commits intofix/dict-list-output-schemafrom
feat/schema-fuzz-tests
Mar 1, 2026
Merged

test: add Go native fuzz tests for schema generation#2792
tempusfrangit merged 2 commits intofix/dict-list-output-schemafrom
feat/schema-fuzz-tests

Conversation

@mfainberg-cf
Copy link
Contributor

Summary

Adds Go native fuzz testing for the schema generation pipeline. Based on the schema gen PR (#2788).

Fuzz Targets

Target Package What it exercises
FuzzResolveSchemaType pkg/schema Arbitrary TypeAnnotation trees through the recursive resolver
FuzzJSONSchema pkg/schema Random SchemaType trees through JSON Schema rendering
FuzzParsePredictor pkg/schema/python Arbitrary bytes as Python source through tree-sitter
FuzzParseTypeAnnotation pkg/schema/python Arbitrary return type strings in predict signatures

Running

# All targets, 30s each (default)
mise run test:fuzz

# Custom duration
FUZZTIME=5m mise run test:fuzz

# Single target
go test ./pkg/schema/ -run='^$' -fuzz=FuzzResolveSchemaType -fuzztime=60s

CI

New fuzz-go job runs on Go changes, 30s per target (~2 min total). Independent of other test jobs — only needs changes.

Approach

Byte encoder/decoder maps arbitrary fuzz corpus bytes to deterministic TypeAnnotation and SchemaType trees, giving the fuzzer structured exploration of the recursive type system rather than just random bytes.

Four fuzz targets exercising the schema pipeline:
- FuzzResolveSchemaType: arbitrary TypeAnnotation trees through the
  recursive resolver
- FuzzJSONSchema: random SchemaType trees through JSON Schema rendering
- FuzzParsePredictor: arbitrary bytes as Python source through the
  tree-sitter parser
- FuzzParseTypeAnnotation: arbitrary return type strings in predict
  signatures

Includes:
- mise task 'test:fuzz' (FUZZTIME=30s per target by default)
- CI job 'fuzz-go' running 30s per target on Go changes
- Byte encoder/decoder for deterministic TypeAnnotation and SchemaType
  tree construction from fuzz corpus
Copy link
Contributor

@markphelps markphelps left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice

@tempusfrangit tempusfrangit marked this pull request as ready for review March 1, 2026 06:39
@tempusfrangit tempusfrangit requested a review from a team as a code owner March 1, 2026 06:39
@tempusfrangit tempusfrangit merged commit 6efe6b7 into fix/dict-list-output-schema Mar 1, 2026
32 checks passed
@tempusfrangit tempusfrangit deleted the feat/schema-fuzz-tests branch March 1, 2026 06:39
@tempusfrangit
Copy link
Member

@markphelps merged this to the bigger refactor of the schema-gen, which we still need to land. i forgot i stacked them, but it’s the right approach.

thanks for looking!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants