a target object could impersonate the metadata proxy and kill the whole run - #269
Merged
Merged
Conversation
…le run
Found in the smoketest fleet on the new machine: 15 of 29 fusil runs (about half
the wall clock) died with
TypeError: '_ShutdownTheme' object is not callable
write_python_code.py:343 in _get_object_methods
_MetaProxy was identified by duck-typing, `getattr(obj, "_fusil_is_meta", False)`.
Any object with a catch-all __getattr__ answers that truthily, and CPython 3.16
ships one -- traceback._ShutdownTheme, the stand-in used when _colorize cannot be
imported during late shutdown:
class _ShutdownTheme:
def __getattr__(self, _): return self
So generation took the metadata branch on it and called _fusil_raw_methods(), which
returns the theme again and is not callable. The TypeError escaped into the MAS and
terminated the fusil PROCESS, not the session -- the fleet supervisor then restarted
the instance from scratch. It happened with --discover-in-target off, where no proxy
can exist at all, so the flag should never have been consulted.
Three more call sites had the same hole: get_arg_number and class_arg_number read
_fusil_arity / _fusil_ctor_arity off whatever answered the flag, and
_fuzz_one_module_object read _fusil_class_name.
The check is now isinstance. That needs the class visible to both the generator and
the arity code, and write_python_code imports arg_numbers, so _MetaProxy moves to
its own module rather than being imported cyclically.
isinstance rather than a duck-type on type(obj) on purpose: fusil injects hostile
METACLASSES deliberately (the metaclass bombs), so moving the lookup from the
instance to its type would still be forgeable. There is a test for each shape.
devdanzin
deleted the
metaproxy-detection-spoofed-by-catchall-getattr
branch
September 3, 2026 04:51
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found while triaging the smoketest fleet on the new machine. 15 of 29 fusil runs — about
half the wall clock — died with the same generator crash:
What happened
_MetaProxywas identified by duck-typing:getattr(obj, "_fusil_is_meta", False). Any objectwith a catch-all
__getattr__answers that truthily — and CPython 3.16 ships one,traceback._ShutdownTheme, the stand-in used when_colorizecannot be imported during lateshutdown:
So generation took the metadata branch on it and called
_fusil_raw_methods()— which returnsthe theme again, and is not callable.
Two things make this worse than a bad session:
TypeErrorescapes into the MAS and terminates the fusil process, so the fleetsupervisor restarts the instance from scratch and the run's progress is lost;
--discover-in-targetoff, where no proxy can exist at all, so the flagshould never have been consulted in the first place.
Three more sites had the same hole
get_arg_numberandclass_arg_numberread_fusil_arity/_fusil_ctor_arityoff whateveranswered the flag, and
_fuzz_one_module_objectread_fusil_class_name. All four now gothrough one
is_meta_proxy().Why isinstance, and why a new module
isinstanceis the only check a target cannot forge. Moving the lookup totype(obj)wouldfix
_ShutdownThemebut not a hostile metaclass with__getattr__— and fusil injectsthose deliberately (the metaclass bombs), so that variant is tested too.
isinstanceneeds the class visible to both the generator and the arity code, andwrite_python_codealready importsarg_numbers— so_MetaProxymoves into its own modulerather than being imported cyclically.
write_python_codere-exports it, so nothing elsechanges.
Tests
tests/python/test_meta_proxy.py: a real proxy is recognised; a catch-all__getattr__cannot impersonate one (asserting first that the old check was truthy on it, so the test
would have caught this); a hostile metaclass cannot either; ordinary objects are not proxies;
and both arity helpers fall through to their normal path instead of reading forged metadata.
1291 tests OK,
ruff checkandruff format --checkboth clean.