Issue with debugpy and post portem debug hook in Python 3.12+

[Andrej730]

Hi! Decided to post it as a separate issue - perhaps it indicate some general problem in debugpy too.

There was a script to setup postmortem debugger that would start debugpy automatically. It worked fine, but in 3.12 stopped wokring. There's some internal error in debugpy - see the script and steps to reproduce it below.

Snippet from @fabioz worked really well for me, but it seems to break in Python 3.12.
Haven't investigated in details yet - any ideas for a workaround?

uv venv --python 3.12
.venv\Scripts\activate
uv pip install debugpy
python test.py
# 2.04s - Error getting exception type.
# Traceback (most recent call last):
#   File ".venv\Lib\site-packages\debugpy\_vendored\pydevd\_pydevd_bundle\pydevd_comm.py", line 1550, in build_exception_info_response
#     stype = frames_list.exc_type.__qualname__
#             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# AttributeError: 'NoneType' object has no attribute '__qualname__'

# test.py
def test():
    for i in range(5):
        print("Hello, world!", i)
        if i == 3:
            raise ValueError("Test exception")

def setup_post_portem():
    import sys
    from types import TracebackType
    from typing import Any, Callable, Type

    def make_debugpy_excepthook() -> Callable[[Type[BaseException], BaseException, TracebackType], Any]:
        """
        Return an excepthook that will initialise debugpy and then call through to
        its exception handler.
        """
        import debugpy

        original_debugpy_excepthook = sys.excepthook

        def debugpy_excepthook(
            type_: Type[BaseException],
            value: BaseException,
            traceback: TracebackType,
        ) -> Any:
            """
            Callback called when an exception is hit and debugpy is enabled.
            """
            if not debugpy.is_client_connected():
                print("Exception thrown. Waiting for debugpy on port 5678.")
                debugpy.listen(5678)
                debugpy.wait_for_client()

            import pydevd
            import threading

            py_db = pydevd.get_global_debugger()
            thread = threading.current_thread()
            additional_info = py_db.set_additional_thread_info(thread)
            additional_info.is_tracing += 1
            try:
                arg = (type_, value, traceback)
                py_db.stop_on_unhandled_exception(py_db, thread, additional_info, arg)
            finally:
                additional_info.is_tracing -= 1
            original_debugpy_excepthook(type_, value, traceback)

        return debugpy_excepthook

    sys.excepthook = make_debugpy_excepthook()

if __name__ == "__main__":
    setup_post_portem()
    test()

Originally posted by @Andrej730 in #723

[rchiodo]

In 3.12 we switched to using sys.monitoring so we don't use sys.excepthook anymore would be my guess. You'd have to hook into it how sys.monitoring does it.

[nshepperd]

This happens because some code (various stuff like contextlib and json encoding) inside stop_on_unhandled_exception triggers a sys.monitoring LINE event on 3.12. Seems like pydevd relies on the built in reentrancy protection in sys.monitoring to prevent it from doing this normally -- monitoring callbacks are normally disabled when entering a monitoring callback. But that doesn't apply when you call into stop_on_unhandled_exception directly, not from a callback. It results in a recursive call stack with do_wait_suspend inside do_wait_suspend which messes up the exception information and it's a big mess.

I managed to devise a reliable fix (draft PR #1996), suspending the sys.monitoring callbacks for the duration of post_mortem(). Needs changes to pydevd to expose the trace disabling flag. Maybe there's a different way to suspend it temporarily, but i couldn't figure it out.