Segmentation fault with CPython 3.13 (indygreg build) on MacOS 14.6.1

[Alexis-D]

Original context: spotDL/spotify-downloader#2240

Repro: using rye:

1. Create repro project:

$ rye init --virtual repro && cd repro
success: Initialized virtual project in /private/var/folders/9t/vvzpk1ws65s3hxp59zl1twv80000gn/T/tmp.goCv86v012/repro
  Run `rye sync` to get started

2. Use CPython 3.13:

$ rye fetch cpython@3.13 && rye pin 3.13
pinned 3.13.0 in /private/var/folders/9t/vvzpk1ws65s3hxp59zl1twv80000gn/T/tmp.goCv86v012/repro/.python-version

3. Confirm python version:

$ python --version
Initializing new virtualenv in /private/var/folders/9t/vvzpk1ws65s3hxp59zl1twv80000gn/T/tmp.goCv86v012/repro/.venv
Python version: cpython@3.13.0
Python 3.13.0

4. Add pydantic_core:

$ rye add pydantic_core
Added pydantic-core>=2.27.0 as regular dependency
Reusing already existing virtualenv
Generating production lockfile: /private/var/folders/9t/vvzpk1ws65s3hxp59zl1twv80000gn/T/tmp.goCv86v012/repro/requirements.lock
Generating dev lockfile: /private/var/folders/9t/vvzpk1ws65s3hxp59zl1twv80000gn/T/tmp.goCv86v012/repro/requirements-dev.lock
Installing dependencies
Resolved 2 packages in 3ms
Installed 2 packages in 3ms
 + pydantic-core==2.27.0
 + typing-extensions==4.12.2
Done!

5. Trigger segfault:

$ python -X dev
Python 3.13.0 experimental free-threading build (main, Oct 16 2024, 08:24:33) [Clang 18.1.8 ] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pydantic_core
Fatal Python error: Segmentation fault

Current thread 0x00000001f83e8f40 (most recent call first):
  File "<frozen importlib._bootstrap>", line 488 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1316 in create_module
  File "<frozen importlib._bootstrap>", line 813 in module_from_spec
  File "<frozen importlib._bootstrap>", line 921 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1331 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1360 in _find_and_load
  File "/private/var/folders/9t/vvzpk1ws65s3hxp59zl1twv80000gn/T/tmp.goCv86v012/repro/.venv/lib/python3.13t/site-packages/pydantic_core/__init__.py", line 6 in <module>
  File "<frozen importlib._bootstrap>", line 488 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 1022 in exec_module
  File "<frozen importlib._bootstrap>", line 935 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1331 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1360 in _find_and_load
  File "<python-input-0>", line 1 in <module>
  File "/Users/alexis/.rye/py/cpython@3.13.0/lib/python3.13t/code.py", line 92 in runcode
  File "/Users/alexis/.rye/py/cpython@3.13.0/lib/python3.13t/_pyrepl/console.py", line 205 in runsource
  File "/Users/alexis/.rye/py/cpython@3.13.0/lib/python3.13t/code.py", line 313 in push
  File "/Users/alexis/.rye/py/cpython@3.13.0/lib/python3.13t/_pyrepl/simple_interact.py", line 160 in run_multiline_interactive_console
  File "/Users/alexis/.rye/py/cpython@3.13.0/lib/python3.13t/_pyrepl/main.py", line 59 in interactive_console
  File "/Users/alexis/.rye/py/cpython@3.13.0/lib/python3.13t/_pyrepl/__main__.py", line 6 in <module>
  File "<frozen runpy>", line 88 in _run_code
  File "<frozen runpy>", line 198 in _run_module_as_main
Segmentation fault: 11

Notes

• This works fine with the Homebrew CPython 3.13.0 build
• Enabling/disabling GIL doesn't seem to change anything
• rye uses those CPython builds: https://github.com/indygreg/python-build-standalone
• I cannot say whether the issue is with CPython/pydantic_core/indigreg's builds. So it is possible this ticket should be redirected to one of those other projects, I am not just to sure where to debug from there.

[Alexis-D]

$ lldb -O 'settings set target.process.stop-on-exec false' -- python -c 'import pydantic_core'
(lldb) settings set target.process.stop-on-exec false
(lldb) target create "python"
Current executable set to '/Users/alexis/.rye/shims/python' (arm64).
(lldb) settings set -- target.run-args  "-c" "import pydantic_core"
(lldb) run
Process 97527 launched: '/Users/alexis/.rye/shims/python' (arm64)
Process 97527 stopped
* thread #2, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
    frame #0: 0x00000001905c7610 libsystem_platform.dylib`_platform_strcmp + 64
libsystem_platform.dylib`:
->  0x1905c7610 <+64>: ldr    q0, [x0], #0x10
    0x1905c7614 <+68>: ldr    q1, [x1], #0x10
    0x1905c7618 <+72>: cmeq.16b v1, v0, v1
    0x1905c761c <+76>: and.16b v0, v0, v1
Target 0: (python3.13t) stopped.
(lldb) bt
* thread #2, queue = 'com.apple.main-thread', stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
  * frame #0: 0x00000001905c7610 libsystem_platform.dylib`_platform_strcmp + 64
    frame #1: 0x00000001018e24cc libpython3.13t.dylib`_PyModule_CreateInitialized + 640
    frame #2: 0x0000000100b99cc0 _pydantic_core.cpython-313t-darwin.so`___lldb_unnamed_symbol3970 + 40
    frame #3: 0x0000000100a80bfc _pydantic_core.cpython-313t-darwin.so`PyInit__pydantic_core + 432
    frame #4: 0x00000001017fed3c libpython3.13t.dylib`_PyImport_RunModInitFunc + 84
    frame #5: 0x00000001017fe9dc libpython3.13t.dylib`import_run_extension + 92
    frame #6: 0x0000000102070a18 libpython3.13t.dylib`_imp_create_dynamic_impl + 300
    frame #7: 0x00000001019d905c libpython3.13t.dylib`cfunction_vectorcall_FASTCALL.llvm.10217531742623833864 + 92
    frame #8: 0x000000010176e8c4 libpython3.13t.dylib`_PyEval_EvalFrameDefault + 78772
    frame #9: 0x00000001019bb53c libpython3.13t.dylib`_PyFunction_Vectorcall + 528
    frame #10: 0x00000001016b0864 libpython3.13t.dylib`_PyObject_VectorcallTstate.llvm.16501924132349618227 + 80
    frame #11: 0x00000001017190b8 libpython3.13t.dylib`object_vacall.llvm.16501924132349618227 + 240
    frame #12: 0x0000000101718ec8 libpython3.13t.dylib`PyObject_CallMethodObjArgs + 108
    frame #13: 0x00000001016dafc8 libpython3.13t.dylib`PyImport_ImportModuleLevelObject + 2952
    frame #14: 0x00000001017817a8 libpython3.13t.dylib`_PyEval_EvalFrameDefault + 156312
    frame #15: 0x00000001017e4edc libpython3.13t.dylib`_PyEval_Vector + 484
    frame #16: 0x00000001019fe5a4 libpython3.13t.dylib`builtin_exec + 492
    frame #17: 0x00000001019d9148 libpython3.13t.dylib`cfunction_vectorcall_FASTCALL_KEYWORDS.llvm.10217531742623833864 + 88
    frame #18: 0x000000010176e8c4 libpython3.13t.dylib`_PyEval_EvalFrameDefault + 78772
    frame #19: 0x00000001019bb53c libpython3.13t.dylib`_PyFunction_Vectorcall + 528
    frame #20: 0x00000001016b0864 libpython3.13t.dylib`_PyObject_VectorcallTstate.llvm.16501924132349618227 + 80
    frame #21: 0x00000001017190b8 libpython3.13t.dylib`object_vacall.llvm.16501924132349618227 + 240
    frame #22: 0x0000000101718ec8 libpython3.13t.dylib`PyObject_CallMethodObjArgs + 108
    frame #23: 0x00000001016dafc8 libpython3.13t.dylib`PyImport_ImportModuleLevelObject + 2952
    frame #24: 0x00000001017817a8 libpython3.13t.dylib`_PyEval_EvalFrameDefault + 156312
    frame #25: 0x00000001017e4edc libpython3.13t.dylib`_PyEval_Vector + 484
    frame #26: 0x00000001017fa670 libpython3.13t.dylib`run_eval_code_obj + 196
    frame #27: 0x00000001017fa04c libpython3.13t.dylib`run_mod.llvm.17647242441854347667 + 148
    frame #28: 0x0000000101903818 libpython3.13t.dylib`_PyRun_SimpleStringFlagsWithName + 532
    frame #29: 0x000000010192b570 libpython3.13t.dylib`Py_RunMain + 648
    frame #30: 0x0000000101894ad0 libpython3.13t.dylib`pymain_main + 472
    frame #31: 0x00000001018948ec libpython3.13t.dylib`Py_BytesMain + 40
    frame #32: 0x000000019020f154 dyld`start + 2476
(lldb) run
There is a running process, kill it and restart?: [Y/n]
Process 97527 exited with status = 9 (0x00000009) killed
Process 97546 launched: '/Users/alexis/.rye/py/cpython@3.13.0/bin/python3.13t' (arm64)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
    import pydantic_core
ModuleNotFoundError: No module named 'pydantic_core'
Process 97546 exited with status = 1 (0x00000001)

[Alexis-D]

Relevant?

• Creating Module With pymodule Cause Segmentation Fault In Free-Threaded Python 3.13 On Mac  PyO3/pyo3#4310
• release: 0.23.0 PyO3/pyo3#4651 -> https://github.com/PyO3/pyo3/releases/tag/v0.23.0

[davidhewitt]

Thanks for the report. It is known that PyO3 0.22 is not compatible with the free-threaded build of 3.13, but the defence was broken until 0.22.6 patch release.

I have opened #1550 which we can ship in the next patch, which will prevent the error. Very soon (probably next week) I plan to bump to PyO3 0.23 which fully supports the free-threaded build.

[Alexis-D]

Great, thanks @davidhewitt!