BUG: Race in computing initial value for reductions under free-threading

[hawkinsp]

Describe the issue:

With:

• CPython 3.13 built with the GIL disabled, using thread-sanitizer under clang,
• NumPy 2.3.0dev commit 35b2c4a built with thread-sanitizer and clang

thread sanitizer reports a race constructing the cached initial value for a reducer.

You may need to run the example a few times, but it reproduced first time for me.

Reproduce the code example:

import concurrent.futures
import threading
import numpy as np

num_threads = 8


x = np.arange(1000)
b = threading.Barrier(num_threads)
def closure():
  b.wait()
  np.sum(x)

with concurrent.futures.ThreadPoolExecutor(max_workers=num_threads) as executor:
  for _ in range(num_threads):
    executor.submit(closure)

Error message:

WARNING: ThreadSanitizer: data race (pid=4069590)
  Write of size 8 at 0x7f6412728ef8 by thread T135:
    #0 get_initial_from_ufunc legacy_array_method.c (_multiarray_umath.cpython-313t-x86_64-linux-gnu.so+0x3de246) (BuildId: a138a702a237ca030803af4168ee423ada9702f7)
    #1 PyUFunc_ReduceWrapper <null> (_multiarray_umath.cpython-313t-x86_64-linux-gnu.so+0x3e0426) (BuildId: a138a702a237ca030803af4168ee423ada9702f7)
    #2 PyUFunc_Reduce ufunc_object.c (_multiarray_umath.cpython-313t-x86_64-linux-gnu.so+0x3f0349) (BuildId: a138a702a237ca030803af4168ee423ada9702f7)
    #3 PyUFunc_GenericReduction ufunc_object.c (_multiarray_umath.cpython-313t-x86_64-linux-gnu.so+0x3efffa) (BuildId: a138a702a237ca030803af4168ee423ada9702f7)
    #4 ufunc_reduce ufunc_object.c (_multiarray_umath.cpython-313t-x86_64-linux-gnu.so+0x3ed841) (BuildId: a138a702a237ca030803af4168ee423ada9702f7)
    #5 cfunction_vectorcall_FASTCALL_KEYWORDS /usr/local/google/home/phawkins/p/cpython/Objects/methodobject.c:441:24 (python3.13+0x28a010) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #6 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #7 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #8 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #9 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e46e2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #10 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #11 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #12 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #13 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1eafaa) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #14 PyObject_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c:327:12 (python3.13+0x1eafaa)
    #15 dispatcher_vectorcall arrayfunction_override.c (_multiarray_umath.cpython-313t-x86_64-linux-gnu.so+0x22d22b) (BuildId: a138a702a237ca030803af4168ee423ada9702f7)
    #16 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1eafaa) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #17 PyObject_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c:327:12 (python3.13+0x1eafaa)
    #18 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:813:23 (python3.13+0x3e24fb) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #19 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #20 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #21 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #22 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef5ef) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #23 method_vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/classobject.c:70:20 (python3.13+0x1ef5ef)
    #24 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #25 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #26 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #27 thread_run /usr/local/google/home/phawkins/p/cpython/./Modules/_threadmodule.c:337:21 (python3.13+0x564292) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #28 pythread_wrapper /usr/local/google/home/phawkins/p/cpython/Python/thread_pthread.h:243:5 (python3.13+0x4bd637) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)

  Previous write of size 8 at 0x7f6412728ef8 by thread T131:
    #0 get_initial_from_ufunc legacy_array_method.c (_multiarray_umath.cpython-313t-x86_64-linux-gnu.so+0x3de246) (BuildId: a138a702a237ca030803af4168ee423ada9702f7)
    #1 PyUFunc_ReduceWrapper <null> (_multiarray_umath.cpython-313t-x86_64-linux-gnu.so+0x3e0426) (BuildId: a138a702a237ca030803af4168ee423ada9702f7)
    #2 PyUFunc_Reduce ufunc_object.c (_multiarray_umath.cpython-313t-x86_64-linux-gnu.so+0x3f0349) (BuildId: a138a702a237ca030803af4168ee423ada9702f7)
    #3 PyUFunc_GenericReduction ufunc_object.c (_multiarray_umath.cpython-313t-x86_64-linux-gnu.so+0x3efffa) (BuildId: a138a702a237ca030803af4168ee423ada9702f7)
    #4 ufunc_reduce ufunc_object.c (_multiarray_umath.cpython-313t-x86_64-linux-gnu.so+0x3ed841) (BuildId: a138a702a237ca030803af4168ee423ada9702f7)
    #5 cfunction_vectorcall_FASTCALL_KEYWORDS /usr/local/google/home/phawkins/p/cpython/Objects/methodobject.c:441:24 (python3.13+0x28a010) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #6 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #7 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #8 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #9 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:1355:26 (python3.13+0x3e46e2) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #10 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #11 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #12 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #13 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1eafaa) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #14 PyObject_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c:327:12 (python3.13+0x1eafaa)
    #15 dispatcher_vectorcall arrayfunction_override.c (_multiarray_umath.cpython-313t-x86_64-linux-gnu.so+0x22d22b) (BuildId: a138a702a237ca030803af4168ee423ada9702f7)
    #16 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1eafaa) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #17 PyObject_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c:327:12 (python3.13+0x1eafaa)
    #18 _PyEval_EvalFrameDefault /usr/local/google/home/phawkins/p/cpython/Python/generated_cases.c.h:813:23 (python3.13+0x3e24fb) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #19 _PyEval_EvalFrame /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_ceval.h:119:16 (python3.13+0x3de62a) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #20 _PyEval_Vector /usr/local/google/home/phawkins/p/cpython/Python/ceval.c:1811:12 (python3.13+0x3de62a)
    #21 _PyFunction_Vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/call.c (python3.13+0x1eb61f) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #22 _PyObject_VectorcallTstate /usr/local/google/home/phawkins/p/cpython/./Include/internal/pycore_call.h:168:11 (python3.13+0x1ef5ef) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #23 method_vectorcall /usr/local/google/home/phawkins/p/cpython/Objects/classobject.c:70:20 (python3.13+0x1ef5ef)
    #24 _PyVectorcall_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:273:16 (python3.13+0x1eb293) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #25 _PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:348:16 (python3.13+0x1eb293)
    #26 PyObject_Call /usr/local/google/home/phawkins/p/cpython/Objects/call.c:373:12 (python3.13+0x1eb315) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #27 thread_run /usr/local/google/home/phawkins/p/cpython/./Modules/_threadmodule.c:337:21 (python3.13+0x564292) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)
    #28 pythread_wrapper /usr/local/google/home/phawkins/p/cpython/Python/thread_pthread.h:243:5 (python3.13+0x4bd637) (BuildId: 9c1c16fb1bb8a435fa6fa4c6944da5d41f654e96)

Python and NumPy Versions:

2.3.0.dev0+git20241219.35b2c4a
3.13.1 experimental free-threading build (tags/v3.13.1:06714517797, Dec 15 2024, 15:38:01) [Clang 18.1.8 (11)]

Runtime Environment:

[{'numpy_version': '2.3.0.dev0+git20241219.35b2c4a',
'python': '3.13.1 experimental free-threading build '
'(tags/v3.13.1:06714517797, Dec 15 2024, 15:38:01) [Clang 18.1.8 '
'(11)]',
'uname': uname_result(system='Linux', node='', release='', version='#1 SMP PREEMPT_DYNAMIC Debian 6.redacted (2024-10-16)', machine='x86_64')},
{'simd_extensions': {'baseline': ['SSE', 'SSE2', 'SSE3'],
'found': ['SSSE3',
'SSE41',
'POPCNT',
'SSE42',
'AVX',
'F16C',
'FMA3',
'AVX2'],
'not_found': ['AVX512F',
'AVX512CD',
'AVX512_KNL',
'AVX512_SKX',
'AVX512_CLX',
'AVX512_CNL',
'AVX512_ICL']}},
{'architecture': 'Zen',
'filepath': '/usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.27.so',
'internal_api': 'openblas',
'num_threads': 128,
'prefix': 'libopenblas',
'threading_layer': 'pthreads',
'user_api': 'blas',
'version': '0.3.27'}]

Context for the issue:

Found when working on free-threading support in JAX.

[seberg]

Doesn't seem too worrying in practice. Anyway, without diving in, my guess is that we can move this to ArrayMethod creation time where it is already guarded (either at import or during promotion).