GeezerVison

Copy link

Copy Markdown

Contributor

vasily-v-ryabov and sergey-miryanov reacted with thumbs up emoji

DinoV force-pushed the nogil/mimalloc_212 branch 7 times, most recently from 8b3ec52 to 06e86d7 Compare September 27, 2023 00:25

DinoV force-pushed the nogil/mimalloc_212 branch 4 times, most recently from e8c4f01 to 2dd8675 Compare September 27, 2023 15:59

DinoV requested a review from ericsnowcurrently September 27, 2023 16:04

DinoV force-pushed the nogil/mimalloc_212 branch from 2dd8675 to 672b6f4 Compare September 27, 2023 17:35

DinoV requested a review from colesbury September 27, 2023 17:36

DinoV force-pushed the nogil/mimalloc_212 branch from 672b6f4 to 41b62cb Compare September 27, 2023 17:52

DinoV force-pushed the nogil/mimalloc_212 branch from 41b62cb to 0dbe76b Compare September 27, 2023 18:19

DinoV force-pushed the nogil/mimalloc_212 branch 7 times, most recently from 7bda1e4 to 35d1ebd Compare September 28, 2023 19:52

DinoV marked this pull request as ready for review September 28, 2023 20:21

DinoV requested a review from rhettinger as a code owner September 28, 2023 20:21

vstinner reviewed Oct 30, 2023

View reviewed changes

vstinner reviewed Oct 30, 2023

View reviewed changes

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

vstinner enabled auto-merge (squash) October 30, 2023 15:25

In file included from ../../Objects/obmalloc.c:15:
In file included from ../../Objects/mimalloc/static.c:37:
In file included from ../../Objects/mimalloc/prim/prim.c:19:
../../Objects/mimalloc/prim/wasi/prim.c:44:15: error: implicit declaration of function 'sbrk' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    void* p = sbrk(size);
              ^
1 error generated.
make: *** [Makefile:2723: Objects/obmalloc.o] Error 1

* Add mimalloc v2.12

Modified src/alloc.c to remove include of alloc-override.c and not
compile new handler.

Did not include the following files:

 - include/mimalloc-new-delete.h
 - include/mimalloc-override.h
 - src/alloc-override-osx.c
 - src/alloc-override.c
 - src/static.c
 - src/region.c

mimalloc is thread safe and shares a single heap across all runtimes,
therefore finalization and getting global allocated blocks across all
runtimes is different.

* mimalloc: minimal changes for use in Python:

 - remove debug spam for freeing large allocations
 - use same bytes (0xDD) for freed allocations in CPython and mimalloc
   This is important for the test_capi debug memory tests

* Don't export mimalloc symbol in libpython.
* Enable mimalloc as Python allocator option.
* Add mimalloc MIT license.
* Log mimalloc in Lib/test/pythoninfo.py.
* Document new mimalloc support.
* Use macro defs for exports as done in:
  python#31164

Co-authored-by: Sam Gross <[email protected]>
Co-authored-by: Christian Heimes <[email protected]>
Co-authored-by: Victor Stinner <[email protected]>

* Add mimalloc v2.12

Modified src/alloc.c to remove include of alloc-override.c and not
compile new handler.

Did not include the following files:

 - include/mimalloc-new-delete.h
 - include/mimalloc-override.h
 - src/alloc-override-osx.c
 - src/alloc-override.c
 - src/static.c
 - src/region.c

mimalloc is thread safe and shares a single heap across all runtimes,
therefore finalization and getting global allocated blocks across all
runtimes is different.

* mimalloc: minimal changes for use in Python:

 - remove debug spam for freeing large allocations
 - use same bytes (0xDD) for freed allocations in CPython and mimalloc
   This is important for the test_capi debug memory tests

* Don't export mimalloc symbol in libpython.
* Enable mimalloc as Python allocator option.
* Add mimalloc MIT license.
* Log mimalloc in Lib/test/pythoninfo.py.
* Document new mimalloc support.
* Use macro defs for exports as done in:
  python#31164

Co-authored-by: Sam Gross <[email protected]>
Co-authored-by: Christian Heimes <[email protected]>
Co-authored-by: Victor Stinner <[email protected]>

DinoV deleted the nogil/mimalloc_212 branch May 31, 2024 18:23

* Add mimalloc v2.12

Modified src/alloc.c to remove include of alloc-override.c and not
compile new handler.

Did not include the following files:

 - include/mimalloc-new-delete.h
 - include/mimalloc-override.h
 - src/alloc-override-osx.c
 - src/alloc-override.c
 - src/static.c
 - src/region.c

mimalloc is thread safe and shares a single heap across all runtimes,
therefore finalization and getting global allocated blocks across all
runtimes is different.

* mimalloc: minimal changes for use in Python:

 - remove debug spam for freeing large allocations
 - use same bytes (0xDD) for freed allocations in CPython and mimalloc
   This is important for the test_capi debug memory tests

* Don't export mimalloc symbol in libpython.
* Enable mimalloc as Python allocator option.
* Add mimalloc MIT license.
* Log mimalloc in Lib/test/pythoninfo.py.
* Document new mimalloc support.
* Use macro defs for exports as done in:
  python#31164

Co-authored-by: Sam Gross <[email protected]>
Co-authored-by: Christian Heimes <[email protected]>
Co-authored-by: Victor Stinner <[email protected]>

… 3.13

Python 3.13 introduced mimalloc memory allocator [1], which registers
an `atexit()` handler [2].

On NetBSD 11, an `atexit()` handler registered in a shared library is
not executed upon the library unload (e.g., via `dlclose()`), but rather
on program exit. So unloading a library that registers an `atexit()`
handler results in segmentation fault in libc's `__cxa_finalize` when
the program exits, because the handler is no longer reachable...

The problem seems to be specific to NetBSD; on other platforms
(including OpenBSD and FreeBSD), the handler is executed upon the
library unload.

[1] python/cpython#109914
[2] https://github.com/python/cpython/blob/05f2f0ac92afa560315eb66fd6576683c7f69e2d/Objects/mimalloc/init.c#L530

… 3.13

Python 3.13 introduced mimalloc memory allocator [1], which registers
an `atexit()` handler [2].

On NetBSD 11, an `atexit()` handler registered in a shared library is
not executed upon the library unload (e.g., via `dlclose()`), but rather
on program exit. So unloading a library that registers an `atexit()`
handler results in segmentation fault in libc's `__cxa_finalize` when
the program exits, because the handler is no longer reachable...

The problem seems to be specific to NetBSD; on other platforms
(including OpenBSD and FreeBSD), the handler is executed upon the
library unload.

[1] python/cpython#109914
[2] https://github.com/python/cpython/blob/05f2f0ac92afa560315eb66fd6576683c7f69e2d/Objects/mimalloc/init.c#L530