GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — gh-143120: pixi builds for free-threading and TSAN by crusaderky · Pull Request #142872 · python/cpython Conversation
|
| script: | ||
| file: ../build.sh | ||
| env: | ||
| PYTHON_VARIANT: "free-threading" |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
copy-paste of asan/recipe.yaml, except this one line
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
copy-paste of asan/pixi.toml
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| script: | ||
| file: ../build.sh | ||
| env: | ||
| PYTHON_VARIANT: "tsan" |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
copy-paste of asan/recipe.yaml, except this one line
Sorry, something went wrong.
There was an error while loading. Please reload this page.
no clue sorry I haven't seen this before |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
The tsan crash is due to the default $ sudo sysctl vm.mmap_rnd_bits vm.mmap_rnd_bits = 32 # too high $ sudo sysctl vm.mmap_rnd_bits=28 # reduce it vm.mmap_rnd_bits = 28
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
There's a lot of copy/paste here, which can make maintenance harder. Does pixi have some concept of code reuse or parametrisation? Maybe not the best example, but something like GitHub Actions' reusable workflows? |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@lucascolley is there anything we can do right now to reduce the duplication here? Or does that require fixes in pixi? |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
As mentioned in the README already, this is blocked on prefix-dev/pixi#4599. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I opened an issue which is needed to make this mergeable. I'm also playing with a NumPy recipe based on this.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
Ready for final review |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
The issue was that - if: c_compiler == "gcc" and "san" in variant then: - libsanitizer - if: c_compiler == "clang" and "san" in variant then: - libcompiler-rt Linux happened to work because, at least on ubuntu-latest and on my desktop, libasan.so and libtsan.so are installed by default system-wide. I've opened an improvement ticket upstream: prefix-dev/pixi#5393 |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
I've removed the symlinks and replaced them with physical copies + a sync script. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Gentle ping. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
trying this out downstream — I get a solve failure for an env that includes the ASAN Python on this branch, ASAN NumPy on main, removing the |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
ASAN NumPy on main points to a specific commit of CPython main. Could you try repointing numpy to numpy/numpy#30510 ? [dependencies] python.git = "https://github.com/crusaderky/cpython" python.rev = "tsan" python.subdirectory = "Tools/pixi-packages/asan" numpy.git = "https://github.com/ngoldbaum/numpy" numpy.rev = "tsan-ft-pixi" numpy.subdirectory = "pixi-packages/asan" |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
Anyway your problem is that you have an upper pin for python in pythran:
but this recipe builds Python 3.15. Alternatively you can fork pythran and just remove the pin and hope it was just out of an abundance of caution. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
ah thanks, makes sense. Using pythran from git source may get around the issue in the short-term |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I tested the tsan+freethreading pixi config with the following and it worked:
[workspace] channels = ["conda-forge"] name = "cpython-pixi-test" platforms = ["linux-64"] preview = ["pixi-build"] [dependencies] python = { git = "https://github.com/crusaderky/cpython", rev = "tsan", subdirectory = "Tools/pixi-packages/tsan-freethreading" }
I'll merge this by tomorrow if there are no objections.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
| - Using a single `pixi.toml` and `recipe.yaml` for all package variants is blocked on https://github.com/prefix-dev/pixi/issues/4599 | ||
| - A workaround can be removed from the build script once https://github.com/prefix-dev/rattler-build/issues/2012 is resolved | ||
| - Using a single `pixi.toml` and `recipe.yaml` for all package variants is blocked on | ||
| [pixi-build-backends#532](https://github.com/prefix-dev/pixi-build-backends/pull/532) |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
No need to update the README.md right now, but noting this repo has been archived and the new PR is at prefix-dev/pixi#5364.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| ``` | ||
| To fix it, try reducing `mmap_rnd_bits`: | ||
|
|
||
| ```bash |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Tip for next time: bash for when there's only commands, and console for when there's commands with output. Compare:
$ sudo sysctl vm.mmap_rnd_bits vm.mmap_rnd_bits = 32 # too high for TSan $ sudo sysctl vm.mmap_rnd_bits=28 # reduce it vm.mmap_rnd_bits = 28
$ sudo sysctl vm.mmap_rnd_bits vm.mmap_rnd_bits = 32 # too high for TSan $ sudo sysctl vm.mmap_rnd_bits=28 # reduce it vm.mmap_rnd_bits = 28
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Co-authored-by: Isuru Fernando <[email protected]> Co-authored-by: Lucas Colley <[email protected]>
Co-authored-by: Isuru Fernando <[email protected]> Co-authored-by: Lucas Colley <[email protected]>
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.