GeezerVison

gcc -c -fno-strict-overflow -Wsign-compare -Wunreachable-code -DNDEBUG -g -O3 -Wall    -D_Py_TIER2=1 -D_Py_JIT -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden  -I./Include/internal -I./Include/internal/mimalloc  -I. -I./Include    -DPy_BUILD_CORE -o Python/jit.o Python/jit.c
In file included from Python/jit.c:392:
./jit_stencils.h:13419:49: error: use of undeclared identifier '_PyCriticalSection_Resume'
    patch_aarch64_16a(code + 0x198, (uintptr_t)&_PyCriticalSection_Resume);

(followed by many more of the same error).

I don't recall if I've tried this configure combination before or not. I'm using Apple's clang on an up-to-date MacOS (14.4.1):

% gcc --version
Apple clang version 15.0.0 (clang-1500.3.9.4)
Target: arm64-apple-darwin23.4.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

CPython versions tested on:

CPython main branch

Operating systems tested on:

macOS

Linked PRs

Activity

smontanaro
added
type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
on May 9, 2024
Eclips4
added
OS-mac

buildThe build process and cross-buildThe build process and cross-build
on May 9, 2024
Eclips4
changed the title ~~[-]--disable-gil & --wirh-experimental-jit fails to build on M1 Mac[/-]~~ [+]--disable-gil & --enable-experimental-jit fails to build on M1 Mac[/+] on May 9, 2024

Eclips4 commented on May 9, 2024

Eclips4

on May 9, 2024

Member

More actions

Full compilation error:

Details

gcc -c -fno-strict-overflow -Wsign-compare -g -Og -Wall    -D_Py_TIER2=1 -D_Py_JIT -std=c11 -Wextra -Wno-unused-parameter -Wno-missing-field-initializers -Wstrict-prototypes -Werror=implicit-function-declaration -fvisibility=hidden  -I./Include/internal -I./Include/internal/mimalloc  -I. -I./Include    -DPy_BUILD_CORE -o Python/jit.o Python/jit.c
In file included from Python/jit.c:392:
./jit_stencils.h:24576:49: error: use of undeclared identifier '_PyCriticalSection_Resume'
    patch_aarch64_16a(code + 0x230, (uintptr_t)&_PyCriticalSection_Resume);
                                                ^
./jit_stencils.h:24577:49: error: use of undeclared identifier '_PyCriticalSection_Resume'
    patch_aarch64_16b(code + 0x234, (uintptr_t)&_PyCriticalSection_Resume);
                                                ^
./jit_stencils.h:24578:49: error: use of undeclared identifier '_PyCriticalSection_Resume'
    patch_aarch64_16c(code + 0x238, (uintptr_t)&_PyCriticalSection_Resume);
                                                ^
./jit_stencils.h:24579:49: error: use of undeclared identifier '_PyCriticalSection_Resume'
    patch_aarch64_16d(code + 0x23c, (uintptr_t)&_PyCriticalSection_Resume);
                                                ^
./jit_stencils.h:24596:49: error: use of undeclared identifier '_PyCriticalSection_BeginSlow'
    patch_aarch64_16a(code + 0x2a8, (uintptr_t)&_PyCriticalSection_BeginSlow);
                                                ^
./jit_stencils.h:24597:49: error: use of undeclared identifier '_PyCriticalSection_BeginSlow'
    patch_aarch64_16b(code + 0x2ac, (uintptr_t)&_PyCriticalSection_BeginSlow);
                                                ^
./jit_stencils.h:24598:49: error: use of undeclared identifier '_PyCriticalSection_BeginSlow'
    patch_aarch64_16c(code + 0x2b0, (uintptr_t)&_PyCriticalSection_BeginSlow);
                                                ^
./jit_stencils.h:24599:49: error: use of undeclared identifier '_PyCriticalSection_BeginSlow'
    patch_aarch64_16d(code + 0x2b4, (uintptr_t)&_PyCriticalSection_BeginSlow);
                                                ^
./jit_stencils.h:45384:49: error: use of undeclared identifier '_PyCriticalSection_Resume'
    patch_aarch64_16a(code + 0x1c0, (uintptr_t)&_PyCriticalSection_Resume);
                                                ^
./jit_stencils.h:45385:49: error: use of undeclared identifier '_PyCriticalSection_Resume'
    patch_aarch64_16b(code + 0x1c4, (uintptr_t)&_PyCriticalSection_Resume);
                                                ^
./jit_stencils.h:45386:49: error: use of undeclared identifier '_PyCriticalSection_Resume'
    patch_aarch64_16c(code + 0x1c8, (uintptr_t)&_PyCriticalSection_Resume);
                                                ^
./jit_stencils.h:45387:49: error: use of undeclared identifier '_PyCriticalSection_Resume'
    patch_aarch64_16d(code + 0x1cc, (uintptr_t)&_PyCriticalSection_Resume);
                                                ^
./jit_stencils.h:45404:49: error: use of undeclared identifier '_PyCriticalSection_BeginSlow'
    patch_aarch64_16a(code + 0x238, (uintptr_t)&_PyCriticalSection_BeginSlow);
                                                ^
./jit_stencils.h:45405:49: error: use of undeclared identifier '_PyCriticalSection_BeginSlow'
    patch_aarch64_16b(code + 0x23c, (uintptr_t)&_PyCriticalSection_BeginSlow);
                                                ^
./jit_stencils.h:45406:49: error: use of undeclared identifier '_PyCriticalSection_BeginSlow'
    patch_aarch64_16c(code + 0x240, (uintptr_t)&_PyCriticalSection_BeginSlow);
                                                ^
./jit_stencils.h:45407:49: error: use of undeclared identifier '_PyCriticalSection_BeginSlow'
    patch_aarch64_16d(code + 0x244, (uintptr_t)&_PyCriticalSection_BeginSlow);
                                                ^
./jit_stencils.h:48060:49: error: use of undeclared identifier '_PyCriticalSection_Resume'
    patch_aarch64_16a(code + 0x358, (uintptr_t)&_PyCriticalSection_Resume);
                                                ^
./jit_stencils.h:48061:49: error: use of undeclared identifier '_PyCriticalSection_Resume'
    patch_aarch64_16b(code + 0x35c, (uintptr_t)&_PyCriticalSection_Resume);
                                                ^
./jit_stencils.h:48062:49: error: use of undeclared identifier '_PyCriticalSection_Resume'
    patch_aarch64_16c(code + 0x360, (uintptr_t)&_PyCriticalSection_Resume);
                                                ^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
make: *** [Python/jit.o] Error 1

Reacted by Savannah Ostrowski

brandtbucher commented on May 9, 2024
brandtbucher
on May 9, 2024
Member
More actions
Thanks! I’m not sure if this is a build configuration that’s been given any thought yet... ~~or if it even makes sense right now (I believe our tier 2 optimization pipeline is disabled on free-threaded builds, but I’d need to check)~~. Regardless, we’re headed towards a future with both of these features and should probably make sure that they at least build together.

@savannahostrowski, interested in taking this too? I’m on my phone now, but I think we probably just need to #include the header where this is defined in jit.c.
brandtbucher
self-assigned this
on May 9, 2024

sobolevn commented on May 9, 2024

sobolevn

on May 9, 2024

Member

More actions

Here' the code sample that is generated:

memcpy(data, data_body, sizeof(data_body));
    patch_64(data + 0x0, instruction->oparg);
    memcpy(code, code_body, sizeof(code_body));
    patch_aarch64_33rx(code + 0x30, (uintptr_t)data);
    patch_aarch64_26r(code + 0x16c, (uintptr_t)code + sizeof(code_body));
    patch_aarch64_33rx(code + 0x190, (uintptr_t)data);
    patch_aarch64_26r(code + 0x1c8, instruction_starts[instruction->error_target]);
    patch_aarch64_16a(code + 0x1d0, (uintptr_t)&_PyEval_FormatExcUnbound);
    patch_aarch64_16b(code + 0x1d4, (uintptr_t)&_PyEval_FormatExcUnbound);
    patch_aarch64_16c(code + 0x1d8, (uintptr_t)&_PyEval_FormatExcUnbound);
    patch_aarch64_16d(code + 0x1dc, (uintptr_t)&_PyEval_FormatExcUnbound);
    patch_aarch64_16a(code + 0x1e8, (uintptr_t)&_PyCriticalSection_Resume);
    patch_aarch64_16b(code + 0x1ec, (uintptr_t)&_PyCriticalSection_Resume);
    patch_aarch64_16c(code + 0x1f0, (uintptr_t)&_PyCriticalSection_Resume);
    patch_aarch64_16d(code + 0x1f4, (uintptr_t)&_PyCriticalSection_Resume);
    patch_aarch64_16a(code + 0x200, (uintptr_t)&_PyThreadState_GetCurrent);
    patch_aarch64_16b(code + 0x204, (uintptr_t)&_PyThreadState_GetCurrent);
    patch_aarch64_16c(code + 0x208, (uintptr_t)&_PyThreadState_GetCurrent);
    patch_aarch64_16d(code + 0x20c, (uintptr_t)&_PyThreadState_GetCurrent);

savannahostrowski commented on May 9, 2024
savannahostrowski
on May 9, 2024
Member
More actions
Yep - feel free to assign to me @brandtbucher! I'll take a look tonight or tomorrow!
brandtbucher
assigned
savannahostrowski
and unassigned
brandtbucher
on May 9, 2024
smontanaro commented on May 9, 2024
smontanaro
on May 9, 2024
ContributorAuthor
More actions
FWIW, I'm not sure this is M1-specific. It compiled just fine on my Dell laptop running XUbuntu 22.04. OTOH, on an old MacBook Air I have laying around (dual core Intel i5, MacOS 12.7.4), it failed to compile Python/jit.c in the same fashion as on my M1 MBP. Maybe it's a MacOS issue?
Reacted by Savannah Ostrowski
brandtbucher
added a commit that references this issue on May 11, 2024
GH-118844: Fix build failures when combining --disable-gil with --ena…
5b941e5
miss-islington
added a commit that references this issue on May 11, 2024
pythonGH-118844: Fix build failures when combining --disable-gil with…
f3612d8
brandtbucher
closed this as completed on May 11, 2024
smontanaro commented on May 13, 2024
smontanaro
on May 13, 2024
ContributorAuthor
More actions
@brandtbucher @savannahostrowski

I saw the close-out of the PR and apparent backport to 3.13. I can build fine on main, but compilation still fails for me on 3.13. On both main and 3.13 I used this configure command (grepped from config.status):

/bin/sh './configure' '--enable-experimental-jit=yes' '--disable-gil' '--with-openssl=/opt/homebrew' 'GDBM_CFLAGS=-I/opt/homebrew/opt/gdbm/include' 'GDBM_LIBS=-L/opt/homebrew/opt/gdbm/lib -lgdbm' 'LIBLZMA_CFLAGS=-I/opt/homebrew/include' 'LIBLZMA_LIBS=-L/opt/homebrew/lib -llzma' $ac_configure_extra_args --no-create --no-recursion

I thought the PR was backported to 3.13, but maybe not? Diffing the main and 3.13 versions of Python/jit.c I see this line is missing from the 3.13 version:

#include "pycore_critical_section.h"

Adding it allows the compilation to succeed. I don't see it in the jit.c source on Github either.
brandtbucher commented on May 13, 2024
brandtbucher
on May 13, 2024
Member
More actions

I saw the close-out of the PR and apparent backport to 3.13. I can build fine on main, but compilation still fails for me on 3.13.

Ah, sorry, looks like the backport failed! Re-running it now...
brandtbucher
added a commit that references this issue on May 13, 2024
[3.13] GH-118844: Fix build failures when combining --disable-gil wit…
9d2c10b
brandtbucher commented on May 13, 2024
brandtbucher
on May 13, 2024
Member
More actions
...done!
brandtbucher
added
topic-JIT
on May 31, 2024
estyxx
added a commit that references this issue on Jul 17, 2024
pythonGH-118844: Fix build failures when combining --disable-gil with…
7652b6e

Metadata

Assignees

savannahostrowski

Labels

OS-macbuildThe build process and cross-buildThe build process and cross-buildtopic-JITtype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

GeezerVison

--disable-gil & --enable-experimental-jit fails to build on M1 Mac #118844

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

Eclips4 commented on May 9, 2024

brandtbucher commented on May 9, 2024

sobolevn commented on May 9, 2024

savannahostrowski commented on May 9, 2024

smontanaro commented on May 9, 2024

smontanaro commented on May 13, 2024

brandtbucher commented on May 13, 2024

brandtbucher commented on May 13, 2024

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions