GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-111650: Generate pyconfig.h on Windows by colesbury · Pull Request #112179 · python/cpython

Conversation

@colesbury

@colesbury colesbury commented Nov 16, 2023 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Prior to this change, the Py_NOGIL macro was not defined when building C API extensions with the --disable-gil build on Windows. Py_NOGIL was only defined as a pre-processor definition in pyproject.props, but that is not used by C-API extensions.

This instead generates the pyconfig.h header on Windows as part of the build process. For now, Py_NOGIL is the only macro that may be conditionally defined in the generated file.

  • Issue: Py_GIL_DISABLED not defined in "free-threaded" Windows C-API extensions  #111650

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@colesbury colesbury added type-feature A feature request or enhancement OS-windows 3.13 only security fixes topic-free-threading labels Nov 16, 2023
@bedevere-app bedevere-app Bot mentioned this pull request Nov 16, 2023
Py_GIL_DISABLED not defined in "free-threaded" Windows C-API extensions #111650
Closed
@colesbury
colesbury force-pushed the generate-pyconfig-win branch 2 times, most recently from e4c3520 to 912e169 Compare November 16, 2023 20:34
@colesbury

This comment was marked as outdated.

Sign in to view
@bedevere-bot

This comment was marked as outdated.

Sign in to view
@colesbury

colesbury commented Nov 16, 2023

Copy link
Copy Markdown
Contributor Author

!buildbot nogil

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot

bedevere-bot commented Nov 16, 2023

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @colesbury for commit b2920c4 🤖

The command will test the builders whose names match following regular expression: nogil

The builders matched are:

  • AMD64 Ubuntu NoGIL Refleaks PR
  • AMD64 Windows Server 2022 NoGIL PR
  • ARM64 MacOS M1 NoGIL PR
  • x86-64 MacOS Intel NoGIL PR
  • AMD64 Ubuntu NoGIL PR
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

colesbury added 3 commits November 20, 2023 10:47
@colesbury
pythongh-111650: Generate pyconfig.h on Windows
6585ab1
Prior to this change, the Py_NOGIL macro was not defined when building
C API extensions with the `--disable-gil` build on Windows. `Py_NOGIL`
was only defined as a pre-processor definition in pyproject.props, but
that is not used by C-API extensions.

This instead generates the `pyconfig.h` header on Windows as part of
the build process. For now, `Py_NOGIL` is the only macro that may
be conditionally defined in the generated file.
@colesbury
Fix test_sysconfig.py
dbe0ddd
@colesbury
Add blurb
25c6028
@colesbury
colesbury force-pushed the generate-pyconfig-win branch from b2920c4 to 25c6028 Compare November 20, 2023 15:47
@colesbury
Rename Py_NOGIL to Py_GIL_DISABLED
b654d10
zooba
zooba reviewed Nov 29, 2023
View reviewed changes
Comment thread PCbuild/pcbuild.proj

<Target Name="PreBuild" BeforeTargets="Build">
<!-- Stick a _PLACEHOLDER=1 after $(PyConfigArgs) to handle both trailing commas and empty $(PyConfigArgs) -->
<Exec Command="powershell.exe $(PySourcePath)PCbuild\generate_pyconfig.ps1 -File $(PySourcePath)PC\pyconfig.h -define $(PyConfigArgs)_PLACEHOLDER=1" />

@zooba zooba Nov 29, 2023

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would rather have this happen in pythoncore.vcxproj if possible, or _freeze_module.vcxproj if necessary (or both). That way we don't have to rely on building through this file to make sure it happens.

It probably also belongs in the regen.targets file, and can then be referenced from the others.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Lib/test/test_sysconfig.py
pyconfig_h = os.path.join(srcdir, 'PC', 'pyconfig.h')
self.assertTrue(os.path.exists(pyconfig_h), pyconfig_h)
if os.name == 'nt':
# <srcdir>/PC/pyconfig.h only exists on Windows.

@zooba zooba Nov 29, 2023

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly, it's probably a better long-term move to get rid of PC/pyconfig.h completely and generate it in Include where the POSIX one would be.

Eventually (though I suspect it's "likely never") we'll get a combined build system. At that point, it'll be much more sensible to have a coherent set of include files.

The code that tries to include header files in installers already assumes it can pick it up directly from a repository checkout (Tools/msi/dev and PC/layout, IIRC). That will need additional logic to pick it up from somewhere else - I'd suggest the build directory $(Py_OutDir) along with every other generated file that we include in the distribution.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread PCbuild/generate_pyconfig.ps1
Comment on lines +36 to +46
foreach ($i in 0..($lines.Length - 1)) {
if ($lines[$i] -match "^#undef (\w+)$") {
$key = $Matches[1]
if ($definedValues.ContainsKey($key)) {
$value = $definedValues[$key]
$lines[$i] = "#define $key $value".TrimEnd()
} else {
$lines[$i] = "/* #undef $key */"
}
}
}

@zooba zooba Nov 29, 2023

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
foreach ($i in 0..($lines.Length - 1)) {
if ($lines[$i] -match "^#undef (\w+)$") {
$key = $Matches[1]
if ($definedValues.ContainsKey($key)) {
$value = $definedValues[$key]
$lines[$i] = "#define $key $value".TrimEnd()
} else {
$lines[$i] = "/* #undef $key */"
}
}
}
$lines = $lines | %{
if ($_ -match "^#undef (\w+)$") {
$key = $Matches[1];
if ($definedValues.ContainsKey($key)) {
"#define $key ${definedValues[$key]}".TrimEnd()
} else {
"/* #undef $key */"
}
} else {
$_
}
}

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@zooba zooba Nov 29, 2023

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, that was a bit of a self-indulgent rewrite that didn't actually make it shorter, simpler, or probably faster 😆

But if you want, we already assume that a copy of Python is available for bootstrapping on Windows (and I believe we require 3.10), so you could make this a Python script. IIRC, $(HostPython) has the location - it should be used elsewhere in regen.targets if you need examples.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread PCbuild/pcbuild.proj
<IncludeSSL Condition="'$(IncludeSSL)' == ''">true</IncludeSSL>
<IncludeTkinter Condition="'$(IncludeTkinter)' == ''">true</IncludeTkinter>
<IncludeUwp Condition="'$(IncludeUwp)' == ''">false</IncludeUwp>
<PyConfigArgs Condition="'$(DisableGil)' == 'true'">Py_GIL_DISABLED=1,$(PyConfigArgs)</PyConfigArgs>

@zooba zooba Nov 29, 2023

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Semicolon separated is the usual for MSBuild files. I know I'll think this is wrong every time I see it - can we maybe $(PyConfigArgs.Replace(`;`, `,`)) later if it's really needed?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@colesbury colesbury closed this Dec 4, 2023
@colesbury
colesbury deleted the generate-pyconfig-win branch February 15, 2024 18:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zooba zooba zooba left review comments
@FFY00 FFY00 Awaiting requested review from FFY00 FFY00 will be requested when the pull request is marked ready for review FFY00 is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

3.13 only security fixes OS-windows topic-free-threading type-feature A feature request or enhancement

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@colesbury @bedevere-bot @zooba
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.