GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-93939: Build C extensions without setup.py by tiran · Pull Request #94474 · python/cpython

Conversation

@tiran

@tiran tiran commented Jul 1, 2022 •
edited by miss-islington
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Combines GH-93940, GH-94452, and GH-94433

  • Issue: Build shared extensions without distutils #93939

Automerge-Triggered-By: GH:tiran

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

haampie reacted with heart emoji erlend-aasland, kumaraditya303, methane, and Osyotr reacted with rocket emoji yan12125 and thesamesam reacted with eyes emoji
All reactions
  • ❤️ 1 reaction
  • 🚀 4 reactions
  • 👀 2 reactions
@bedevere-bot bedevere-bot added the awaiting core review label Jul 1, 2022
@tiran tiran added the DO-NOT-MERGE label Jul 1, 2022
@tiran
tiran force-pushed the gh-93939-setup-stdlib-all branch 3 times, most recently from 1a3a959 to af65a4c Compare July 6, 2022 11:58
@tiran tiran changed the title gh-93939: [PoC] Build Python without setup.py gh-93939: Build C extensions without setup.py Jul 6, 2022
@tiran tiran added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 6, 2022
@bedevere-bot

bedevere-bot commented Jul 6, 2022

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @tiran for commit 1469caf9703e411b73a00169a6bf0c5e688f5e72 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 6, 2022
@tiran
tiran force-pushed the gh-93939-setup-stdlib-all branch from 1469caf to 68b3032 Compare July 7, 2022 11:00
@tiran
tiran marked this pull request as ready for review July 7, 2022 13:10
@tiran tiran added 🔨 test-with-buildbots Test PR w/ buildbots; report in status section build The build process and cross-build and removed DO-NOT-MERGE labels Jul 7, 2022
@bedevere-bot

bedevere-bot commented Jul 7, 2022

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @tiran for commit 68b3032229ee0185f029772e5e43f72cbb4b402d 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 7, 2022
@tiran
tiran force-pushed the gh-93939-setup-stdlib-all branch from 68b3032 to 11701ba Compare July 8, 2022 09:55
@merwok

merwok commented Jul 8, 2022

Copy link
Copy Markdown
Member

Is there a need for force pushes? Github notifications lead to nothing, which is a little annoying.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@pradyunsg

pradyunsg commented Jul 8, 2022

Copy link
Copy Markdown
Member

FWIW, You can click on "force-pushed" in the event about the force push, to see what changed between the two force-pushes.

Submitting feedback to GitHub to have the notifications for force-pushed go to that, instead of "oh, I can't show you what changed" may help make GitHub prioritze fixing that higher.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@pradyunsg pradyunsg mentioned this pull request Jul 8, 2022
PEP 632: Remove the distutils package (Python 3.12) #92584
Closed
@tiran

tiran commented Jul 8, 2022

Copy link
Copy Markdown
Member Author

I only force push to rebase. It is the easiest and fastest way for me to resolve merge conflicts. The PR has frequent conflicts, even right now.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@tiran

tiran commented Jul 8, 2022

Copy link
Copy Markdown
Member Author

image

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@tiran
tiran force-pushed the gh-93939-setup-stdlib-all branch 2 times, most recently from 40e1554 to 87f243d Compare July 9, 2022 11:19
@tiran tiran added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 9, 2022
@bedevere-bot

bedevere-bot commented Jul 9, 2022

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @tiran for commit 87f243d764d4909388aed35e59f177d598c096ca 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Jul 9, 2022
@tiran
tiran requested review from erlend-aasland and vstinner July 9, 2022 15:53
@tiran

tiran commented Jul 12, 2022

Copy link
Copy Markdown
Member Author

setup.py's check_extension_import feature is now provided by a script. Here is an example output with tk-dev missing and a modified _sqlite that is missing _sqlite/util.c dependency.

[ERROR] _sqlite3 failed to import: build/lib.linux-x86_64-3.12/_sqlite3.cpython-312-x86_64-linux-gnu.so: undefined symbol: _pysqlite_seterror

The necessary bits to build these optional modules were not found:
_tkinter                                                       
To find the necessary bits, look in configure.ac and config.log.

Following modules built successfully but were removed because they could not be imported:
_sqlite3                                                       

Checked 109 modules (30 built-in, 76 shared, 1 n/a on linux-x86_64, 0 disabled, 1 missing, 1 failed on import)

validate_tzpath is performed by configure.ac

xxsubtype is now an optional extension. It is skipped when Python is built without test C extension modules.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@tiran
Add whatsnew entry
45e1ba0
@tiran
tiran force-pushed the gh-93939-setup-stdlib-all branch from 013af91 to 45e1ba0 Compare July 13, 2022 21:31
@tiran tiran added 🤖 automerge labels Jul 14, 2022
@miss-islington

miss-islington commented Jul 14, 2022

Copy link
Copy Markdown
Contributor

Sorry, I can't merge this PR. Reason: Base branch was modified. Review and try the merge again..

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@miss-islington
miss-islington merged commit 81dca70 into python:main Jul 14, 2022
@bedevere-bot bedevere-bot removed the awaiting merge label Jul 14, 2022
@tiran
tiran deleted the gh-93939-setup-stdlib-all branch July 14, 2022 07:52
@tiran tiran mentioned this pull request Jul 14, 2022
_decimal fails to build with --with-pydebug --with-lto on GCC #94847
Closed
@tiran

tiran commented Jul 14, 2022

Copy link
Copy Markdown
Member Author

#94848 addresses the build issue with LTO + pydebug.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@arhadthedev arhadthedev mentioned this pull request Jul 14, 2022
Deprecate and remove distutils #85454
Closed
@eendebakpt eendebakpt mentioned this pull request Jul 16, 2022
gh-72793: C implementation of parts of copy.deepcopy #91610
Closed
@vstinner

vstinner commented Aug 3, 2022

Copy link
Copy Markdown
Member

My little precious setup.py... YOU WILL NEVER BE FORGOTTEN 😭😭😭

_bootsubprocess.py (that I wrote)... I hate you, I'm glad that you're dead! If I recall correctly, I had to write _bootsubprocess to use the clean subprocess module, rather than naive fork+exec code, in distutils. distutils... I also hate you by the way.

erlend-aasland, zooba, and Eclips4 reacted with laugh emoji
All reactions
  • 😄 3 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ayappanec

ayappanec commented Aug 26, 2022

Copy link
Copy Markdown
Contributor

This broke the AIX build with the below error.

./python -E -S -m sysconfig --generate-posix-vars ;
if test $? -ne 0 ; then
echo "generate-posix-vars failed" ;
rm -f ./pybuilddir.txt ;
exit 1 ;
fi

Traceback (most recent call last):
File "/home/buildusr/cpython_master/cpython/Lib/_aix_support.py", line 7, in
import subprocess
File "/home/buildusr/cpython_master/cpython/Lib/subprocess.py", line 104, in
from _posixsubprocess import fork_exec as _fork_exec
ModuleNotFoundError: No module named '_posixsubprocess'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in _run_code
File "/home/buildusr/cpython_master/cpython/Lib/sysconfig.py", line 851, in
_main()
File "/home/buildusr/cpython_master/cpython/Lib/sysconfig.py", line 839, in _main
_generate_posix_vars()
File "/home/buildusr/cpython_master/cpython/Lib/sysconfig.py", line 511, in _generate_posix_vars
pybuilddir = f'build/lib.{get_platform()}-{_PY_VERSION_SHORT}'
^^^^^^^^^^^^^^
File "/home/buildusr/cpython_master/cpython/Lib/sysconfig.py", line 781, in get_platform
from _aix_support import aix_platform
File "/home/buildusr/cpython_master/cpython/Lib/_aix_support.py", line 11, in
import _bootsubprocess as subprocess
ModuleNotFoundError: No module named '_bootsubprocess'
generate-posix-vars failed
gmake: *** [Makefile:871: pybuilddir.txt] Error 1
gmake: *** Waiting for unfinished jobs....

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@tiran

tiran commented Aug 26, 2022

Copy link
Copy Markdown
Member Author

Please open a new bug for the problem.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ayappanec ayappanec mentioned this pull request Aug 26, 2022
AIX build is broken because of _bootsubprocess removal. #96305
Closed
@hroncok hroncok mentioned this pull request Oct 26, 2022
--with-system-libmpdec / --with-system-expat still uses the vendored headers #98707
Closed
@hroncok

hroncok commented Oct 26, 2022

Copy link
Copy Markdown
Contributor

I found a regression, reported as #98707

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hroncok hroncok mentioned this pull request Oct 27, 2022
make install DESTDIR=... does not create the lib-dynload directory in DESTDIR if the real lib-dynload directory exists #98782
Closed
@haampie

haampie commented Jan 13, 2023

Copy link
Copy Markdown
Contributor

Thanks for doing this!

vstinner reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@danielsp13 danielsp13 mentioned this pull request Feb 7, 2023
[M2 - Dev] Problema en test de versión 3.12-rc danielsp13/CorrectIt#76
Closed
@AA-Turner AA-Turner mentioned this pull request Aug 16, 2023
GH-92584: Remove references to Distutils in configure.rst #108043
Merged
This was referenced Dec 26, 2023
Improve termcap detection in setup.py #67473
Closed
Need setup.py to pick up -isystem flags from CPPFLAGS #81944
Closed
Python build should check CPATH, C_INCLUDE_PATH for module dependencies #65770
Closed
setup.py only adds /prefix/lib, not /prefix/lib64 #58996
Closed
Support OpenCSW in setup.py #59848
Closed
setup.py: do not add system header locations when cross compiling #64410
Closed
@vasiliyk vasiliyk mentioned this pull request Nov 2, 2026
python should inherit the library search path from the compiler for stdlib extensions #56627
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@zware zware zware approved these changes
@erlend-aasland erlend-aasland Awaiting requested review from erlend-aasland erlend-aasland is a code owner
@vstinner vstinner Awaiting requested review from vstinner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

build The build process and cross-build

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

11 participants

@tiran @bedevere-bot @merwok @pradyunsg @nascheme @miss-islington @vstinner @ayappanec @hroncok @haampie @zware
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.