GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-92584: Remove the distutils package by vstinner · Pull Request #99061 · python/cpython

Conversation

@vstinner

@vstinner vstinner commented Nov 3, 2022 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Remove the distutils package. It was deprecated in Python 3.10 by PEP 632 "Deprecate distutils module". For projects still using distutils and cannot be updated to something else, the setuptools project can be installed: it still provides distutils.

  • Remove Lib/distutils/ directory
  • Remove Doc/installing/ directory
  • Remove test_distutils
  • Remove references to distutils
  • Skip test_check_c_globals and test_peg_generator since they use distutils
  • Issue: PEP 632: Remove the distutils package (Python 3.12) #92584

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

zware, FFY00, arhadthedev, and SpecLad reacted with hooray emoji
All reactions
  • 🎉 4 reactions
@vstinner
vstinner requested review from a team, lysnikolaou and pablogsal as code owners November 3, 2022 14:33
@bedevere-bot bedevere-bot mentioned this pull request Nov 3, 2022
PEP 632: Remove the distutils package (Python 3.12) #92584
Closed
@bedevere-bot bedevere-bot added the awaiting core review label Nov 3, 2022
@vstinner

vstinner commented Nov 3, 2022

Copy link
Copy Markdown
Member Author

There are still many references to distutils. I prefer to remove/update them in separated PRs to keep this PR as small as possible.

Doc/distributing/index.rst:7::Email: [email protected]
Doc/distributing/index.rst:44:* ``distutils`` is the original build and distribution system first added
Doc/distributing/index.rst:45:  to the Python standard library in 1998. While direct use of ``distutils``
Doc/distributing/index.rst:51:* `setuptools`_ is a (largely) drop-in replacement for ``distutils`` first
Doc/distributing/index.rst:53:  ``distutils`` tools was the ability to declare dependencies on other
Doc/distributing/index.rst:55:  alternative to ``distutils`` that offers consistent support for more
Doc/distributing/index.rst:58:  command to ``distutils``/`setuptools`_. This produces a cross platform

Doc/extending/building.rst:53:Python 3.12 and newer no longer come with distutils. Please refer to the
Doc/extending/newtypes_tutorial.rst:203:   from distutils.core import setup, Extension
Doc/extending/newtypes_tutorial.rst:223:   While this documentation showcases the standard :mod:`distutils` module
Doc/extending/newtypes_tutorial.rst:530:   from distutils.core import setup, Extension
Doc/extending/windows.rst:17:Module authors are encouraged to use the distutils approach for building

Doc/includes/setup.py:1:from distutils.core import setup, Extension
Doc/includes/test.py:192:from distutils.util import get_platform

Doc/install/index.rst:15:   The entire ``distutils`` package has been deprecated and will be
Doc/install/index.rst:18:   :ref:`What's New <distutils-deprecated>` entry for more information.
Doc/install/index.rst:47:In Python 2.0, the ``distutils`` API was first added to the standard library.
Doc/install/index.rst:56:using ``distutils`` directly.
Doc/install/index.rst:634:   that change the installation path will be ignored from all distutils configuration
Doc/install/index.rst:748:| system       | :file:`{prefix}/lib/python{ver}/distutils/distutils.cfg` | \(1)  |
Doc/install/index.rst:750:| personal     | :file:`$HOME/.pydistutils.cfg`                           | \(2)  |
Doc/install/index.rst:760:| system       | :file:`{prefix}\\Lib\\distutils\\distutils.cfg` | \(4)  |
Doc/install/index.rst:762:| personal     | :file:`%HOME%\\pydistutils.cfg`                 | \(5)  |
Doc/install/index.rst:776:   :file:`{prefix}/lib/python1.5/site-packages/distutils`, so the system
Doc/install/index.rst:791:   :file:`C:\\Python\\Lib\\distutils\\distutils.cfg`. Under Python 1.5.2, the
Doc/install/index.rst:794:   :file:`C:\\Program Files\\Python\\distutils\\distutils.cfg` in a standard Python

Doc/library/importlib.metadata.rst:90:``console_scripts``, ``distutils.commands`` and others.  Each group contains a
Doc/library/importlib.metadata.rst:130:    ['console_scripts', 'distutils.commands', 'distutils.setup_keywords', 'egg_info.writers', 'setuptools.installation']

Doc/library/sysconfig.rst:72:Every new component that is installed using :mod:`distutils` or a

Doc/library/zipapp.rst:351:command line tools, or you can take advantage of the fact that distutils
Doc/library/zipapp.rst:354:   >>> from distutils.ccompiler import new_compiler
Doc/library/zipapp.rst:355:   >>> import distutils.sysconfig
Doc/library/zipapp.rst:364:   >>>     cc.add_include_dir(distutils.sysconfig.get_python_inc())

Doc/using/configure.rst:681:the :mod:`distutils` module.
Doc/using/configure.rst:754:   distutils :envvar:`CFLAGS` once Python is installed (:issue:`21121`).
Doc/using/configure.rst:877:   the distutils :envvar:`LDFLAGS` once Python is installed (:issue:`35257`).

Doc/using/mac.rst:130:* Packages can be installed via the standard Python distutils mode (``python
Doc/using/windows.rst:1208:`cx_Freeze <https://cx-freeze.readthedocs.io/en/latest/>`_ is a ``distutils``

Lib/_aix_support.py:9:    # _aix_support is used in distutils by setup.py to build C extensions,

Lib/_osx_support.py:91:    # the build of Python itself (distutils is used to build standard library
Lib/_osx_support.py:396:        # see also distutils.sysconfig
Lib/_osx_support.py:452:    distutils.sysconfig.get_config_vars() is first
Lib/_osx_support.py:458:    requested (in distutils.sysconfig.customize_compiler).
Lib/_osx_support.py:460:    Currently called from distutils.sysconfig
Lib/_osx_support.py:484:    in distutils.sysconfig.customize_compiler.
Lib/_osx_support.py:501:    # called from get_platform() in sysconfig and distutils.util
Lib/_osx_support.py:504:    # distutils' perspective is what MACOSX_DEPLOYMENT_TARGET is set
Lib/_osx_support.py:519:        # Otherwise, distutils may consider this a cross-compiling

Lib/test/test__osx_support.py:316:        # indirectly by test_sysconfig and test_distutils

Lib/site.py:84:# for distutils.commands.install

Lib/sysconfig.py:431:                # This is disabled when called from distutils.sysconfig

Lib/test/support/__init__.py:1853:    # TODO (PEP 632): alternate check without using distutils
Lib/test/support/__init__.py:1854:    from distutils import ccompiler, sysconfig, spawn, errors

Lib/test/test_check_c_globals.py:6:# TODO: gh-92584: c-analyzer uses distutils which was removed in Python 3.12
Lib/test/test_check_c_globals.py:7:raise unittest.SkipTest("distutils has been removed in Python 3.12")

Lib/test/test_importlib/test_windows.py:21:    # Port of distutils.util.get_platform().

Lib/test/test_peg_generator/__init__.py:6:# TODO: gh-92584: peg_generator uses distutils which was removed in Python 3.12
Lib/test/test_peg_generator/__init__.py:7:raise unittest.SkipTest("distutils has been removed in Python 3.12")

Mac/BuildScript/build-installer.py:1332:    # distutils.tests.test_sysconfig.SysconfigTestCase tests that
Mac/BuildScript/build-installer.py:1334:    # distutils.sysconfig is exactly the same for both CFLAGS and

Mac/PythonLauncher/Makefile.in:18:# by distutils

Makefile.pre.in:86:# Use it when a compiler flag should _not_ be part of the distutils CFLAGS
Makefile.pre.in:90:# Use it when a linker flag should _not_ be part of the distutils LDFLAGS
Makefile.pre.in:187:# by distutils. The export statement is needed to ensure that the

PC/pyconfig.h:276:                        generally taken care of by distutils.) */

Tools/c-analyzer/c_parser/preprocessor/__init__.py:228:    # matching distutils.ccompiler.compiler_class:
Tools/c-analyzer/c_parser/preprocessor/common.py:2:import distutils.ccompiler
Tools/c-analyzer/c_parser/preprocessor/common.py:80:    compiler = distutils.ccompiler.new_compiler(

Tools/peg_generator/pegen/build.py:46:    of distutils (this is useful in case you want to use a temporary directory).
Tools/peg_generator/pegen/build.py:52:    import distutils.log
Tools/peg_generator/pegen/build.py:53:    from distutils.core import Distribution, Extension
Tools/peg_generator/pegen/build.py:54:    from distutils.tests.support import fixup_build_ext  # type: ignore
Tools/peg_generator/pegen/build.py:56:    from distutils.ccompiler import new_compiler
Tools/peg_generator/pegen/build.py:57:    from distutils.dep_util import newer_group
Tools/peg_generator/pegen/build.py:58:    from distutils.sysconfig import customize_compiler
Tools/peg_generator/pegen/build.py:61:        distutils.log.set_threshold(distutils.log.DEBUG)

configure.ac:727:# it may influence the way we can build extensions, so distutils
configure.ac:905:  By default, distutils will build C++ extension modules with "$CXX".
configure.ac:1375:# distutils.unixccompiler to know if it should add --enable-new-dtags
configure.ac:1732:  # compile working code using it and both test_distutils and test_gdb are
configure.ac:2450:        # component of the string returned by distutils.get_platform().
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Nov 3, 2022

Copy link
Copy Markdown
Member Author

The doc build fails when I just removed Doc/installing/index.rst. I reverted this change.

Warning: /home/runner/work/cpython/cpython/Doc/distributing/index.rst:19: WARNING: undefined label: installing-index
Warning: /home/runner/work/cpython/cpython/Doc/install/index.rst:23: WARNING: undefined label: installing-index
Warning: /home/runner/work/cpython/cpython/Doc/install/index.rst:58: WARNING: undefined label: installing-index
Warning: /home/runner/work/cpython/cpython/Doc/library/ensurepip.rst:34: WARNING: undefined label: installing-index
Warning: /home/runner/work/cpython/cpython/Doc/tutorial/venv.rst:105: WARNING: undefined label: installing-index
Warning: /home/runner/work/cpython/cpython/Doc/tutorial/venv.rst:207: WARNING: undefined label: installing-index
Warning: /home/runner/work/cpython/cpython/Doc/tutorial/whatnow.rst:24: WARNING: undefined label: installing-index
Warning: /home/runner/work/cpython/cpython/Doc/whatsnew/2.7.rst:2690: WARNING: undefined label: installing-index
Warning: /home/runner/work/cpython/cpython/Doc/whatsnew/3.4.rst:240: WARNING: undefined label: installing-index

It should be removed in a separated PR.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Nov 3, 2022

Copy link
Copy Markdown
Member Author

Optional Windows (x86) CI job failed, I re-run the job ("2 re-run tests: test_asyncio test_threading"). It's just a random error, unrelated to this PR.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Nov 3, 2022

Copy link
Copy Markdown
Member Author

My colleague @hroncok is already preparing Python packages in Fedora for this removal: Help needed triaging build failures without distutils. 250 packages are impacted: a build dependency to setuptools should be added to get distutils.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner vstinner added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Nov 3, 2022
@bedevere-bot

bedevere-bot commented Nov 3, 2022

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @vstinner for commit 5334545 🤖

If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Nov 3, 2022
@hroncok

hroncok commented Nov 3, 2022

Copy link
Copy Markdown
Contributor

My colleague @hroncok is already preparing Python packages in Fedora for this removal: Help needed triaging build failures without distutils. 250 packages are impacted: a build dependency to setuptools should be added to get distutils.

A handful of packages also falls to build with setutpools, because the distutils module from setuptools is not backward compatible with distutils.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

zware
zware approved these changes Nov 3, 2022
View reviewed changes

@zware zware left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (though there's two more lines that can be removed in PC/layout/support).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread PC/layout/support/python.props Resolved

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot added awaiting merge and removed awaiting core review labels Nov 3, 2022
@vstinner

vstinner commented Nov 3, 2022

Copy link
Copy Markdown
Member Author

A handful of packages also falls to build with setutpools, because the distutils module from setuptools is not backward compatible with distutils.

That's... very scary :-(

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@CAM-Gerlach

CAM-Gerlach commented Nov 3, 2022

Copy link
Copy Markdown
Member

The naming is confusing, but Doc/installing, which is stated to be removed in the OP (but later the docs changes were decided for a separate PR) is not the distutils docs, its the modern(ish), thin-shim guide on installing things with Python, and only mentions distutils one place, in the context of describing it as a deprecated legacy module. So that one sentence should be updated, but the rest is unrelated to distutils and its removal, if desired, is seems out of scope here.

Meanwhile, Doc/install is the actual distutils documentation, which should presumably be removed (it can always be accessed in older Python versions) but is not listed above or removed by this PR.

@vstinner Would you like me to help take care of the docs changes/removals in a followup PR, since I'm fairly familiar with them? They're also somewhat covered by a separate issue, #94249

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

vstinner added 2 commits November 3, 2022 19:03
@vstinner
gh-92584: Remove the distutils package
a8923b4
Remove the distutils package. It was deprecated in Python 3.10 by PEP
632 "Deprecate distutils module". For projects still using distutils
and cannot be updated to something else, the setuptools project can
be installed: it still provides distutils.

* Remove Lib/distutils/ directory
* Remove test_distutils
* Remove references to distutils
* Skip test_check_c_globals and test_peg_generator since they use
  distutils
@vstinner
Update PC/layout/
eca9891
@vstinner

vstinner commented Nov 3, 2022

Copy link
Copy Markdown
Member Author

The test suite passed on most buildbots: great! I will not wait for the few remaining buildbots (like Refleaks buildbots, removing pure Python code is unlikely to introduce such leak).

I addressed @zware's review in a second commit. I will merge my PR as soon as the CI pass on it.

zware reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Nov 3, 2022

Copy link
Copy Markdown
Member Author

@vstinner Would you like me to help take care of the docs changes/removals in a followup PR

Yes! Help on updating the doc is welcomed. There is a lot of documentation mentioning distutils, and deciding if the doc must be removed or not is not easy.

CAM-Gerlach reacted with hooray emoji
All reactions
  • 🎉 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner
vstinner merged commit 0faa0ba into python:main Nov 3, 2022
@bedevere-bot bedevere-bot removed the awaiting merge label Nov 3, 2022
@vstinner
vstinner deleted the remove_distutils branch November 3, 2022 18:27
@vstinner

vstinner commented Nov 3, 2022

Copy link
Copy Markdown
Member Author

Ok, I removed the distutils packages. Now test_check_c_globals and test_peg_generator should be updated to no longer use distutils. And remaining references to distutils #99061 (comment) should be removed. @CAM-Gerlach is volunteer to handle the Doc/ part.

CAM-Gerlach reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@CAM-Gerlach CAM-Gerlach mentioned this pull request Nov 3, 2022
[Docs] remove distutils legacy ref when setuptools docs are complete #94249
Closed
9 tasks
@rgommers rgommers mentioned this pull request Nov 6, 2022
PEP 632, deprecation of distutils and numpy.distutils numpy/numpy#18588
Closed
encukou
encukou reviewed Nov 7, 2022
View reviewed changes
Comment thread Lib/test/libregrtest/utils.py
Comment on lines -127 to -128
# Clear assorted module caches.
# Don't worry about resetting the cache if the module is not loaded

@encukou encukou Nov 7, 2022 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This comment isn't specific to distutils. But it's probably not worth putting it back – it's not that hard to see what the code below does.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@hugovk hugovk mentioned this pull request Nov 29, 2022
[Distutils] strtobool #99867
Closed
@astrojuanlu astrojuanlu mentioned this pull request Jan 31, 2023
Updated distutils.Version to packaging.Version plotly/plotly.py#3897
Merged
4 of 5 tasks
@fperrin fperrin mentioned this pull request Jul 24, 2023
Fix binary type representation robshakir/pyangbind#297
Merged
@gaborbernat gaborbernat mentioned this pull request Jun 3, 2026
_osx_support._default_sysroot is dead in CPython but used downstream with no public replacement #150881
Open
@coderabbitai coderabbitai Bot mentioned this pull request Jun 11, 2026
integrate VSA (Video Sparse Attention) into block-sparse API flashinfer-ai/flashinfer#3250
Merged
5 tasks done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@encukou encukou encukou left review comments
@zware zware zware approved these changes
@pablogsal pablogsal Awaiting requested review from pablogsal pablogsal is a code owner
@lysnikolaou lysnikolaou Awaiting requested review from lysnikolaou lysnikolaou is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

6 participants

@vstinner @bedevere-bot @hroncok @CAM-Gerlach @encukou @zware
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.