GeezerVison is showing github.com — gh-92584: Remove the distutils package by vstinner · Pull Request #99061 · python/cpython
Conversation
vstinner
commented
Nov 3, 2022
•
edited by bedevere-bot
Loading
Uh oh!
There was an error while loading. Please reload this page.
Remove the distutils package. It was deprecated in Python 3.10 by PEP 632 "Deprecate distutils module". For projects still using distutils and cannot be updated to something else, the setuptools project can be installed: it still provides distutils.
- Remove Lib/distutils/ directory
- Remove Doc/installing/ directory
- Remove test_distutils
- Remove references to distutils
- Skip test_check_c_globals and test_peg_generator since they use distutils
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
zware, FFY00, arhadthedev, and SpecLad reacted with hooray emoji
|
There are still many references to distutils. I prefer to remove/update them in separated PRs to keep this PR as small as possible.
Doc/distributing/index.rst:7::Email: [email protected]
Doc/distributing/index.rst:44:* ``distutils`` is the original build and distribution system first added
Doc/distributing/index.rst:45: to the Python standard library in 1998. While direct use of ``distutils``
Doc/distributing/index.rst:51:* `setuptools`_ is a (largely) drop-in replacement for ``distutils`` first
Doc/distributing/index.rst:53: ``distutils`` tools was the ability to declare dependencies on other
Doc/distributing/index.rst:55: alternative to ``distutils`` that offers consistent support for more
Doc/distributing/index.rst:58: command to ``distutils``/`setuptools`_. This produces a cross platform
Doc/extending/building.rst:53:Python 3.12 and newer no longer come with distutils. Please refer to the
Doc/extending/newtypes_tutorial.rst:203: from distutils.core import setup, Extension
Doc/extending/newtypes_tutorial.rst:223: While this documentation showcases the standard :mod:`distutils` module
Doc/extending/newtypes_tutorial.rst:530: from distutils.core import setup, Extension
Doc/extending/windows.rst:17:Module authors are encouraged to use the distutils approach for building
Doc/includes/setup.py:1:from distutils.core import setup, Extension
Doc/includes/test.py:192:from distutils.util import get_platform
Doc/install/index.rst:15: The entire ``distutils`` package has been deprecated and will be
Doc/install/index.rst:18: :ref:`What's New <distutils-deprecated>` entry for more information.
Doc/install/index.rst:47:In Python 2.0, the ``distutils`` API was first added to the standard library.
Doc/install/index.rst:56:using ``distutils`` directly.
Doc/install/index.rst:634: that change the installation path will be ignored from all distutils configuration
Doc/install/index.rst:748:| system | :file:`{prefix}/lib/python{ver}/distutils/distutils.cfg` | \(1) |
Doc/install/index.rst:750:| personal | :file:`$HOME/.pydistutils.cfg` | \(2) |
Doc/install/index.rst:760:| system | :file:`{prefix}\\Lib\\distutils\\distutils.cfg` | \(4) |
Doc/install/index.rst:762:| personal | :file:`%HOME%\\pydistutils.cfg` | \(5) |
Doc/install/index.rst:776: :file:`{prefix}/lib/python1.5/site-packages/distutils`, so the system
Doc/install/index.rst:791: :file:`C:\\Python\\Lib\\distutils\\distutils.cfg`. Under Python 1.5.2, the
Doc/install/index.rst:794: :file:`C:\\Program Files\\Python\\distutils\\distutils.cfg` in a standard Python
Doc/library/importlib.metadata.rst:90:``console_scripts``, ``distutils.commands`` and others. Each group contains a
Doc/library/importlib.metadata.rst:130: ['console_scripts', 'distutils.commands', 'distutils.setup_keywords', 'egg_info.writers', 'setuptools.installation']
Doc/library/sysconfig.rst:72:Every new component that is installed using :mod:`distutils` or a
Doc/library/zipapp.rst:351:command line tools, or you can take advantage of the fact that distutils
Doc/library/zipapp.rst:354: >>> from distutils.ccompiler import new_compiler
Doc/library/zipapp.rst:355: >>> import distutils.sysconfig
Doc/library/zipapp.rst:364: >>> cc.add_include_dir(distutils.sysconfig.get_python_inc())
Doc/using/configure.rst:681:the :mod:`distutils` module.
Doc/using/configure.rst:754: distutils :envvar:`CFLAGS` once Python is installed (:issue:`21121`).
Doc/using/configure.rst:877: the distutils :envvar:`LDFLAGS` once Python is installed (:issue:`35257`).
Doc/using/mac.rst:130:* Packages can be installed via the standard Python distutils mode (``python
Doc/using/windows.rst:1208:`cx_Freeze <https://cx-freeze.readthedocs.io/en/latest/>`_ is a ``distutils``
Lib/_aix_support.py:9: # _aix_support is used in distutils by setup.py to build C extensions,
Lib/_osx_support.py:91: # the build of Python itself (distutils is used to build standard library
Lib/_osx_support.py:396: # see also distutils.sysconfig
Lib/_osx_support.py:452: distutils.sysconfig.get_config_vars() is first
Lib/_osx_support.py:458: requested (in distutils.sysconfig.customize_compiler).
Lib/_osx_support.py:460: Currently called from distutils.sysconfig
Lib/_osx_support.py:484: in distutils.sysconfig.customize_compiler.
Lib/_osx_support.py:501: # called from get_platform() in sysconfig and distutils.util
Lib/_osx_support.py:504: # distutils' perspective is what MACOSX_DEPLOYMENT_TARGET is set
Lib/_osx_support.py:519: # Otherwise, distutils may consider this a cross-compiling
Lib/test/test__osx_support.py:316: # indirectly by test_sysconfig and test_distutils
Lib/site.py:84:# for distutils.commands.install
Lib/sysconfig.py:431: # This is disabled when called from distutils.sysconfig
Lib/test/support/__init__.py:1853: # TODO (PEP 632): alternate check without using distutils
Lib/test/support/__init__.py:1854: from distutils import ccompiler, sysconfig, spawn, errors
Lib/test/test_check_c_globals.py:6:# TODO: gh-92584: c-analyzer uses distutils which was removed in Python 3.12
Lib/test/test_check_c_globals.py:7:raise unittest.SkipTest("distutils has been removed in Python 3.12")
Lib/test/test_importlib/test_windows.py:21: # Port of distutils.util.get_platform().
Lib/test/test_peg_generator/__init__.py:6:# TODO: gh-92584: peg_generator uses distutils which was removed in Python 3.12
Lib/test/test_peg_generator/__init__.py:7:raise unittest.SkipTest("distutils has been removed in Python 3.12")
Mac/BuildScript/build-installer.py:1332: # distutils.tests.test_sysconfig.SysconfigTestCase tests that
Mac/BuildScript/build-installer.py:1334: # distutils.sysconfig is exactly the same for both CFLAGS and
Mac/PythonLauncher/Makefile.in:18:# by distutils
Makefile.pre.in:86:# Use it when a compiler flag should _not_ be part of the distutils CFLAGS
Makefile.pre.in:90:# Use it when a linker flag should _not_ be part of the distutils LDFLAGS
Makefile.pre.in:187:# by distutils. The export statement is needed to ensure that the
PC/pyconfig.h:276: generally taken care of by distutils.) */
Tools/c-analyzer/c_parser/preprocessor/__init__.py:228: # matching distutils.ccompiler.compiler_class:
Tools/c-analyzer/c_parser/preprocessor/common.py:2:import distutils.ccompiler
Tools/c-analyzer/c_parser/preprocessor/common.py:80: compiler = distutils.ccompiler.new_compiler(
Tools/peg_generator/pegen/build.py:46: of distutils (this is useful in case you want to use a temporary directory).
Tools/peg_generator/pegen/build.py:52: import distutils.log
Tools/peg_generator/pegen/build.py:53: from distutils.core import Distribution, Extension
Tools/peg_generator/pegen/build.py:54: from distutils.tests.support import fixup_build_ext # type: ignore
Tools/peg_generator/pegen/build.py:56: from distutils.ccompiler import new_compiler
Tools/peg_generator/pegen/build.py:57: from distutils.dep_util import newer_group
Tools/peg_generator/pegen/build.py:58: from distutils.sysconfig import customize_compiler
Tools/peg_generator/pegen/build.py:61: distutils.log.set_threshold(distutils.log.DEBUG)
configure.ac:727:# it may influence the way we can build extensions, so distutils
configure.ac:905: By default, distutils will build C++ extension modules with "$CXX".
configure.ac:1375:# distutils.unixccompiler to know if it should add --enable-new-dtags
configure.ac:1732: # compile working code using it and both test_distutils and test_gdb are
configure.ac:2450: # component of the string returned by distutils.get_platform().
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
The doc build fails when I just removed Doc/installing/index.rst. I reverted this change.
Warning: /home/runner/work/cpython/cpython/Doc/distributing/index.rst:19: WARNING: undefined label: installing-index
Warning: /home/runner/work/cpython/cpython/Doc/install/index.rst:23: WARNING: undefined label: installing-index
Warning: /home/runner/work/cpython/cpython/Doc/install/index.rst:58: WARNING: undefined label: installing-index
Warning: /home/runner/work/cpython/cpython/Doc/library/ensurepip.rst:34: WARNING: undefined label: installing-index
Warning: /home/runner/work/cpython/cpython/Doc/tutorial/venv.rst:105: WARNING: undefined label: installing-index
Warning: /home/runner/work/cpython/cpython/Doc/tutorial/venv.rst:207: WARNING: undefined label: installing-index
Warning: /home/runner/work/cpython/cpython/Doc/tutorial/whatnow.rst:24: WARNING: undefined label: installing-index
Warning: /home/runner/work/cpython/cpython/Doc/whatsnew/2.7.rst:2690: WARNING: undefined label: installing-index
Warning: /home/runner/work/cpython/cpython/Doc/whatsnew/3.4.rst:240: WARNING: undefined label: installing-index
It should be removed in a separated PR.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Optional Windows (x86) CI job failed, I re-run the job ("2 re-run tests: test_asyncio test_threading"). It's just a random error, unrelated to this PR.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
🤖 New build scheduled with the buildbot fleet by @vstinner for commit 5334545 🤖
If you want to schedule another build, you need to add the ":hammer: test-with-buildbots" label again.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
My colleague @hroncok is already preparing Python packages in Fedora for this removal: Help needed triaging build failures without distutils. 250 packages are impacted: a build dependency to setuptools should be added to get distutils.
A handful of packages also falls to build with setutpools, because the distutils module from setuptools is not backward compatible with distutils.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM (though there's two more lines that can be removed in PC/layout/support).
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
A handful of packages also falls to build with setutpools, because the distutils module from setuptools is not backward compatible with distutils.
That's... very scary :-(
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
The naming is confusing, but Doc/installing, which is stated to be removed in the OP (but later the docs changes were decided for a separate PR) is not the distutils docs, its the modern(ish), thin-shim guide on installing things with Python, and only mentions distutils one place, in the context of describing it as a deprecated legacy module. So that one sentence should be updated, but the rest is unrelated to distutils and its removal, if desired, is seems out of scope here.
Meanwhile, Doc/install is the actual distutils documentation, which should presumably be removed (it can always be accessed in older Python versions) but is not listed above or removed by this PR.
@vstinner Would you like me to help take care of the docs changes/removals in a followup PR, since I'm fairly familiar with them? They're also somewhat covered by a separate issue, #94249
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Remove the distutils package. It was deprecated in Python 3.10 by PEP
632 "Deprecate distutils module". For projects still using distutils
and cannot be updated to something else, the setuptools project can
be installed: it still provides distutils.
* Remove Lib/distutils/ directory
* Remove test_distutils
* Remove references to distutils
* Skip test_check_c_globals and test_peg_generator since they use
distutils
|
The test suite passed on most buildbots: great! I will not wait for the few remaining buildbots (like Refleaks buildbots, removing pure Python code is unlikely to introduce such leak).
I addressed @zware's review in a second commit. I will merge my PR as soon as the CI pass on it.
|
zware reacted with thumbs up emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
@vstinner Would you like me to help take care of the docs changes/removals in a followup PR
Yes! Help on updating the doc is welcomed. There is a lot of documentation mentioning distutils, and deciding if the doc must be removed or not is not easy.
|
CAM-Gerlach reacted with hooray emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Ok, I removed the distutils packages. Now test_check_c_globals and test_peg_generator should be updated to no longer use distutils. And remaining references to distutils #99061 (comment) should be removed. @CAM-Gerlach is volunteer to handle the Doc/ part.
|
CAM-Gerlach reacted with thumbs up emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Comment on lines
-127
to
-128
|
|
# Clear assorted module caches.
|
|
|
# Don't worry about resetting the cache if the module is not loaded
|
encukou
Nov 7, 2022
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This comment isn't specific to distutils. But it's probably not worth putting it back – it's not that hard to see what the code below does.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|