GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-112026: Restore removed private C API by vstinner · Pull Request #112115 · python/cpython

Conversation

@vstinner

@vstinner vstinner commented Nov 15, 2023 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Restore removed private C API functions, macros and structures which have no simple replacement for now:

  • _PyDict_GetItem_KnownHash()
  • _PyDict_NewPresized()
  • _PyHASH_BITS
  • _PyHASH_IMAG
  • _PyHASH_INF
  • _PyHASH_MODULUS
  • _PyHASH_MULTIPLIER
  • _PyLong_Copy()
  • _PyLong_FromDigits()
  • _PyLong_New()
  • _PyLong_Sign()
  • _PyObject_CallMethodId()
  • _PyObject_CallMethodNoArgs()
  • _PyObject_CallMethodOneArg()
  • _PyObject_CallOneArg()
  • _PyObject_EXTRA_INIT
  • _PyObject_FastCallDict()
  • _PyObject_GetAttrId()
  • _PyObject_Vectorcall()
  • _PyObject_VectorcallMethod()
  • _PyStack_AsDict()
  • _PyThread_CurrentFrames()
  • _PyUnicodeWriter structure
  • _PyUnicodeWriter_Dealloc()
  • _PyUnicodeWriter_Finish()
  • _PyUnicodeWriter_Init()
  • _PyUnicodeWriter_Prepare()
  • _PyUnicodeWriter_PrepareKind()
  • _PyUnicodeWriter_WriteASCIIString()
  • _PyUnicodeWriter_WriteChar()
  • _PyUnicodeWriter_WriteLatin1String()
  • _PyUnicodeWriter_WriteStr()
  • _PyUnicodeWriter_WriteSubstring()
  • _PyUnicode_AsString()
  • _PyUnicode_FromId()
  • _PyVectorcall_Function()
  • _Py_IDENTIFIER()
  • _Py_c_abs()
  • _Py_c_diff()
  • _Py_c_neg()
  • _Py_c_pow()
  • _Py_c_prod()
  • _Py_c_quot()
  • _Py_c_sum()
  • _Py_static_string()
  • _Py_static_string_init()
  • Issue: [C API] Revert of private functions removed in Python 3.13 causing most problems #112026

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@vstinner
vstinner requested review from markshannon, methane and tiran as code owners November 15, 2023 15:17
@bedevere-app bedevere-app Bot added the awaiting core review label Nov 15, 2023
@bedevere-app bedevere-app Bot mentioned this pull request Nov 15, 2023
[C API] Revert of private functions removed in Python 3.13 causing most problems #112026
Closed
@vstinner

vstinner commented Nov 15, 2023

Copy link
Copy Markdown
Member Author

Last weeks, other functions removed in Python 3.13 were reported as causing problems.

The following functions were already restored since Python 3.13 alpha1:

  • _PyLong_AsByteArray()
  • _PyLong_FromByteArray()

Not restored, use new public functions instead:

  • _PyObject_VisitManagedDict(): use PyObject_VisitManagedDict()
  • _PyObject_ClearManagedDict(): use PyObject_ClearManagedDict()
  • _PyThreadState_UncheckedGet(): use PyThreadState_GetUnchecked()
  • _PyList_Extend(): use PyList_Extend(), different API
  • _PyDict_Pop(): use PyDict_Pop(), different API

Not restored:

  • _PyDict_ContainsId()
  • _PyDict_DelItemId()
  • _PyDict_GetItemIdWithError()
  • _PyDict_SetItemId()
  • _PyEval_GetBuiltinId()
  • _PyLong_NumBits()
  • _PyObject_CallMethodIdNoArgs()
  • _PyObject_CallMethodIdObjArgs()
  • _PyObject_CallMethodIdOneArg()
  • _PyObject_LookupSpecialId()
  • _PyObject_SetAttrId()
  • _PyObject_VectorcallMethodId()
  • _PyTime_AsSecondsDouble()
  • _PyTime_FromTimeval()
  • _PyTime_GetSystemClock()
  • _PyTime_t
  • _PyType_LookupId()
  • _PyUnicode_EqualToASCIIId()

Pending PR:

  • _Py_HashDouble(): gh-111545: Add PyHash_Double() function #112095
  • _Py_HashPointer(): gh-111545: Add Py_HashPointer() function #112096

Removed:

  • _PyErr_WriteUnraisable()

Not restored, deprecated:

  • 2to3 module removal
  • <ctype.h>
  • <unistd.h>
  • Logger.warn()
  • PyCFunction_Call()
  • PyEval_AcquireLock()
  • PyEval_CallObject()
  • PyEval_InitThreads()
  • PyEval_ReleaseLock()
  • PyObject_AsCharBuffer()
  • PyObject_AsReadBuffer()
  • PyObject_AsWriteBuffer()
  • PySys_SetArgv()
  • PySys_SetArgvEx()
  • PySys_SetPath()
  • PyUnicode_AS_DATA
  • PyUnicode_AS_UNICODE()
  • PyUnicode_GetSize()
  • Py_SetProgramName()
  • Py_TRASHCAN_SAFE_BEGIN and Py_TRASHCAN_SAFE_END
  • importlib.resources removed functions
  • locale.resetlocale()
  • typing.re
  • unittest-related removals
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner

vstinner commented Nov 15, 2023

Copy link
Copy Markdown
Member Author

With this change, I can successfully install cffi 1.16.0 and numpy 1.26.2.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner

vstinner commented Nov 15, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

With this change, I can successfully install cffi 1.16.0 and numpy 1.26.2.

Oh, on Fedora 39, GCC only emits a warning when using an undefined function, and so the build was successful, but numpy calls a function which doesn't exist anymore! My script to test numpy:

set -e -x
wget https://files.pythonhosted.org/packages/dd/2b/205ddff2314d4eea852e31d53b8e55eb3f32b292efc3dd86bd827ab9019d/numpy-1.26.2.tar.gz
rm -rf numpy-1.26.2
tar -xf numpy-1.26.2.tar.gz
cd numpy-1.26.2/
/opt/py313/bin/python3.13 -m venv env
export CFLAGS='-Werror=implicit-function-declaration'
export LANG=
env/bin/python -m pip install . -v

In fact, 3 other changes are needed by numpy (UPDATE: done):

  • Restore #include <ctype.h>: gh-112026: Add again <unistd.h> include in Python.h #112046
  • Restore removed _PyDict_GetItemStringWithError() function.
  • Restore _Py_HashDouble() function.

With these additional changes, I can successfully build numpy with -Werror=implicit-function-declaration compiler flag.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner
pythongh-112026: Restore removed private C API
84c0452 
Restore removed private C API functions, macros and structures which
have no simple replacement for now:

* _PyDict_GetItem_KnownHash()
* _PyDict_NewPresized()
* _PyHASH_BITS
* _PyHASH_IMAG
* _PyHASH_INF
* _PyHASH_MODULUS
* _PyHASH_MULTIPLIER
* _PyLong_Copy()
* _PyLong_FromDigits()
* _PyLong_New()
* _PyLong_Sign()
* _PyObject_CallMethodId()
* _PyObject_CallMethodNoArgs()
* _PyObject_CallMethodOneArg()
* _PyObject_CallOneArg()
* _PyObject_EXTRA_INIT
* _PyObject_FastCallDict()
* _PyObject_GetAttrId()
* _PyObject_Vectorcall()
* _PyObject_VectorcallMethod()
* _PyStack_AsDict()
* _PyThread_CurrentFrames()
* _PyUnicodeWriter structure
* _PyUnicodeWriter_Dealloc()
* _PyUnicodeWriter_Finish()
* _PyUnicodeWriter_Init()
* _PyUnicodeWriter_Prepare()
* _PyUnicodeWriter_PrepareKind()
* _PyUnicodeWriter_WriteASCIIString()
* _PyUnicodeWriter_WriteChar()
* _PyUnicodeWriter_WriteLatin1String()
* _PyUnicodeWriter_WriteStr()
* _PyUnicodeWriter_WriteSubstring()
* _PyUnicode_AsString()
* _PyUnicode_FromId()
* _PyVectorcall_Function()
* _Py_HashDouble()
* _Py_HashPointer()
* _Py_IDENTIFIER()
* _Py_c_abs()
* _Py_c_diff()
* _Py_c_neg()
* _Py_c_pow()
* _Py_c_prod()
* _Py_c_quot()
* _Py_c_sum()
* _Py_static_string()
* _Py_static_string_init()
@vstinner
vstinner force-pushed the restore_private_apis branch from 034b90f to 84c0452 Compare November 15, 2023 16:08
@vstinner

vstinner commented Nov 15, 2023

Copy link
Copy Markdown
Member Author

Restore _Py_HashDouble() function.

I updated the PR to restore _Py_HashDouble() and _Py_HashPointer() functions.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner
vstinner enabled auto-merge (squash) November 15, 2023 16:35
@vstinner
vstinner merged commit 5846924 into python:main Nov 15, 2023
@vstinner
vstinner deleted the restore_private_apis branch November 15, 2023 16:38
@bedevere-app bedevere-app Bot removed the awaiting core review label Nov 15, 2023
This was referenced Nov 15, 2023
_Py_c_sum, _Py_c_diff, etc - were removed from Python.h #112019
Closed
Consider restoring _PyLong_New() function as public #111415
Closed
Consider restoring _PyHASH_INF/BITS/MODULUS/IMAG as public #111389
Closed
Is it OK to remove _PyObject_Vectorcall? python/steering-council#212
Closed
skirpichev added a commit to skirpichev/gmpy that referenced this pull request Jan 30, 2024
@skirpichev
Drop workaround for CPython 3.13, see python/cpython#112115
bf9a4c1
skirpichev added a commit to skirpichev/gmpy that referenced this pull request Jan 30, 2024
@skirpichev
Revert _PyLong_New() workaround, see python/cpython#112115
336da3e
skirpichev added a commit to skirpichev/gmpy that referenced this pull request Jan 30, 2024
@skirpichev
Drop workaround for CPython 3.13, see python/cpython#112115
4b5ef78
@skirpichev skirpichev mentioned this pull request Feb 3, 2024
Misc fixes gmpy2/gmpy2#466
Merged
aisk pushed a commit to aisk/cpython that referenced this pull request Feb 11, 2024
@vstinner @aisk
pythongh-112026: Restore removed private C API (python#112115)
cd920a5 
Restore removed private C API functions, macros and structures which
have no simple replacement for now:

* _PyDict_GetItem_KnownHash()
* _PyDict_NewPresized()
* _PyHASH_BITS
* _PyHASH_IMAG
* _PyHASH_INF
* _PyHASH_MODULUS
* _PyHASH_MULTIPLIER
* _PyLong_Copy()
* _PyLong_FromDigits()
* _PyLong_New()
* _PyLong_Sign()
* _PyObject_CallMethodId()
* _PyObject_CallMethodNoArgs()
* _PyObject_CallMethodOneArg()
* _PyObject_CallOneArg()
* _PyObject_EXTRA_INIT
* _PyObject_FastCallDict()
* _PyObject_GetAttrId()
* _PyObject_Vectorcall()
* _PyObject_VectorcallMethod()
* _PyStack_AsDict()
* _PyThread_CurrentFrames()
* _PyUnicodeWriter structure
* _PyUnicodeWriter_Dealloc()
* _PyUnicodeWriter_Finish()
* _PyUnicodeWriter_Init()
* _PyUnicodeWriter_Prepare()
* _PyUnicodeWriter_PrepareKind()
* _PyUnicodeWriter_WriteASCIIString()
* _PyUnicodeWriter_WriteChar()
* _PyUnicodeWriter_WriteLatin1String()
* _PyUnicodeWriter_WriteStr()
* _PyUnicodeWriter_WriteSubstring()
* _PyUnicode_AsString()
* _PyUnicode_FromId()
* _PyVectorcall_Function()
* _Py_HashDouble()
* _Py_HashPointer()
* _Py_IDENTIFIER()
* _Py_c_abs()
* _Py_c_diff()
* _Py_c_neg()
* _Py_c_pow()
* _Py_c_prod()
* _Py_c_quot()
* _Py_c_sum()
* _Py_static_string()
* _Py_static_string_init()
@lysnikolaou lysnikolaou mentioned this pull request May 9, 2024
Avoid thread unsafe borrowed references under free-threading cython/cython#6178
Merged
@vstinner vstinner mentioned this pull request May 20, 2024
gh-111389: Add PyHASH_MULTIPLIER constant #119214
Merged
Glyphack pushed a commit to Glyphack/cpython that referenced this pull request Sep 2, 2024
@vstinner @Glyphack
pythongh-112026: Restore removed private C API (python#112115)
2af790f 
Restore removed private C API functions, macros and structures which
have no simple replacement for now:

* _PyDict_GetItem_KnownHash()
* _PyDict_NewPresized()
* _PyHASH_BITS
* _PyHASH_IMAG
* _PyHASH_INF
* _PyHASH_MODULUS
* _PyHASH_MULTIPLIER
* _PyLong_Copy()
* _PyLong_FromDigits()
* _PyLong_New()
* _PyLong_Sign()
* _PyObject_CallMethodId()
* _PyObject_CallMethodNoArgs()
* _PyObject_CallMethodOneArg()
* _PyObject_CallOneArg()
* _PyObject_EXTRA_INIT
* _PyObject_FastCallDict()
* _PyObject_GetAttrId()
* _PyObject_Vectorcall()
* _PyObject_VectorcallMethod()
* _PyStack_AsDict()
* _PyThread_CurrentFrames()
* _PyUnicodeWriter structure
* _PyUnicodeWriter_Dealloc()
* _PyUnicodeWriter_Finish()
* _PyUnicodeWriter_Init()
* _PyUnicodeWriter_Prepare()
* _PyUnicodeWriter_PrepareKind()
* _PyUnicodeWriter_WriteASCIIString()
* _PyUnicodeWriter_WriteChar()
* _PyUnicodeWriter_WriteLatin1String()
* _PyUnicodeWriter_WriteStr()
* _PyUnicodeWriter_WriteSubstring()
* _PyUnicode_AsString()
* _PyUnicode_FromId()
* _PyVectorcall_Function()
* _Py_HashDouble()
* _Py_HashPointer()
* _Py_IDENTIFIER()
* _Py_c_abs()
* _Py_c_diff()
* _Py_c_neg()
* _Py_c_pow()
* _Py_c_prod()
* _Py_c_quot()
* _Py_c_sum()
* _Py_static_string()
* _Py_static_string_init()
@MSeifert04 MSeifert04 mentioned this pull request Sep 24, 2024
Version 0.13: Python 3.13 compatibility and minor updates MSeifert04/iteration_utilities#346
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tiran tiran Awaiting requested review from tiran
@methane methane Awaiting requested review from methane methane is a code owner
@markshannon markshannon Awaiting requested review from markshannon markshannon is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

1 participant

@vstinner
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.