GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-85858: Remove PyUnicode_InternImmortal() function by vstinner · Pull Request #92579 · python/cpython

gh-85858: Remove PyUnicode_InternImmortal() function #92579

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vstinner merged 1 commit into python:main from vstinner:remove_immortal
May 13, 2022

Uh oh!

There was an error while loading. Please reload this page.
Merged

gh-85858: Remove PyUnicode_InternImmortal() function #92579

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
b3f00dc
gh-85858: Remove PyUnicode_InternImmortal() function
vstinner May 9, 2022
File filter
Filter by extension
Clear filters
Conversations
Failed to load comments.
Loading

Uh oh!

There was an error while loading. Please reload this page.
Jump to
Jump to file
Failed to load files.
Loading

Uh oh!

There was an error while loading. Please reload this page.
Diff view
Diff view

There are no files selected for viewing

1 change: 0 additions & 1 deletion 1 Doc/data/stable_abi.dat
Show comments
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Oops, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
4 changes: 4 additions & 0 deletions 4 Doc/whatsnew/3.12.rst
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -174,3 +174,7 @@ Removed
* :c:func:`PyUnicode_GET_SIZE`
* :c:func:`PyUnicode_GetSize`
* :c:func:`PyUnicode_GET_DATA_SIZE`

* Remove the ``PyUnicode_InternImmortal()`` function and the
``SSTATE_INTERNED_IMMORTAL`` macro.
(Contributed by Victor Stinner in :gh:`85858`.)
13 changes: 3 additions & 10 deletions 13 Include/cpython/unicodeobject.h
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,15 +98,9 @@ typedef struct {
Py_ssize_t length; /* Number of code points in the string */
Py_hash_t hash; /* Hash value; -1 if not set */
struct {
/*
SSTATE_NOT_INTERNED (0)
SSTATE_INTERNED_MORTAL (1)
SSTATE_INTERNED_IMMORTAL (2)

If interned != SSTATE_NOT_INTERNED, the two references from the
dictionary to this object are *not* counted in ob_refcnt.
*/
unsigned int interned:2;
Comment thread Outdated

@serhiy-storchaka serhiy-storchaka May 10, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would keep it 2-bit for a time.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@vstinner vstinner May 11, 2022

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PyASCIIObject is not part of the stable ABI. Why do you care about keeping 2 bits for interned?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
/* If interned is set, the two references from the
dictionary to this object are *not* counted in ob_refcnt. */
unsigned int interned:1;
/* Character size:

- PyUnicode_1BYTE_KIND (1):
Expand Down Expand Up @@ -189,7 +183,6 @@ PyAPI_FUNC(int) _PyUnicode_CheckConsistency(
/* Interning state. */
#define SSTATE_NOT_INTERNED 0
#define SSTATE_INTERNED_MORTAL 1
#define SSTATE_INTERNED_IMMORTAL 2

/* Use only if you know it's a string */
static inline unsigned int PyUnicode_CHECK_INTERNED(PyObject *op) {
Expand Down
4 changes: 0 additions & 4 deletions 4 Include/unicodeobject.h
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -256,10 +256,6 @@ PyAPI_FUNC(PyObject *) PyUnicode_InternFromString(
const char *u /* UTF-8 encoded string */
);

// PyUnicode_InternImmortal() is deprecated since Python 3.10
// and will be removed in Python 3.12. Use PyUnicode_InternInPlace() instead.
Py_DEPRECATED(3.10) PyAPI_FUNC(void) PyUnicode_InternImmortal(PyObject **);
Comment thread Outdated

@serhiy-storchaka serhiy-storchaka May 10, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is not 2 versions too short period for removing from the public C API? I think we should be more conservative in such cases and prolong the deprecation to at least 4 years.

I would keep it as an alias of PyUnicode_InternInPlace which may print a deprecation warning.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@methane methane May 10, 2022

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are no use from top4000 packages.
https://github.com/hpyproject/top4000-pypi-packages/search?q=PyUnicode_InternImmortal

We can remove it in early alpha and resurrect it if some packages are broken by this.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@vstinner vstinner May 11, 2022

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is not 2 versions too short period for removing from the public C API?

I followed PEP 387 process.

I think we should be more conservative in such cases and prolong the deprecation to at least 4 years.

Why do you care of this function in specific? Are you aware of projects using it?

There are no use from top4000 packages.

Same when I looked in December 2021:
#85858 (comment)

History of this function: #85858 (comment)

We can remove it in early alpha and resurrect it if some packages are broken by this.

Sure. I did that multiple times and I'm fine with reverting if it causes too many issues.

Removing the function early in 3.12 dev cycle gives more time to users to detect if their project is affected, to give them more time to decide how to update their code, or to ask to revert the change.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

/* --- wchar_t support for platforms which support it --------------------- */

#ifdef HAVE_WCHAR_H
Expand Down
2 changes: 2 additions & 0 deletions 2 Misc/NEWS.d/next/C API/2022-05-09-23-16-38.gh-issue-85858.VIcNDL.rst
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Remove the ``PyUnicode_InternImmortal()`` function and the
``SSTATE_INTERNED_IMMORTAL`` macro. Patch by Victor Stinner.
1 change: 1 addition & 0 deletions 1 Misc/stable_abi.toml
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1563,6 +1563,7 @@
added = '3.2'
[function.PyUnicode_InternImmortal]
added = '3.2'
abi_only = true
[function.PyUnicode_InternInPlace]
added = '3.2'
[data.PyUnicode_Type]
Expand Down
69 changes: 17 additions & 52 deletions 69 Objects/unicodeobject.c
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1516,13 +1516,8 @@ unicode_dealloc(PyObject *unicode)
}
#endif

switch (PyUnicode_CHECK_INTERNED(unicode)) {
case SSTATE_NOT_INTERNED:
break;

case SSTATE_INTERNED_MORTAL:
{
#ifdef INTERNED_STRINGS
if (PyUnicode_CHECK_INTERNED(unicode)) {
/* Revive the dead object temporarily. PyDict_DelItem() removes two
references (key and value) which were ignored by
PyUnicode_InternInPlace(). Use refcnt=3 rather than refcnt=2
Expand All @@ -1536,17 +1531,8 @@ unicode_dealloc(PyObject *unicode)
}
assert(Py_REFCNT(unicode) == 1);
Py_SET_REFCNT(unicode, 0);
#endif
break;
}

case SSTATE_INTERNED_IMMORTAL:
_PyObject_ASSERT_FAILED_MSG(unicode, "Immortal interned string died");
break;

default:
Py_UNREACHABLE();
}
#endif

if (_PyUnicode_HAS_UTF8_MEMORY(unicode)) {
PyObject_Free(_PyUnicode_UTF8(unicode));
Expand Down Expand Up @@ -14674,31 +14660,22 @@ PyUnicode_InternInPlace(PyObject **p)
refcnt. unicode_dealloc() and _PyUnicode_ClearInterned() take care of
this. */
Py_SET_REFCNT(s, Py_REFCNT(s) - 2);
_PyUnicode_STATE(s).interned = SSTATE_INTERNED_MORTAL;
_PyUnicode_STATE(s).interned = 1;
#else
// PyDict expects that interned strings have their hash
// (PyASCIIObject.hash) already computed.
(void)unicode_hash(s);
#endif
}

// Function kept for the stable ABI.
PyAPI_FUNC(void) PyUnicode_InternImmortal(PyObject **);
void
PyUnicode_InternImmortal(PyObject **p)
{
if (PyErr_WarnEx(PyExc_DeprecationWarning,
"PyUnicode_InternImmortal() is deprecated; "
"use PyUnicode_InternInPlace() instead", 1) < 0)
{
// The function has no return value, the exception cannot
// be reported to the caller, so just log it.
PyErr_WriteUnraisable(NULL);
}

PyUnicode_InternInPlace(p);
if (PyUnicode_CHECK_INTERNED(*p) != SSTATE_INTERNED_IMMORTAL) {
_PyUnicode_STATE(*p).interned = SSTATE_INTERNED_IMMORTAL;
Py_INCREF(*p);
}
// Leak a reference on purpose
Py_INCREF(*p);
}

PyObject *
Expand Down Expand Up @@ -14733,37 +14710,25 @@ _PyUnicode_ClearInterned(PyInterpreterState *interp)
fprintf(stderr, "releasing %zd interned strings\n",
PyDict_GET_SIZE(interned));

Py_ssize_t immortal_size = 0, mortal_size = 0;
Py_ssize_t total_length = 0;
#endif
Py_ssize_t pos = 0;
PyObject *s, *ignored_value;
while (PyDict_Next(interned, &pos, &s, &ignored_value)) {
switch (PyUnicode_CHECK_INTERNED(s)) {
case SSTATE_INTERNED_IMMORTAL:
Py_SET_REFCNT(s, Py_REFCNT(s) + 1);
#ifdef INTERNED_STATS
immortal_size += PyUnicode_GET_LENGTH(s);
#endif
break;
case SSTATE_INTERNED_MORTAL:
// Restore the two references (key and value) ignored
// by PyUnicode_InternInPlace().
Py_SET_REFCNT(s, Py_REFCNT(s) + 2);
assert(PyUnicode_CHECK_INTERNED(s));
// Restore the two references (key and value) ignored
// by PyUnicode_InternInPlace().
Py_SET_REFCNT(s, Py_REFCNT(s) + 2);
#ifdef INTERNED_STATS
mortal_size += PyUnicode_GET_LENGTH(s);
total_length += PyUnicode_GET_LENGTH(s);
#endif
break;
case SSTATE_NOT_INTERNED:
/* fall through */
default:
Py_UNREACHABLE();
}
_PyUnicode_STATE(s).interned = SSTATE_NOT_INTERNED;

_PyUnicode_STATE(s).interned = 0;
}
#ifdef INTERNED_STATS
fprintf(stderr,
"total size of all interned strings: %zd/%zd mortal/immortal\n",
mortal_size, immortal_size);
"total length of all interned strings: %zd characters\n",
total_length);
#endif

PyDict_Clear(interned);
Expand Down
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.