GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Add Py_HOLD_REF() macro to hold a strong reference to a Python object while executing code · Issue #99481 · python/cpython

Add Py_HOLD_REF() macro to hold a strong reference to a Python object while executing code #99481

New issue
Copy link
New issue
Copy link
Closed
Closed
Add Py_HOLD_REF() macro to hold a strong reference to a Python object while executing code#99481
Copy link
Labels
type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

Description

@vstinner
vstinner
opened on Nov 14, 2022
Issue body actions

Reference counting in Python is complicated. It's never obvious if we are holding a strong reference or a borrowed reference to a Python object.

If we have a borrowed reference to a Python object, to make sure that it cannot be deleted while we execute arbitrary Python code, it's needed to creating a temporary strong reference by increasing temporarily its reference counter. Example from Modules/_abc.c:

    subclasses = PyObject_CallMethod(self, "__subclasses__", NULL);
    (...)
    PyObject *scls = PyList_GET_ITEM(subclasses, pos);

    Py_INCREF(scls);  // create a temporary strong reference
    int r = PyObject_IsSubclass(subclass, scls);
    Py_DECREF(scls);  // delete the temporary strong reference

The scls variable is assigned to a borrowed reference to a list item. The list is the result of the __subclasses__() method. The code creates a temporarily strong reference to call PyObject_IsSubclass().

IMO in terms of semantics, such code is hard to understand if read aloud in English, since INCREF and DECREF modify the object in-place. Py_INCREF() and Py_DECREF() are like the low-level implementation, I would prefer to have an abstraction on top of it.

That's also why I added Py_NewRef() in Python 3.10: Py_NewRef() semantics is simpler: it creates a new strong reference. In terms of semantics, it doesn't modify the object in-place, even if it's possible to write something like var = Py_NewRef(var) (I discourage to write such code). See issue #99300 for the usage of the Py_NewRef() function.

It would be nice to have a macro to clarify the scope of the temporary strong reference rather than having to INCREF/DECREF temporarily. Something like:

#define Py_HOLD_REF(var, code) \
    Py_INCREF(var); \
    code; \
    Py_DECREF(var)

Example of usage:

Py_HOLD_REF(scls,
    int r = PyObject_IsSubclass(subclass, scls);
);

Example holding two (different) strong references:

Py_HOLD_REF(keys,
    Py_HOLD_REF(values,
        PyObject *res = PyTuple_Pack(2, keys, values);
    )    // note: there is no ";" here
);

Alternative compact syntax:

Py_HOLD_REF(keys, Py_HOLD_REF(values,
    PyObject *res = PyTuple_Pack(2, keys, values);
));

Obviously, the macro name is open for bikeshedding :-)

Activity

  1. vstinner
    added
    type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
    on Nov 14, 2022
  2. vstinner commented on Nov 14, 2022

    @vstinner
    vstinner
    on Nov 14, 2022
    MemberAuthor
    More actions

    See also issue #98724 and comments on my PR #99100 which discuss Py_CLEAR(), Py_SETREF() and Py_XSETREF().

  3. vstinner commented on Dec 13, 2022

    @vstinner
    vstinner
    on Dec 13, 2022
    MemberAuthor
    More actions

    I'm not sure that such macro makes the code easier to understand or to maintain. I close the issue.

  4. vstinner
    closed this as completedon Dec 13, 2022
  5. vstinner commented on Dec 13, 2022

    @vstinner
    vstinner
    on Dec 13, 2022
    MemberAuthor
    More actions

    In programming languages like C++, it would be possible to create such API by relying on destructors. There are CPython C++ binding which uses such API to hold the GIL for example: the GIL is released at the end of the scope of a variable which "holds the GIL".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      You are visitor 01402503 — GeezerVison does not store the pages you look up.
      Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.