GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-117139: Add header for tagged pointers by Fidget-Spinner · Pull Request #118330 · python/cpython

gh-117139: Add header for tagged pointers #118330

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Fidget-Spinner merged 19 commits into python:main from Fidget-Spinner:tag_headers
Apr 30, 2024

Uh oh!

There was an error while loading. Please reload this page.
Merged

gh-117139: Add header for tagged pointers #118330

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
209ddac
Add header for tagged pointers
Fidget-Spinner Apr 26, 2024
f783bc4
cleanup
Fidget-Spinner Apr 26, 2024
4c52acc
end EOL
Fidget-Spinner Apr 26, 2024
9e0ed42
fix warnings
Fidget-Spinner Apr 27, 2024
4610cac
format
Fidget-Spinner Apr 27, 2024
f0df2bc
make macro names more symmetrical
Fidget-Spinner Apr 28, 2024
35ac0ce
fix warning
Fidget-Spinner Apr 28, 2024
c08a6f5
address rest of review
Fidget-Spinner Apr 30, 2024
9b5f8b2
remove MAX_UNTAG_SCRATCH
Fidget-Spinner Apr 30, 2024
046e308
rename file
Fidget-Spinner Apr 30, 2024
1a7087e
rename header guards
Fidget-Spinner Apr 30, 2024
2090980
add xnewref
Fidget-Spinner Apr 30, 2024
2f50bcb
Add Py_STACKREF_NULL
Fidget-Spinner Apr 30, 2024
5961992
fix Windows build
Fidget-Spinner Apr 30, 2024
6bf6d44
try fix Windows for real this time
Fidget-Spinner Apr 30, 2024
003d274
address review
Fidget-Spinner Apr 30, 2024
7501357
revert hasdeferred refcount back to original file
Fidget-Spinner Apr 30, 2024
53608d7
remove decref_owned
Fidget-Spinner Apr 30, 2024
0131fc1
address review
Fidget-Spinner Apr 30, 2024
File filter
Filter by extension
Clear filters
Conversations
Failed to load comments.
Loading

Uh oh!

There was an error while loading. Please reload this page.
Jump to
Jump to file
Failed to load files.
Loading

Uh oh!

There was an error while loading. Please reload this page.
Diff view
Diff view
Next Next commit
Add header for tagged pointers
  • Loading branch information

Uh oh!

There was an error while loading. Please reload this page.

@Fidget-Spinner
Fidget-Spinner committed Apr 26, 2024
commit 209ddacef59b09ba732a05924f1ebf8ff0e0d044

There are no files selected for viewing

14 changes: 0 additions & 14 deletions 14 Include/internal/pycore_object.h
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -159,20 +159,6 @@ static inline void _Py_ClearImmortal(PyObject *op)
op = NULL; \
} while (0)

// Mark an object as supporting deferred reference counting. This is a no-op
// in the default (with GIL) build. Objects that use deferred reference
// counting should be tracked by the GC so that they are eventually collected.
extern void _PyObject_SetDeferredRefcount(PyObject *op);

static inline int
_PyObject_HasDeferredRefcount(PyObject *op)
{
#ifdef Py_GIL_DISABLED
return (op->ob_gc_bits & _PyGC_BITS_DEFERRED) != 0;
#else
return 0;
#endif
}

#if !defined(Py_GIL_DISABLED)
static inline void
Expand Down
208 changes: 208 additions & 0 deletions 208 Include/internal/pycore_tagged.h
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,208 @@
#ifndef Py_INTERNAL_TAGGED_H
Comment thread
Fidget-Spinner marked this conversation as resolved.
Resolved

Uh oh!

There was an error while loading. Please reload this page.
#define Py_INTERNAL_TAGGED_H
#ifdef __cplusplus
extern "C" {
#endif

#ifndef Py_BUILD_CORE
# error "this header requires Py_BUILD_CORE define"
#endif

#include <stddef.h>

// Mark an object as supporting deferred reference counting. This is a no-op
// in the default (with GIL) build. Objects that use deferred reference
// counting should be tracked by the GC so that they are eventually collected.
extern void _PyObject_SetDeferredRefcount(PyObject *op);

static inline int
_PyObject_HasDeferredRefcount(PyObject *op)
{
#ifdef Py_GIL_DISABLED
return (op->ob_gc_bits & _PyGC_BITS_DEFERRED) != 0;
#else
return 0;
#endif
}
Comment thread Outdated

@colesbury colesbury Apr 29, 2024

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor, but can we keep the _PyObject_HasDeferredRefcount and _PyObject_SetDeferredRefcount functions in pycore_object.h? We can #include pycore_object.h in pycore_tagged.h if needed.

  • Keeps accessors for ob_gc_bits in the same file
  • _PyStackRef use is mostly limited to the interpreter and some supporting functions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@Fidget-Spinner Fidget-Spinner Apr 30, 2024

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried this in the original branch but we can't easily for some reason. We can't #include pycore_object.h in pycore_tagged.h because it causes anything that includes pycore_interp.h to fail to compile. Not sure why.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@ericsnowcurrently ericsnowcurrently Apr 30, 2024

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, my old enemy...circular includes. When this has happened to me in the past, I solved this by splitting out the problematic parts of the header file into a new "pycore_..._state.h". It holds just the structs needed by pycore_interp.h. Then pycore_interp.h includes only the "_state.h" file. You can see examples in Include/internal.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
colesbury and Fidget-Spinner reacted with thumbs up emoji
All reactions
  • 👍 2 reactions

typedef union {
uintptr_t bits;
} _PyStackRef;

#ifdef Py_GIL_DISABLED
#define Py_TAG_DEFERRED (0b1)
Comment thread
Fidget-Spinner marked this conversation as resolved.
Resolved

Uh oh!

There was an error while loading. Please reload this page.
#define Py_TAG (Py_TAG_DEFERRED)
#else
#define Py_TAG 0
#endif

#if defined(Py_TAG_TEST)
Comment thread
Fidget-Spinner marked this conversation as resolved.
Resolved

Uh oh!

There was an error while loading. Please reload this page.
#define Py_STACKREF_UNTAG_BORROWED(tagged) ((PyObject *)(uintptr_t)((tagged).bits & (~Py_TAG_TEST)))
#elif defined(Py_GIL_DISABLED)
static inline PyObject *
_Py_STACKREF_UNTAG_BORROWED(_PyStackRef tagged) {
PyObject *cleared = ((PyObject *)((tagged).bits & (~Py_TAG)));
return cleared;
}
#define Py_STACKREF_UNTAG_BORROWED(tagged) _Py_STACKREF_UNTAG_BORROWED(tagged)
#else
#define Py_STACKREF_UNTAG_BORROWED(tagged) ((PyObject *)(uintptr_t)((tagged).bits))
#endif

#if defined(Py_TAG_TEST)
#define Py_STACKREF_TAG(obj) ((_PyStackRef){.bits = ((uintptr_t)(obj) | Py_TAG_TEST)})
#elif defined(Py_GIL_DISABLED)
static inline _PyStackRef
_Py_STACKREF_TAG(PyObject *obj) {
// Make sure we don't take an already tagged value.
assert(Py_STACKREF_UNTAG_BORROWED(((_PyStackRef){.bits = ((uintptr_t)(obj))})) == obj);
return ((_PyStackRef){.bits = ((uintptr_t)(obj))});
}
#define Py_STACKREF_TAG(obj) _Py_STACKREF_TAG(_PyObject_CAST(obj))
#else
#define Py_STACKREF_TAG(obj) ((_PyStackRef){.bits = ((uintptr_t)(obj))})
#endif

#if defined(Py_TAG_TEST)
#define Py_STACKREF_TAG_DEFERRED(obj) ((_PyStackRef){.bits = ((uintptr_t)(obj) | Py_TAG_TEST)})
#elif defined(Py_GIL_DISABLED)
static inline _PyStackRef
_Py_STACKREF_TAG_DEFERRED(PyObject *obj) {
// Make sure we don't take an already tagged value.
assert(Py_STACKREF_UNTAG_BORROWED(((_PyStackRef){.bits = ((uintptr_t)(obj))})) == obj);
int is_deferred = (obj != NULL && _PyObject_HasDeferredRefcount(obj));
int tag = (is_deferred ? Py_TAG_DEFERRED : 0);
return ((_PyStackRef){.bits = ((uintptr_t)(obj) | tag)});
}
#define Py_STACKREF_TAG_DEFERRED(obj) _Py_STACKREF_TAG_DEFERRED(_PyObject_CAST(obj))
#else
#define Py_STACKREF_TAG_DEFERRED(obj) ((_PyStackRef){.bits = ((uintptr_t)(obj))})
#endif

#if defined(Py_TAG_TEST)
#define Py_STACKREF_UNTAG_OWNED(tagged) Py_STACKREF_UNTAG_BORROWED(tagged)
#elif defined(Py_GIL_DISABLED)
static inline PyObject *
_Py_STACKREF_UNTAG_OWNED(_PyStackRef tagged) {
if ((tagged.bits & Py_TAG_DEFERRED) == Py_TAG_DEFERRED) {
assert(_PyObject_HasDeferredRefcount(Py_STACKREF_UNTAG_BORROWED(tagged)));
return Py_NewRef(Py_STACKREF_UNTAG_BORROWED(tagged));
}
return Py_STACKREF_UNTAG_BORROWED(tagged);
}
#define Py_STACKREF_UNTAG_OWNED(tagged) _Py_STACKREF_UNTAG_OWNED(tagged)
#else
#define Py_STACKREF_UNTAG_OWNED(tagged) Py_STACKREF_UNTAG_BORROWED(tagged)
#endif

#define MAX_UNTAG_SCRATCH 10
Comment thread
Fidget-Spinner marked this conversation as resolved.
Resolved

Uh oh!

There was an error while loading. Please reload this page.

static inline void
_Py_untag_stack_borrowed(PyObject **dst, const _PyStackRef *src, size_t length) {
for (size_t i = 0; i < length; i++) {
dst[i] = Py_STACKREF_UNTAG_BORROWED(src[i]);
}
}

static inline void
_Py_untag_stack_owned(PyObject **dst, const _PyStackRef *src, size_t length) {
for (size_t i = 0; i < length; i++) {
dst[i] = Py_STACKREF_UNTAG_OWNED(src[i]);
}
}


#define Py_XSETREF_STACKREF(dst, src) \
do { \
_PyStackRef *_tmp_dst_ptr = _Py_CAST(_PyStackRef*, &(dst)); \
_PyStackRef _tmp_old_dst = (*_tmp_dst_ptr); \
*_tmp_dst_ptr = (src); \
Py_XDECREF_STACKREF(_tmp_old_dst); \
} while (0)

#define Py_SETREF_STACKREF(dst, src) \
do { \
_PyStackRef *_tmp_dst_ptr = _Py_CAST(_PyStackRef*, &(dst)); \
_PyStackRef _tmp_old_dst = (*_tmp_dst_ptr); \
*_tmp_dst_ptr = (src); \
Py_DECREF_STACKREF(_tmp_old_dst); \
} while (0)

#define Py_CLEAR_STACKREF(op) \
do { \
_PyStackRef *_tmp_op_ptr = _Py_CAST(_PyStackRef*, &(op)); \
_PyStackRef _tmp_old_op = (*_tmp_op_ptr); \
if (Py_STACKREF_UNTAG_BORROWED(_tmp_old_op) != NULL) { \
*_tmp_op_ptr = Py_STACKREF_TAG(_Py_NULL); \
Py_DECREF_STACKREF(_tmp_old_op); \
} \
} while (0)

#if defined(Py_GIL_DISABLED)
static inline void
Comment thread
Fidget-Spinner marked this conversation as resolved.
Resolved

Uh oh!

There was an error while loading. Please reload this page.
_Py_DecRef_StackRef(_PyStackRef tagged) {
if ((tagged.bits & Py_TAG_DEFERRED) == Py_TAG_DEFERRED) {
return;
}
Py_DECREF(Py_STACKREF_UNTAG_BORROWED(tagged));
}
#define Py_DECREF_STACKREF(op) _Py_DecRef_StackRef(op)
#else
#define Py_DECREF_STACKREF(op) Py_DECREF(Py_STACKREF_UNTAG_BORROWED(op))
#endif

#if defined(Py_GIL_DISABLED)
static inline _PyStackRef
_Py_NewRef_StackRef_Owned(_PyStackRef tagged) {
if ((tagged.bits & Py_TAG_DEFERRED) == Py_TAG_DEFERRED) {
Py_INCREF(Py_STACKREF_UNTAG_BORROWED(tagged));
uintptr_t res = tagged.bits & (~Py_TAG_DEFERRED);
return ((_PyStackRef){.bits = res});
}
return tagged;
}
#define Py_NEWREF_STACKREF_OWNED(op) _Py_NewRef_StackRef_Owned(op)
#else
#define Py_NEWREF_STACKREF_OWNED(op) (op)
#endif

#define Py_DECREF_STACKREF_OWNED(op) Py_DECREF(Py_STACKREF_UNTAG_BORROWED(op));

#if defined(Py_GIL_DISABLED)
static inline void
_Py_IncRef_StackRef(_PyStackRef tagged) {
if ((tagged.bits & Py_TAG_DEFERRED) == Py_TAG_DEFERRED) {
assert(_PyObject_HasDeferredRefcount(Py_STACKREF_UNTAG_BORROWED(tagged)));
return;
}
return Py_INCREF(Py_STACKREF_UNTAG_BORROWED(tagged));
}
#define Py_INCREF_STACKREF(op) _Py_IncRef_StackRef(op)
#else
#define Py_INCREF_STACKREF(op) Py_INCREF(Py_STACKREF_UNTAG_BORROWED(op))
#endif

static inline void
_Py_XDECREF_STACKREF(_PyStackRef op)
{
if (Py_STACKREF_UNTAG_BORROWED(op) != NULL) {
Py_DECREF_STACKREF(op);
}
}

#define Py_XDECREF_STACKREF(op) _Py_XDECREF_STACKREF(op)

static inline _PyStackRef
Py_NewRef_StackRef(_PyStackRef obj)
{
Py_INCREF_STACKREF(obj);
return obj;
}


#define Py_NewRef_StackRef(op) Py_NewRef_StackRef(op)

#ifdef __cplusplus
}
#endif
#endif /* !Py_INTERNAL_TAGGED_H */
1 change: 1 addition & 0 deletions 1 Makefile.pre.in
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1225,6 +1225,7 @@ PYTHON_HEADERS= \
$(srcdir)/Include/internal/pycore_structseq.h \
$(srcdir)/Include/internal/pycore_symtable.h \
$(srcdir)/Include/internal/pycore_sysmodule.h \
$(srcdir)/Include/internal/pycore_tagged.h \
$(srcdir)/Include/internal/pycore_time.h \
$(srcdir)/Include/internal/pycore_token.h \
$(srcdir)/Include/internal/pycore_traceback.h \
Expand Down
1 change: 1 addition & 0 deletions 1 Objects/codeobject.c
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
#include "pycore_opcode_utils.h" // RESUME_AT_FUNC_START
#include "pycore_pystate.h" // _PyInterpreterState_GET()
#include "pycore_setobject.h" // _PySet_NextEntry()
#include "pycore_tagged.h" // _PyObject_SetDeferredRefcount
#include "pycore_tuple.h" // _PyTuple_ITEMS()
#include "clinic/codeobject.c.h"

Expand Down
1 change: 1 addition & 0 deletions 1 Objects/descrobject.c
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
#include "pycore_modsupport.h" // _PyArg_UnpackStack()
#include "pycore_object.h" // _PyObject_GC_UNTRACK()
#include "pycore_pystate.h" // _PyThreadState_GET()
#include "pycore_tagged.h" // _PyObject_SetDeferredRefcount
#include "pycore_tuple.h" // _PyTuple_ITEMS()


Expand Down
1 change: 1 addition & 0 deletions 1 Objects/funcobject.c
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
#include "pycore_modsupport.h" // _PyArg_NoKeywords()
#include "pycore_object.h" // _PyObject_GC_UNTRACK()
#include "pycore_pyerrors.h" // _PyErr_Occurred()
#include "pycore_tagged.h" // _PyObject_SetDeferredRefcount


static const char *
Expand Down
1 change: 1 addition & 0 deletions 1 Objects/moduleobject.c
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
#include "pycore_object.h" // _PyType_AllocNoTrack
#include "pycore_pyerrors.h" // _PyErr_FormatFromCause()
#include "pycore_pystate.h" // _PyInterpreterState_GET()
#include "pycore_tagged.h" // _PyObject_SetDeferredRefcount

#include "osdefs.h" // MAXPATHLEN

Expand Down
1 change: 1 addition & 0 deletions 1 Objects/typeobject.c
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
#include "pycore_pyerrors.h" // _PyErr_Occurred()
#include "pycore_pystate.h" // _PyThreadState_GET()
#include "pycore_symtable.h" // _Py_Mangle()
#include "pycore_tagged.h" // _PyObject_SetDeferredRefcount
#include "pycore_typeobject.h" // struct type_cache
#include "pycore_unionobject.h" // _Py_union_type_or
#include "pycore_weakref.h" // _PyWeakref_GET_REF()
Expand Down
1 change: 1 addition & 0 deletions 1 PCbuild/pythoncore.vcxproj
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -291,6 +291,7 @@
<ClInclude Include="..\Include\internal\pycore_structseq.h" />
<ClInclude Include="..\Include\internal\pycore_sysmodule.h" />
<ClInclude Include="..\Include\internal\pycore_symtable.h" />
<ClInclude Include="..\Include\internal\pycore_tagged.h" />
<ClInclude Include="..\Include\internal\pycore_time.h" />
<ClInclude Include="..\Include\internal\pycore_token.h" />
<ClInclude Include="..\Include\internal\pycore_traceback.h" />
Expand Down
3 changes: 3 additions & 0 deletions 3 PCbuild/pythoncore.vcxproj.filters
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -789,6 +789,9 @@
<ClInclude Include="..\Include\internal\pycore_symtable.h">
<Filter>Include\internal</Filter>
</ClInclude>
<ClInclude Include="..\Include\internal\pycore_tagged.h">
<Filter>Include\internal</Filter>
</ClInclude>
<ClInclude Include="..\Include\internal\pycore_time.h">
<Filter>Include\internal</Filter>
</ClInclude>
Expand Down
1 change: 1 addition & 0 deletions 1 Python/gc_free_threading.c
Show comments
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@
#include "pycore_object_stack.h"
#include "pycore_pyerrors.h"
#include "pycore_pystate.h" // _PyThreadState_GET()
#include "pycore_tagged.h" // _PyObject_SetDeferredRefcount
#include "pycore_time.h" // _PyTime_GetPerfCounter()
#include "pycore_tstate.h" // _PyThreadStateImpl
#include "pycore_weakref.h" // _PyWeakref_ClearRef()
Expand Down
Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.