GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-103509: PEP 697 -- Limited C API for Extending Opaque Types by encukou · Pull Request #103511 · python/cpython

Conversation

@encukou

@encukou encukou commented Apr 13, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

PEP 697 describes the change. Some details it doesn't cover:

  • _PyHeapType_GET_MEMBERS is moved from Include/internal to the only .c file that uses it.
  • Tests demonstrate the API is usable for the purpose, and verify the layout with brutish-force.

TODO:

  • What's New entry
  • Fix the alignof & max_align_t
  • Issue: PEP 697 -- Limited C API for Extending Opaque Types #103509

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@encukou
pythongh-103509: PEP 697 -- Limited C API for Extending Opaque Types
a9679fa
@encukou
encukou requested review from a team and markshannon as code owners April 13, 2023 14:56
@bedevere-bot bedevere-bot mentioned this pull request Apr 13, 2023
PEP 697 -- Limited C API for Extending Opaque Types #103509
Closed
@bedevere-bot bedevere-bot added the awaiting core review label Apr 13, 2023
@encukou
Add a blurb
b8dc072
@encukou
encukou marked this pull request as draft April 13, 2023 15:01
@erlend-aasland
erlend-aasland self-requested a review April 16, 2023 21:25
@encukou

encukou commented Apr 17, 2023

Copy link
Copy Markdown
Member Author

typeobject.c includes <stddef.h> and <stdalign.h>, which should define max_align_t and alignof in C11. Yet, the Windows build complains:

D:\a\cpython\cpython\Objects\typeobject.c(3558,34): warning C4013: 'alignof' undefined; assuming extern returning int [D:\a\cpython\cpython\PCbuild\_freeze_module.vcxproj]
D:\a\cpython\cpython\Objects\typeobject.c(3558,53): error C2065: 'max_align_t': undeclared identifier [D:\a\cpython\cpython\PCbuild\_freeze_module.vcxproj]

What am I missing?
@zooba, could you help?

(The reference I linked to lists alignof as removed in C23, which is a bit misleading. The macro will be removed because it will become a keyword.)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

encukou added 2 commits April 17, 2023 14:12
@encukou
Fix PyMember_SetOne error return (thanks MSVC!)
1fc5c52
@encukou
Add What's New entry
075ca51
@zooba

zooba commented Apr 17, 2023

Copy link
Copy Markdown
Member

Looks like you need stdalign.h for alignof (defines the alignof macro to _Alignof), and max_align_t only seems to exist for C++ via cstddef, where it's mapped to double.

encukou reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@encukou

encukou commented Apr 19, 2023

Copy link
Copy Markdown
Member Author

That means I can't use max_align_t in CPython? Well, I can work around it.
But I was under the impression that MSVC did support C11. I can start the discussion to add max_align_t (and possibly alignof?) as exceptions to PEP 7. Do you know of any other C11 required features MSVC (or our configuration of it) doesn't support?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

encukou added 6 commits April 19, 2023 14:48
@encukou
Fix warning in tests
439de5d
@encukou
Work around lack of alignof & max_align_t
291731b
@encukou
Use the same Sphinx role for Py_TPFLAGS_ITEMS_AT_END as for other typ…
2ba8084
…e flags

Now is not the time to fix that.
@encukou
Add ALIGNOF_MAX_ALIGN_T to configure & PC/pyconfig.h
b03d431
@encukou
Use ALIGNOF_MAX_ALIGN_T
ec9d5a8
@encukou
Fix typo
5cab814
@encukou
encukou marked this pull request as ready for review April 20, 2023 08:52
@encukou
encukou requested a review from corona10 as a code owner April 20, 2023 08:52
@encukou encukou added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 20, 2023
@bedevere-bot

bedevere-bot commented Apr 20, 2023

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @encukou for commit ec9d5a8 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 20, 2023
@encukou
encukou marked this pull request as draft April 20, 2023 09:41
@arhadthedev

arhadthedev commented Apr 20, 2023

Copy link
Copy Markdown
Member

The TraceRefs buildbot failure:

Modules/gcmodule.c:450: visit_decref: Assertion "!_PyObject_IsFreed(op)" failed
Enable tracemalloc to get the memory block allocation traceback

is likely caused by gh-103621.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

encukou added 3 commits April 20, 2023 15:53
@encukou
Don't include <stdalign.h>, it's not always available on Windows
3ade585
@encukou
Define ALIGNOF_MAX_ALIGN_T with long double if it's not available
986bf26
@encukou
tests: Compute data_offset in C to avoid overflow issues
e7838ee
arhadthedev
arhadthedev requested changes Apr 21, 2023
View reviewed changes

@arhadthedev arhadthedev left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

void* cannot participate in address arithmetics. You need to cast both data_ptr and instance to char * to get distance in bytes.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

encukou reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
Comment thread Modules/_testcapi/heaptype_relative.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Modules/_testcapi/heaptype_relative.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot

bedevere-bot commented Apr 27, 2023

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @encukou for commit 402ecc6 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 27, 2023
encukou added 2 commits April 28, 2023 13:36
@encukou
Don't rely on PyObject* being aligned to ALIGNOF_MAX_ALIGN_T
1701688
Smaller objects might not be aligned to ALIGNOF_MAX_ALIGN_T.
The offsets calculated for PEP 697 should be aligned, though.
@encukou
Merge in main branch
db5d49b
@encukou encukou added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 28, 2023
@bedevere-bot

bedevere-bot commented Apr 28, 2023

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @encukou for commit db5d49b 🤖

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label Apr 28, 2023
erlend-aasland
erlend-aasland reviewed Apr 28, 2023
View reviewed changes

@erlend-aasland erlend-aasland left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice!

Sorry for the late review; it took some time to dig through everything.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Doc/c-api/object.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/c-api/type.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/c-api/object.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/c-api/object.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/c-api/object.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Objects/typeobject.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread PC/python3dll.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Python/structmember.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Objects/typeobject.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Objects/typeobject.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

encukou and others added 7 commits May 2, 2023 15:15
@encukou @erlend-aasland
Apply suggestions from code review
9d9911d
Co-authored-by: Erlend E. Aasland <[email protected]>
@encukou
Raise TypeError on missing flag
06bcf5b
@encukou
Wrap new tests in a class
0b69748
@encukou
Use PyModule_AddIntMacro in tests
33c5258
@encukou
Test failure of extending non-ITEMS_AT_END variable-sized types
88dade7
@encukou
Test error cases around members
8720b25
@encukou
Merge in the main branch
0474e69
@encukou

encukou commented May 2, 2023

Copy link
Copy Markdown
Member Author

Thank you for the thorough review!
This should address the remaining comments.

erlend-aasland reacted with heart emoji
All reactions
  • ❤️ 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@encukou
encukou requested a review from arhadthedev May 2, 2023 15:20
@erlend-aasland

erlend-aasland commented May 2, 2023 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Thanks! I've got some more comments, but we can deal with those in a follow-up PR if you think it's worth it. Let's land this before the feature freeze kicks in.

(I'm on mobile now; I won't be able to check the changes until tomorrow.)

encukou reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

arhadthedev
arhadthedev reviewed May 2, 2023
View reviewed changes

@arhadthedev arhadthedev left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, keep in mind that I neither participated in the PR discussion nor seen the PEP before. So I looked at the rendered docs in the same fashion as an outside user would do.

Here are three optional rough proposals and a forgotten full stop. However, they are non-binding suggestions that can be properly considered later, after 3.12b1 is out (otherwise the two remaining weeks will pass swiftly in the bikeshedding).

I've limited myself to the documentation because I have no expertise in the C API to check such a volume of code in such a short period of time.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Doc/c-api/object.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/c-api/object.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/c-api/object.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/c-api/type.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

@encukou @arhadthedev
Apply suggestions from code review
a975de9
Co-authored-by: Oleg Iarygin <[email protected]>
encukou
encukou commented May 3, 2023
View reviewed changes

@encukou encukou left a comment

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you! Fresh eyes is exactly what the documentation needs.

It will also need a tutorial to bring the concepts together -- but when trying to write a tutorial, I keep finding rough edges that should be fixed first :)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Doc/c-api/object.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/c-api/object.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

@encukou
Merge in the main branch
b9ddf21
arhadthedev
arhadthedev approved these changes May 3, 2023
View reviewed changes
@encukou

encukou commented May 4, 2023

Copy link
Copy Markdown
Member Author

Thanks for the reviews!

I'll merge, and leave the issues open for other PRs.

erlend-aasland and arhadthedev reacted with hooray emoji erlend-aasland and arhadthedev reacted with rocket emoji
All reactions
  • 🎉 2 reactions
  • 🚀 2 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@encukou
encukou merged commit cd9a56c into python:main May 4, 2023
@bedevere-bot bedevere-bot removed the awaiting core review label May 4, 2023
@encukou
encukou deleted the extend-opaque-sq branch May 4, 2023 07:56
@erlend-aasland

erlend-aasland commented May 4, 2023

Copy link
Copy Markdown
Contributor

Thanks for the reviews!

I'll merge, and leave the issues open for other PRs.

Good job! This is a very nice feature.

encukou reacted with heart emoji
All reactions
  • ❤️ 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@encukou

encukou commented May 4, 2023

Copy link
Copy Markdown
Member Author

Thank you!

FWIW, here's a follow-up on the alignof(max_align_t): https://discuss.python.org/t/requiring-compilers-c11-standard-mode-to-build-cpython/26481

erlend-aasland reacted with eyes emoji
All reactions
  • 👀 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@hugovk hugovk mentioned this pull request Dec 29, 2023
gh-101100: Fix Sphinx warnings in c-api/structures.rst #113564
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@erlend-aasland erlend-aasland erlend-aasland left review comments
@arhadthedev arhadthedev arhadthedev approved these changes
@markshannon markshannon Awaiting requested review from markshannon markshannon is a code owner
@corona10 corona10 Awaiting requested review from corona10 corona10 is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@encukou @zooba @bedevere-bot @arhadthedev @erlend-aasland
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.