GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-119609: Add PyUnicode_Export() and PyUnicode_Import() functions by vstinner · Pull Request #119610 · python/cpython

Conversation

@vstinner

@vstinner vstinner commented May 27, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member
  • Issue: [C API] PEP 756: Add PyUnicode_Export() and PyUnicode_Import() functions #119609

📚 Documentation preview 📚: https://cpython-previews--119610.org.readthedocs.build/

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@vstinner
vstinner requested review from a team and encukou as code owners May 27, 2024 15:47
@bedevere-app bedevere-app Bot added the awaiting core review label May 27, 2024
@bedevere-app bedevere-app Bot mentioned this pull request May 27, 2024
[C API] PEP 756: Add PyUnicode_Export() and PyUnicode_Import() functions #119609
Closed
@vstinner

vstinner commented May 27, 2024

Copy link
Copy Markdown
Member Author

Docs build failed because of #119607

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner
vstinner force-pushed the unicode_native branch from f6b67a3 to 2f8bf77 Compare May 27, 2024 18:56
@vstinner

vstinner commented May 27, 2024

Copy link
Copy Markdown
Member Author

Docs build failed because of #119607

PR rebased on top of this fix.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

erlend-aasland
erlend-aasland reviewed May 28, 2024
View reviewed changes
Comment thread Objects/unicodeobject.c Resolved

Uh oh!

There was an error while loading. Please reload this page.

@zooba zooba assigned encukou May 28, 2024
@zooba

zooba commented May 28, 2024

Copy link
Copy Markdown
Member

Petr indicated on the issue that he's got Thoughts and will write them up this week.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner
vstinner marked this pull request as draft May 29, 2024 12:21
@bedevere-app bedevere-app Bot removed the awaiting core review label May 29, 2024
@vstinner
vstinner force-pushed the unicode_native branch from 2f8bf77 to 4d77192 Compare June 11, 2024 12:38
@vstinner vstinner changed the title gh-119609: Add PyUnicode_AsNativeFormat() function gh-119609: Add PyUnicode_Export() and PyUnicode_Import() functions Jun 13, 2024
vstinner and others added 6 commits June 21, 2024 10:37
@vstinner
pythongh-119609: Add PyUnicode_Export() function
32a373a
Add PyUnicode_Export() and PyUnicode_Import() functions to the C API.
@vstinner
stable_abi.toml: Add constants
c42cebc
@vstinner @encukou
Export UCS1 as UCS2
1310b6d
Co-Authored-By: Petr Viktorin <[email protected]>
@vstinner
Add test_unicode_export_import_roundtrip()
1c78103
@vstinner
Rename size parameter to nbytes
62f4598
@vstinner
Update doc
53cd937
@vstinner
vstinner force-pushed the unicode_native branch from 11b9f43 to 53cd937 Compare June 21, 2024 08:59
@vstinner
vstinner marked this pull request as ready for review June 21, 2024 08:59
@bedevere-app bedevere-app Bot added the awaiting core review label Jun 21, 2024
@vstinner
Make sure that exported string ends with NUL character
3a08491
@vstinner

vstinner commented Jun 21, 2024

Copy link
Copy Markdown
Member Author

@encukou: Please review the updated PR.

@encukou: I prefer to make sure that the exported string ends with a NUL character, rather than making sure that it's not the case. It's convenient and cheap.

I rebased the PR on the main branch, I fixed merge conflicts, I updated the doc for new function names, I included some of Petr's suggestions. I marked the PR as ready for review (it's no longer a draft).

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner
tests: rename size to nbytes
a9a9083
encukou
encukou reviewed Jun 21, 2024
View reviewed changes

@encukou encukou left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!
(I don't know about the other C API WG members.)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread Doc/c-api/unicode.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/c-api/unicode.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/c-api/unicode.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread Doc/c-api/unicode.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

vstinner and others added 3 commits June 22, 2024 17:17
@vstinner @encukou
Update Doc/c-api/unicode.rst
1297c52
Co-authored-by: Petr Viktorin <[email protected]>
@vstinner @encukou
Update Doc/c-api/unicode.rst
d8e3d5d
Co-authored-by: Petr Viktorin <[email protected]>
@vstinner
Address Petr's review
da67c89
@vstinner

vstinner commented Jun 22, 2024

Copy link
Copy Markdown
Member Author

@davidism: Does the proposed API solve your MarkupSafe use case?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner
Merge branch 'main' into unicode_native
894921a
@vstinner vstinner mentioned this pull request Jun 24, 2024
PEP 756: Add PyUnicode_Export() and PyUnicode_Import() to the limited C API capi-workgroup/decisions#33
Closed
@vstinner

vstinner commented Jun 24, 2024

Copy link
Copy Markdown
Member Author

I created Add PyUnicode_Export() and PyUnicode_Import() to the limited C API issue in the C API WG Decisions project.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@davidism

davidism commented Jun 24, 2024 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

I really appreciate you thinking about this after talking at PyCon! From what I can tell (I'm not very familiar with C), the MarkupSafe code would mostly remain the same, but use the two new abi3 functions instead of the existing functions? Perhaps you could show a brief example or a high level explanation of what I would change? https://github.com/pallets/markupsafe/blob/d12057361ad75c4569e2e61712c234acc69d5d0b/src/markupsafe/_speedups.c

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner vstinner mentioned this pull request Jun 26, 2024
[WIP] Use PyUnicode_Export() and PyUnicode_Import() pallets/markupsafe#451
Closed
@vstinner

vstinner commented Jun 26, 2024

Copy link
Copy Markdown
Member Author

@davidism:

I really appreciate you thinking about this after talking at PyCon!

You're welcome.

From what I can tell (I'm not very familiar with C), the MarkupSafe code would mostly remain the same, but use the two new abi3 functions instead of the existing functions? Perhaps you could show a brief example or a high level explanation of what I would change?

I created a PR to show how these functions can be used: pallets/markupsafe#451

The stable ABI is less efficient since it requires to allocate a UCS1/UCS2/UCS4 buffer first, write into the buffer, and only then create a Python str object from this buffer. That's because PyUnicode_New() is excluded from the stable ABI.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner
Fix doc: add uint32_t type
f3857d8
@vstinner

vstinner commented Sep 6, 2024

Copy link
Copy Markdown
Member Author

I wrote a different PR using Py_buffer: #123738. I close this PR for now.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner vstinner closed this Sep 6, 2024
@vstinner vstinner mentioned this pull request Oct 2, 2024
Vote on PEP 756: Add PyUnicode_Export() and PyUnicode_Import() to the limited C API capi-workgroup/decisions#44
Closed
2 of 5 tasks
@vstinner
vstinner deleted the unicode_native branch January 14, 2025 08:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@encukou encukou encukou left review comments
@erlend-aasland erlend-aasland erlend-aasland left review comments

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@encukou encukou

Labels

awaiting core review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@vstinner @zooba @davidism @encukou @erlend-aasland
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.