GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-111089: PyUnicode_AsUTF8AndSize() sets size on error by vstinner · Pull Request #111106 · python/cpython

Conversation

@vstinner

@vstinner vstinner commented Oct 20, 2023 •
edited by github-actions Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

On error, PyUnicode_AsUTF8AndSize() now sets the size argument to 0, to avoid undefined value.

  • Issue: [C API] Change PyUnicode_AsUTF8() to return NULL on embedded null characters #111089

📚 Documentation preview 📚: https://cpython-previews--111106.org.readthedocs.build/

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@vstinner vstinner added the skip news label Oct 20, 2023
@bedevere-app bedevere-app Bot mentioned this pull request Oct 20, 2023
[C API] Change PyUnicode_AsUTF8() to return NULL on embedded null characters #111089
Closed
@bedevere-app bedevere-app Bot added the awaiting core review label Oct 20, 2023
@vstinner vstinner mentioned this pull request Oct 20, 2023
Add tests for failing PyUnicode_AsUTF8AndSize() with psize=NULL #111100
Merged
@vstinner

vstinner commented Oct 20, 2023

Copy link
Copy Markdown
Member Author

See also PR #111100: Add tests for failing PyUnicode_AsUTF8AndSize() with psize=NULL.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner vstinner mentioned this pull request Oct 20, 2023
gh-111089: PyUnicode_AsUTF8() now raises on embedded NUL #111091
Merged
serhiy-storchaka
serhiy-storchaka approved these changes Oct 20, 2023
View reviewed changes

@serhiy-storchaka serhiy-storchaka left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. See also #110865.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Comment thread Doc/c-api/unicode.rst Outdated

In the case of an error, ``NULL`` is returned with an exception set and no
*size* is stored.
On error, set an exception, set *size* to 0 (if it's not NULL) and return

@serhiy-storchaka serhiy-storchaka Oct 20, 2023

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
On error, set an exception, set *size* to 0 (if it's not NULL) and return
On error, set an exception, set *size* to ``0`` (if it's not ``NULL``) and return

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting core review labels Oct 20, 2023
@serhiy-storchaka

serhiy-storchaka commented Oct 20, 2023

Copy link
Copy Markdown
Member

Why set size to 0, and not to -1? There is larger chance to miss error. For example PyUnicode_FromStringAndSize(NULL, 0) and PyBytes_FromStringAndSize(NULL, 0) return empty string and empty bytes object, but if size is negative, they raise exception.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner
pythongh-111089: PyUnicode_AsUTF8AndSize() sets size on error
ae7734d 
On error, PyUnicode_AsUTF8AndSize() now sets the size argument to -1,
to avoid undefined value.
@vstinner
vstinner force-pushed the asutf8andsize branch from 585158d to ae7734d Compare October 20, 2023 15:52
@vstinner

vstinner commented Oct 20, 2023

Copy link
Copy Markdown
Member Author

Why set size to 0, and not to -1?

That's a very good idea! I updated my PR to set size of -1 on error.

I just picked 0 randomly. I forgot that size can be negative.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner vstinner mentioned this pull request Oct 20, 2023
For output parameters passed by pointer, set them on error: avoid uninitialized values capi-workgroup/api-evolution#32
Open
serhiy-storchaka
serhiy-storchaka approved these changes Oct 20, 2023
View reviewed changes
@vstinner

vstinner commented Oct 20, 2023

Copy link
Copy Markdown
Member Author

Oh, tests failed with:

Testing of doctests in the sources finished, look at the results in build/doctest/output.txt.
python: ./Include/internal/pycore_typeobject.h:105: _PyType_GetModuleState: Assertion `et->ht_module' failed.
Aborted (core dumped)
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@vstinner
vstinner merged commit f1e751e into python:main Oct 20, 2023
@vstinner
vstinner deleted the asutf8andsize branch October 20, 2023 18:03
@bedevere-app bedevere-app Bot removed the awaiting merge label Oct 20, 2023
@vstinner

vstinner commented Oct 20, 2023

Copy link
Copy Markdown
Member Author

I failed to reproduce the Sphinx crash locally. And the test passed when run again.

I merged my PR, thanks for review Serhiy.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
aisk pushed a commit to aisk/cpython that referenced this pull request Feb 11, 2024
@vstinner @aisk
pythongh-111089: PyUnicode_AsUTF8AndSize() sets size on error (python…
0df9740 
…#111106)

On error, PyUnicode_AsUTF8AndSize() now sets the size argument to -1,
to avoid undefined value.
Glyphack pushed a commit to Glyphack/cpython that referenced this pull request Sep 2, 2024
@vstinner @Glyphack
pythongh-111089: PyUnicode_AsUTF8AndSize() sets size on error (python…
cb8a69a 
…#111106)

On error, PyUnicode_AsUTF8AndSize() now sets the size argument to -1,
to avoid undefined value.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@serhiy-storchaka serhiy-storchaka serhiy-storchaka approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@vstinner @serhiy-storchaka
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.