GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-160518: Fix hz encoder dropping half of the ~~ escape by gpshead · Pull Request #160519 · python/cpython

Conversation

@gpshead

@gpshead gpshead commented Oct 23, 2026 •
edited by bedevere-app Bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

hz_encode() wrote the first byte of the ~~ escape, consumed the input character, and only then wrote the second byte. When the output buffer was full at that point, multibytecodec_encode() enlarged it and resumed at the next character, so the second ~ was lost. The escape is now written with WRITEBYTE2 before NEXT(1, 2), like the other multi-byte writes in this file, so the retry starts at the tilde again.

The test sweeps the text length across the initial buffer size and its first enlargement; it fails at 4 of the 120 lengths without the fix. Backports to branches older than 3.14 would need assertEndsWith replaced, but 3.14 and 3.15 have it.

Fixes #160518

ANT-2026-FXW1XYKR

  • Issue: hz codec encodes ~ as a single ~ when the output buffer fills mid-escape #160518

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@gpshead
pythongh-160518: Fix hz encoder dropping half of the ~~ escape
11342d9 
Write both bytes of the escape before consuming the input character, so
that a retry after the output buffer is enlarged starts at the tilde.
@gpshead gpshead added needs backport to 3.14 bugs and security fixes needs backport to 3.15 bugs and security fixes labels Oct 23, 2026
@gpshead gpshead self-assigned this Oct 23, 2026
@bedevere-app bedevere-app Bot mentioned this pull request Oct 23, 2026
hz codec encodes ~ as a single ~ when the output buffer fills mid-escape #160518
Closed
@gpshead
gpshead marked this pull request as ready for review October 23, 2026 22:00
@gpshead
gpshead requested a review from corona10 as a code owner October 23, 2026 22:00
@bedevere-app bedevere-app Bot added the awaiting core review label Oct 23, 2026
@gpshead gpshead added the sprint label Oct 23, 2026
@github-project-automation github-project-automation Bot added this to Sprint Oct 23, 2026
@github-project-automation github-project-automation Bot moved this to Todo in Sprint Oct 23, 2026
@gpshead
gpshead merged commit 1259f16 into python:main Oct 31, 2026
65 checks passed

Uh oh!

There was an error while loading. Please reload this page.
@miss-islington-app

miss-islington-app Bot commented Oct 31, 2026

Copy link
Copy Markdown

Thanks @gpshead for the PR 🌮🎉.. I'm working now to backport this PR to: 3.14, 3.15.
🐍🍒⛏🤖
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-app bedevere-app Bot removed the awaiting core review label Oct 31, 2026
@github-project-automation github-project-automation Bot moved this from Todo to Done in Sprint Oct 31, 2026
@gpshead
gpshead deleted the scanfix-codecscn-fxw1xykr branch October 31, 2026 12:57
@bedevere-app

bedevere-app Bot commented Oct 31, 2026

Copy link
Copy Markdown

GH-161002 is a backport of this pull request to the 3.15 branch.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-app bedevere-app Bot removed the needs backport to 3.15 bugs and security fixes label Oct 31, 2026
@bedevere-app

bedevere-app Bot commented Oct 31, 2026

Copy link
Copy Markdown

GH-161003 is a backport of this pull request to the 3.14 branch.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-app bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label Oct 31, 2026
gpshead added a commit that referenced this pull request Nov 1, 2026
@miss-islington @gpshead
[3.14] gh-160518: Fix hz encoder dropping half of the ~~ escape (GH-1…
922b7f7 
…60519) (#161003)

gh-160518: Fix hz encoder dropping half of the ~~ escape (GH-160519)

Write both bytes of the escape before consuming the input character, so
that a retry after the output buffer is enlarged starts at the tilde.
(cherry picked from commit 1259f16)

Co-authored-by: Gregory P. Smith <[email protected]>
gpshead added a commit that referenced this pull request Nov 1, 2026
@miss-islington @gpshead
[3.15] gh-160518: Fix hz encoder dropping half of the ~~ escape (GH-1…
3135a08 
…60519) (#161002)

gh-160518: Fix hz encoder dropping half of the ~~ escape (GH-160519)

Write both bytes of the escape before consuming the input character, so
that a retry after the output buffer is enlarged starts at the tilde.
(cherry picked from commit 1259f16)

Co-authored-by: Gregory P. Smith <[email protected]>
@gpshead gpshead mentioned this pull request Nov 1, 2026
gh-159739: Merge the duplicate NEWS entries for the hz tilde fix #161085
Merged
gpshead added a commit that referenced this pull request Nov 1, 2026
@gpshead
gh-159739: Merge the duplicate NEWS entries for the hz tilde fix (GH-…
b71fadc 
…161085)

GH-160400 and GH-160519 fixed the same hz encoder bug and each added
a NEWS entry for it.  Keep one, in the gh-159739 entry.
gpshead added a commit that referenced this pull request Nov 1, 2026
@gpshead
[3.15] gh-159739: Merge the duplicate NEWS entries for the hz tilde f…
9b917b1 
…ix (GH-161085) (#161141)

GH-160400 and GH-160519 fixed the same hz encoder bug and each added
a NEWS entry for it.  Keep one, in the gh-159739 entry.

(cherry picked from commit b71fadc)
gpshead added a commit that referenced this pull request Nov 1, 2026
@gpshead
[3.14] gh-159739: Merge the duplicate NEWS entries for the hz tilde f…
98a26e9 
…ix (GH-161085) (#161142)

GH-160400 and GH-160519 fixed the same hz encoder bug and each added
a NEWS entry for it.  Keep one, in the gh-159739 entry.

(cherry picked from commit b71fadc)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@corona10 corona10 Awaiting requested review from corona10 corona10 is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@gpshead gpshead

Labels

sprint

Projects

Sprint
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

hz codec encodes ~ as a single ~ when the output buffer fills mid-escape

Uh oh!

There was an error while loading. Please reload this page.

1 participant

@gpshead
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.