GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — gh-150771: Fix email serialization for shift_jis and euc-jp by bhuvi27 · Pull Request #151120 · python/cpython

Conversation

bhuvi27 commented Jun 9, 2026 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Contributor

Fixes #150771

Creating a message with set_content(..., charset='shift_jis') or charset='euc-jp' raised UnicodeEncodeError on str(m) because the payload was encoded with the input charset while the Content-Type uses the output
charset (iso-2022-jp).

Use Charset.output_charset in set_text_content so the payload and Content-Type agree from the start.

Issue: Issues with Shift_JIS and EUC-JP in email #150771

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

bhuvi27 requested a review from a team as a code owner

June 9, 2026 02:15

bedevere-app Bot added the awaiting review label

Jun 9, 2026

bedevere-app Bot mentioned this pull request

Jun 9, 2026

Issues with Shift_JIS and EUC-JP in email #150771

Closed

bedevere-app Bot commented Jun 9, 2026

Copy link

Copy Markdown

Most changes to Python require a NEWS entry. Add one using the blurb_it web app or the blurb command-line tool.

If this change has little impact on Python users, wait for a maintainer to apply the skip news label instead.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bhuvi27 force-pushed the gh-150771-fix-email-shift-jis-euc-jp branch from 5928e5b to f071b32 Compare

June 9, 2026 02:48


          pythongh-150771: Fix email serialization for shift_jis and euc-jp

6795f58

Convert surrogate-escaped payloads through the input charset before
encoding to iso-2022-jp, fixing UnicodeEncodeError when printing
messages created with set_content().

bhuvi27 force-pushed the gh-150771-fix-email-shift-jis-euc-jp branch from f071b32 to 6795f58 Compare

June 9, 2026 02:49

bhuvi27 added 5 commits

June 10, 2026 08:44


          Merge branch 'main' into pythongh-150771-fix-email-shift-jis-euc-jp

2f08540


          Merge branch 'main' into pythongh-150771-fix-email-shift-jis-euc-jp

24f6067


          pythongh-150771: Use output charset in set_text_content for shift_jis…

f6c53cb

…/euc-jp

Encode the payload with the charset output mapping (iso-2022-jp) when
set_content is called with shift_jis or euc-jp, instead of patching
serialization in body_encode and set_payload. Reverts those changes.


          Merge remote-tracking branch 'origin/pythongh-150771-fix-email-shift-…

52810e5

…jis-euc-jp'


          pythongh-150771: Fix NEWS entry doc references for docs CI

e2f3984

Use plain backticks for set_content() instead of a broken :func: target.

serhiy-storchaka approved these changes

Jun 14, 2026

View reviewed changes

serhiy-storchaka left a comment

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. 👍

Do you think an assertion for bytes(m) would be useful?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

Comment thread

Lib/test/test_email/test_contentmanager.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

bedevere-app Bot added awaiting merge and removed awaiting review labels

Jun 14, 2026


          pythongh-150771: Add bytes(m) assertions for shift_jis and euc-jp tests

Mirror test_set_text_charset_cp949 per review feedback.

Co-authored-by: Cursor <[email protected]>

serhiy-storchaka reviewed

Jun 16, 2026

View reviewed changes

Comment thread

Lib/test/test_email/test_contentmanager.py Resolved

Uh oh!

There was an error while loading. Please reload this page.


          pythongh-150771: Use Unicode escapes in bytes(m) test expectations

5736d59

Match cp949 test style per review feedback.

serhiy-storchaka merged commit 0777a58 into python:main

Jun 16, 2026

54 checks passed

Uh oh!

There was an error while loading. Please reload this page.

bedevere-app Bot removed the awaiting merge label

Jun 16, 2026

serhiy-storchaka added needs backport to 3.13

only security fixes

needs backport to 3.14

bugs and security fixes

needs backport to 3.15 bugs and security fixes labels

Jun 16, 2026

miss-islington-app Bot commented Jun 16, 2026

Copy link

Copy Markdown

Thanks @bhuvi27 for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.14.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

miss-islington-app Bot commented Jun 16, 2026

Copy link

Copy Markdown

Thanks @bhuvi27 for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.15.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

miss-islington-app Bot commented Jun 16, 2026

Copy link

Copy Markdown

Thanks @bhuvi27 for the PR, and @serhiy-storchaka for merging it 🌮🎉.. I'm working now to backport this PR to: 3.13.
🐍🍒⛏🤖 I'm not a witch! I'm not a witch!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

miss-islington-app Bot commented Jun 16, 2026

Copy link

Copy Markdown

Sorry, @bhuvi27 and @serhiy-storchaka, I could not cleanly backport this to 3.14 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 0777a58d8012bbdd0d72654b56f9112686ae6ff0 3.14

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

miss-islington-app Bot assigned serhiy-storchaka

Jun 16, 2026

miss-islington-app Bot commented Jun 16, 2026

Copy link

Copy Markdown

Sorry, @bhuvi27 and @serhiy-storchaka, I could not cleanly backport this to 3.13 due to a conflict.
Please backport using cherry_picker on command line.

cherry_picker 0777a58d8012bbdd0d72654b56f9112686ae6ff0 3.13

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bedevere-app Bot commented Jun 16, 2026

Copy link

Copy Markdown

GH-151541 is a backport of this pull request to the 3.15 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bedevere-app Bot removed the needs backport to 3.15 bugs and security fixes label

Jun 16, 2026

serhiy-storchaka pushed a commit that referenced this pull request

Jun 16, 2026


          [3.15] gh-150771: Fix email serialization for shift_jis and euc-jp (G…

ab61101

…H-151120) (GH-151541)

Encode the payload with output_charset instead of input_charset.
(cherry picked from commit 0777a58)

Co-authored-by: dev <[email protected]>

philthompson10 pushed a commit to philthompson10/cpython that referenced this pull request

Jun 17, 2026


          pythongh-150771: Fix email serialization for shift_jis and euc-jp (py…

6f4f0e3

…thonGH-151120)

Encode the payload with output_charset instead of input_charset.

StanFromIreland commented Jul 3, 2026

Copy link

Copy Markdown

Member

@bhuvi27 can you please do the backports?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bhuvi27 mentioned this pull request

Jul 3, 2026

[3.14] gh-150771: Fix email serialization for shift_jis and euc-jp (GH-151120) #152952

Merged

bedevere-app Bot commented Jul 3, 2026

Copy link

Copy Markdown

GH-152952 is a backport of this pull request to the 3.14 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bedevere-app Bot removed the needs backport to 3.14 bugs and security fixes label

Jul 3, 2026

bhuvi27 mentioned this pull request

Jul 3, 2026

[3.13] gh-150771: Fix email serialization for shift_jis and euc-jp (GH-151120) #152953

Merged

bedevere-app Bot commented Jul 3, 2026

Copy link

Copy Markdown

GH-152953 is a backport of this pull request to the 3.13 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

bedevere-app Bot removed the needs backport to 3.13 only security fixes label

Jul 3, 2026

bhuvi27 commented Jul 3, 2026

Copy link

Copy Markdown

Contributor Author

Opened backports: #152952 (3.14) and #152953 (3.13). Both drop test_set_text_charset_cp949 since it depends on gh-53144, which was not backported.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

serhiy-storchaka removed their assignment

Jul 15, 2026

Yhg1s pushed a commit that referenced this pull request

Aug 4, 2026


          [3.13] gh-150771: Fix email serialization for shift_jis and euc-jp (G…

afbced6

…H-151120) (#152953)

Encode the payload with output_charset instead of input_charset.
(cherry picked from commit 0777a58)

terryjreedy pushed a commit that referenced this pull request

Sep 8, 2026


          [3.14] gh-150771: Fix email serialization for shift_jis and euc-jp (G…

665ceff

…H-151120) (#152952)

Encode the payload with output_charset instead of input_charset.
(cherry picked from commit 0777a58)

Reviewers

serhiy-storchaka

serhiy-storchaka approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Issues with Shift_JIS and EUC-JP in email

Uh oh!

There was an error while loading. Please reload this page.

3 participants

Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

GeezerVison

Conversation

bhuvi27 commented Jun 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

bedevere-app Bot commented Jun 9, 2026

Uh oh!

serhiy-storchaka left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

miss-islington-app Bot commented Jun 16, 2026

Uh oh!

miss-islington-app Bot commented Jun 16, 2026

Uh oh!

miss-islington-app Bot commented Jun 16, 2026

Uh oh!

miss-islington-app Bot commented Jun 16, 2026

Uh oh!

miss-islington-app Bot commented Jun 16, 2026

Uh oh!

bedevere-app Bot commented Jun 16, 2026

Uh oh!

StanFromIreland commented Jul 3, 2026

Uh oh!

bedevere-app Bot commented Jul 3, 2026

Uh oh!

bedevere-app Bot commented Jul 3, 2026

Uh oh!

bhuvi27 commented Jul 3, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

bhuvi27 commented Jun 9, 2026 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.