GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-39040: Fix parsing of email mime headers with whitespace between encoded-words. by maxking · Pull Request #17620 · python/cpython

Conversation

@maxking

@maxking maxking commented Dec 16, 2019 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

In certain malformed content-disposition headers, parameter values are quoted
and split as encoded words on two lines with extra whitespaces. This fixes the
issue by removing the extra whitespace between the two encoded words.

https://bugs.python.org/issue39040

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@maxking
bpo-39040: Fix parsing of email headers with whitespace between encod…
9da6ddf
…ed-words.

In certain malformed content-disposition headers, parameter values are quoted
and split as encoded words on two lines with extra whitespaces. This fixes the
issue by removing the extra whitespace between the two encoded words.
@maxking
maxking requested a review from a team as a code owner December 16, 2019 02:51
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label Dec 16, 2019
@bedevere-bot bedevere-bot added the awaiting core review label Dec 16, 2019
@maxking maxking added needs backport to 3.7 type-bug An unexpected behavior, bug, or error labels Dec 16, 2019
bitdancer
bitdancer reviewed Dec 16, 2019
View reviewed changes
Comment thread Lib/email/_header_value_parser.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

bitdancer
bitdancer reviewed Dec 16, 2019
View reviewed changes
Comment thread Lib/email/_header_value_parser.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

bitdancer
bitdancer reviewed Dec 16, 2019
View reviewed changes
Comment thread Lib/test/test_email/test_headerregistry.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

bitdancer
bitdancer reviewed Dec 16, 2019
View reviewed changes
Comment thread Misc/NEWS.d/next/Library/2019-12-15-18-47-20.bpo-39040.tKa0Qs.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

bitdancer
bitdancer requested changes Dec 16, 2019
View reviewed changes

@bitdancer bitdancer left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, I forgot to start a review :(

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-bot bedevere-bot added awaiting changes and removed awaiting core review labels Dec 16, 2019
@bedevere-bot

bedevere-bot commented Dec 16, 2019

Copy link
Copy Markdown

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@maxking maxking changed the title bpo-39040: Fix parsing of email headers with whitespace between encoded-words. bpo-39040: Fix parsing of email mime headers with whitespace between encoded-words. Dec 17, 2019
@maxking
Remove empty lines, add a new test and update the news entry.
bf2cb76
@maxking

maxking commented Dec 17, 2019

Copy link
Copy Markdown
Contributor Author

I have made the requested changes; please review again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot added awaiting change review and removed awaiting changes labels Dec 17, 2019
@bedevere-bot

bedevere-bot commented Dec 17, 2019

Copy link
Copy Markdown

Thanks for making the requested changes!

@bitdancer: please review the changes made to this pull request.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot
bedevere-bot requested a review from bitdancer December 17, 2019 03:29
bitdancer
bitdancer requested changes Dec 17, 2019
View reviewed changes
Comment thread Lib/test/test_email/test_headerregistry.py Outdated
[],
'attachment; filename="File Name With Spaces.pdf"',
('Content-Disposition: attachment; '
'filename="File Name With Spaces.pdf"\n'),

@bitdancer bitdancer Dec 17, 2019

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not quite. We need a test with something like "File =?utf-8?q?Name?= With Spaces.pdf". That should have spaces around Name...we want to make sure we aren't removing spaces around encoded words that aren't next to other encoded words.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@maxking maxking Dec 18, 2019

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@bitdancer bitdancer Dec 24, 2019

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not seeing the changes?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@maxking maxking Dec 24, 2019

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You might be looking at the outdated diff, from a previous commit.

There should be 4 commits in the PR, and your requested changes are in the last 2 commits.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-bot bedevere-bot added awaiting changes and removed awaiting change review labels Dec 17, 2019
@bedevere-bot

bedevere-bot commented Dec 17, 2019

Copy link
Copy Markdown

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

maxking added 2 commits December 17, 2019 17:33
@maxking
Update test to include spaces around an encoded word.
004f37d
@maxking
Fix the encoded word syntax and resultant error.
016ceb3
@maxking

maxking commented Dec 18, 2019

Copy link
Copy Markdown
Contributor Author

I have made the requested changes; please review again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the awaiting changes label Dec 18, 2019
@bedevere-bot bedevere-bot added the awaiting change review label Dec 18, 2019
@bedevere-bot

bedevere-bot commented Dec 18, 2019

Copy link
Copy Markdown

Thanks for making the requested changes!

@bitdancer: please review the changes made to this pull request.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot
bedevere-bot requested a review from bitdancer December 18, 2019 01:38
@csabella
csabella removed the request for review from bitdancer May 28, 2020 23:24
@csabella csabella added the needs backport to 3.9 label May 28, 2020
@csabella
csabella requested a review from bitdancer May 28, 2020 23:24
bitdancer
bitdancer approved these changes May 28, 2020
View reviewed changes
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting change review labels May 28, 2020
@bitdancer
bitdancer merged commit 21017ed into python:master May 29, 2020
@miss-islington

miss-islington commented May 29, 2020

Copy link
Copy Markdown
Contributor

Thanks @maxking for the PR, and @bitdancer for merging it 🌮🎉.. I'm working now to backport this PR to: 3.7, 3.8, 3.9.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the awaiting merge label May 29, 2020
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request May 29, 2020
@maxking @miss-islington
bpo-39040: Fix parsing of email mime headers with whitespace between …
b265321
…encoded-words. (pythongh-17620)

* bpo-39040: Fix parsing of email headers with encoded-words inside a quoted string.

It is fairly common to find malformed mime headers (especially content-disposition
headers) where the parameter values, instead of being encoded to RFC
standards, are "encoded" by doing RFC 2047 "encoded word" encoding, and
then enclosing the whole thing in quotes.  The processing of these malformed
headers was incorrectly leaving the spaces between encoded words in the decoded
text (whitespace between adjacent encoded words is supposed to be stripped on
decoding).  This changeset fixes the encoded word processing inside quoted strings
(bare-quoted-string) to do correct RFC 2047 decoding by stripping that
whitespace.
(cherry picked from commit 21017ed)

Co-authored-by: Abhilash Raj <[email protected]>
@bitdancer bitdancer removed needs backport to 3.7 labels May 29, 2020
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request May 29, 2020
@maxking @miss-islington
bpo-39040: Fix parsing of email mime headers with whitespace between …
569b03b
…encoded-words. (pythongh-17620)

* bpo-39040: Fix parsing of email headers with encoded-words inside a quoted string.

It is fairly common to find malformed mime headers (especially content-disposition
headers) where the parameter values, instead of being encoded to RFC
standards, are "encoded" by doing RFC 2047 "encoded word" encoding, and
then enclosing the whole thing in quotes.  The processing of these malformed
headers was incorrectly leaving the spaces between encoded words in the decoded
text (whitespace between adjacent encoded words is supposed to be stripped on
decoding).  This changeset fixes the encoded word processing inside quoted strings
(bare-quoted-string) to do correct RFC 2047 decoding by stripping that
whitespace.
(cherry picked from commit 21017ed)

Co-authored-by: Abhilash Raj <[email protected]>
@bitdancer bitdancer added the needs backport to 3.8 label May 29, 2020
@miss-islington

miss-islington commented May 29, 2020

Copy link
Copy Markdown
Contributor

Thanks @maxking for the PR, and @bitdancer for merging it 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

miss-islington added a commit that referenced this pull request May 29, 2020
@miss-islington @maxking
bpo-39040: Fix parsing of email mime headers with whitespace between …
a6ae02d
…encoded-words. (gh-17620)

* bpo-39040: Fix parsing of email headers with encoded-words inside a quoted string.

It is fairly common to find malformed mime headers (especially content-disposition
headers) where the parameter values, instead of being encoded to RFC
standards, are "encoded" by doing RFC 2047 "encoded word" encoding, and
then enclosing the whole thing in quotes.  The processing of these malformed
headers was incorrectly leaving the spaces between encoded words in the decoded
text (whitespace between adjacent encoded words is supposed to be stripped on
decoding).  This changeset fixes the encoded word processing inside quoted strings
(bare-quoted-string) to do correct RFC 2047 decoding by stripping that
whitespace.
(cherry picked from commit 21017ed)

Co-authored-by: Abhilash Raj <[email protected]>
miss-islington added a commit that referenced this pull request May 29, 2020
@miss-islington @maxking
bpo-39040: Fix parsing of email mime headers with whitespace between …
6381ee0
…encoded-words. (gh-17620)

* bpo-39040: Fix parsing of email headers with encoded-words inside a quoted string.

It is fairly common to find malformed mime headers (especially content-disposition
headers) where the parameter values, instead of being encoded to RFC
standards, are "encoded" by doing RFC 2047 "encoded word" encoding, and
then enclosing the whole thing in quotes.  The processing of these malformed
headers was incorrectly leaving the spaces between encoded words in the decoded
text (whitespace between adjacent encoded words is supposed to be stripped on
decoding).  This changeset fixes the encoded word processing inside quoted strings
(bare-quoted-string) to do correct RFC 2047 decoding by stripping that
whitespace.
(cherry picked from commit 21017ed)

Co-authored-by: Abhilash Raj <[email protected]>
miss-islington added a commit that referenced this pull request May 29, 2020
@miss-islington @maxking
bpo-39040: Fix parsing of email mime headers with whitespace between …
5f977e0
…encoded-words. (gh-17620)

* bpo-39040: Fix parsing of email headers with encoded-words inside a quoted string.

It is fairly common to find malformed mime headers (especially content-disposition
headers) where the parameter values, instead of being encoded to RFC
standards, are "encoded" by doing RFC 2047 "encoded word" encoding, and
then enclosing the whole thing in quotes.  The processing of these malformed
headers was incorrectly leaving the spaces between encoded words in the decoded
text (whitespace between adjacent encoded words is supposed to be stripped on
decoding).  This changeset fixes the encoded word processing inside quoted strings
(bare-quoted-string) to do correct RFC 2047 decoding by stripping that
whitespace.
(cherry picked from commit 21017ed)

Co-authored-by: Abhilash Raj <[email protected]>
CuriousLearner added a commit to CuriousLearner/cpython that referenced this pull request May 30, 2020
@CuriousLearner
Merge branch 'master' of github.com:python/cpython into bpo-26543
d3d7100
* 'master' of github.com:python/cpython: (497 commits)
  bpo-40061: Fix a possible refleak in _asynciomodule.c (pythonGH-19748)
  bpo-40798: Generate a different message for already removed elements (pythonGH-20483)
  closes bpo-29017: Update the bindings for Qt information with PySide2 (pythonGH-20149)
  bpo-39885: Make IDLE context menu cut and copy work again (pythonGH-18951)
  bpo-29882: Add an efficient popcount method for integers (python#771)
  Further de-linting of zoneinfo module (python#20499)
  bpo-40780: Fix failure of _Py_dg_dtoa to remove trailing zeros (pythonGH-20435)
  Indicate that abs() method accept argument that implement __abs__(), just like call() method in the docs (pythonGH-20509)
  bpo-39040: Fix parsing of email mime headers with whitespace between encoded-words. (pythongh-17620)
  bpo-40784: Fix sqlite3 deterministic test (pythonGH-20448)
  bpo-30064: Properly skip unstable loop.sock_connect() racing test (pythonGH-20494)
  Note the output ordering of combinatoric functions (pythonGH-19732)
  bpo-40474: Updated coverage.yml to better report coverage stats (python#19851)
  bpo-40806: Clarify that itertools.product immediately consumes its inpt (pythonGH-20492)
  bpo-1294959: Try to clarify the meaning of platlibdir (pythonGH-20332)
  bpo-37878: PyThreadState_DeleteCurrent() was not removed (pythonGH-20489)
  bpo-40777: Initialize PyDateTime_IsoCalendarDateType.tp_base at run-time (pythonGH-20493)
  bpo-40755: Add missing multiset operations to Counter() (pythonGH-20339)
  bpo-25920: Remove socket.getaddrinfo() lock on macOS (pythonGH-20177)
  bpo-40275: Fix test.support.threading_helper (pythonGH-20488)
  ...
@flokli flokli mentioned this pull request Oct 15, 2021
bpo-28764: mailbox.mbox: handle lines with non-ascii more graceful #23553
Closed
@manfred-kaiser manfred-kaiser mannequin mentioned this pull request Jun 19, 2022
Wrong attachement filename when mail mime header was too long #83221
Closed
@terryjreedy terryjreedy removed the needs backport to 3.9 label Feb 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bitdancer bitdancer bitdancer approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

type-bug An unexpected behavior, bug, or error

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

7 participants

@maxking @bedevere-bot @miss-islington @bitdancer @the-knights-who-say-ni @terryjreedy @csabella
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.