GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — [3.11] gh-135034: Normalize link targets in tarfile, add `os.path.realpath(strict='allow_missing')` (GH-135037) by Yhg1s · Pull Request #135068 · python/cpython

Conversation

Yhg1s commented Jun 3, 2025 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Member

Addresses CVEs 2024-12718, 2025-4138, 2025-4330, and 2025-4517.
(cherry picked from commit 3612d8f)

Co-authored-by: Łukasz Langa [email protected]
Co-authored-by: Petr Viktorin [email protected]
Co-authored-by: Seth Michael Larson [email protected]
Co-authored-by: Adam Turner [email protected]
Co-authored-by: Serhiy Storchaka [email protected]

Issue: Multiple tarfile extraction filter bypasses (filter="tar"/filter="data") #135034

📚 Documentation preview 📚: https://cpython-previews--135068.org.readthedocs.build/

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

ambv and others added 2 commits

June 3, 2025 13:35


          [3.11] pythongh-135034: Normalize link targets in tarfile, add `os.pa…

371b4ea

…th.realpath(strict='allow_missing')` (pythonGH-135037)

Addresses CVEs 2024-12718, 2025-4138, 2025-4330, and 2025-4517.
(cherry picked from commit 3612d8f)
(cherry picked from commit c358142)

Co-authored-by: Łukasz Langa <[email protected]>
Signed-off-by: Łukasz Langa <[email protected]>
Co-authored-by: Petr Viktorin <[email protected]>
Co-authored-by: Seth Michael Larson <[email protected]>
Co-authored-by: Adam Turner <[email protected]>
Co-authored-by: Serhiy Storchaka <[email protected]>


          Fix test failure in test_tarfile, remove tests that aren't suitable for

698a6d4

security backports.

Yhg1s requested a review from ethanfurman as a code owner

June 3, 2025 11:45

bedevere-app Bot added the awaiting core review label

Jun 3, 2025

This was referenced Jun 3, 2025

gh-135034: Normalize link targets in tarfile, add os.path.realpath(strict='allow_missing') #135037

Merged

Multiple tarfile extraction filter bypasses (filter="tar"/filter="data") #135034

Closed

Yhg1s added 5 commits

June 3, 2025 14:10


          Fix a few tests, and a glaring logic error in the posixpath backport.


          Stop using :cve:, which doesn't work in 3.11 and earlier.

8ab383f


          Fix test_tarfile (for not exporting LinkFallbackError) and remove the

6d4d4d6

non-strict test for unreadable symlinks, whch isn't supported before 3.13.


          Fix Windows test_ntpath failures.

630ffa3


          Fix merge error in backport.

28d8d22

Yhg1s requested review from ambv, encukou and pablogsal

June 3, 2025 13:42


          Remove backport of a test that won't pass under 3.11.

92a788a

encukou approved these changes

Jun 3, 2025

View reviewed changes

bedevere-app Bot added awaiting merge and removed awaiting core review labels

Jun 3, 2025

ambv merged commit 4633f3f into python:3.11

Jun 3, 2025

bedevere-app Bot removed the awaiting merge label

Jun 3, 2025

kulikjak commented Jun 4, 2025

Copy link

Copy Markdown

Contributor

After this update, I am getting the following test failure:

======================================================================
ERROR: test_realpath_limit_attack (test.test_tarfile.TestExtractionFilters.test_realpath_limit_attack) [fully_trusted]
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/test/test_tarfile.py", line 3669, in test_realpath_limit_attack
    with (self.subTest('fully_trusted'),
  File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/contextlib.py", line 137, in __enter__
    return next(self.gen)
           ^^^^^^^^^^^^^^
  File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/test/test_tarfile.py", line 3458, in check_context
    self.expected_paths = set(self.outerdir.glob('**/*'))
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/pathlib.py", line 958, in glob
    for p in selector.select_from(self):
  File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/pathlib.py", line 411, in _select_from
    for starting_point in self._iterate_directories(parent_path, is_dir, scandir):
  File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/pathlib.py", line 401, in _iterate_directories
    for p in self._iterate_directories(path, is_dir, scandir):
  File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/pathlib.py", line 401, in _iterate_directories
    for p in self._iterate_directories(path, is_dir, scandir):
  File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/pathlib.py", line 401, in _iterate_directories
    for p in self._iterate_directories(path, is_dir, scandir):
  [Previous line repeated 14 more times]
  File "/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/Python-3.11.13/Lib/pathlib.py", line 395, in _iterate_directories
    entry_is_dir = entry.is_dir(follow_symlinks=False)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
OSError: [Errno 78] File name too long: '/builds/jkulik/python-3.13.4-3.11.13-3.9.23/components/python/python311/build/amd64/build/test_python_22904æ/@test_22904_tmpæ-tardir/outerdir/dest/ddddddddddddddddddd
dddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/dddddddddddddddddd
ddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddd
dddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/dddddddddddddddd
ddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/ddddddddddddddddddddddddddddddddddddddddddddddddddd/lllllllllllllll
lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll
llllllllllllllllllllllllllllllll'

Interestingly, it's just 3.11.13 that's failing - both 3.9.23 and 3.13.4 (and all other supported versions we are running in our internal buildbot) pass. 3.13 has a very different pathlib implementation, but 3.9 is pretty similar so I investigated the differences, and when I print the entires from here (_iterate_directories 3.9 / 3.11):

            with scandir(parent_path) as scandir_it:
                entries = list(scandir_it)

I get very different results:
3.9:

[<DirEntry 'newfile'>, <DirEntry 'flag'>, <DirEntry 'dest'>]
[<DirEntry 'a'>, <DirEntry 'flaglink'>, <DirEntry 'escape'>]

3.11:

[<DirEntry 'dest'>, <DirEntry 'flag'>, <DirEntry 'newfile'>]
[<DirEntry 'a'>, <DirEntry 'escape'>, <DirEntry 'flaglink'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>]
[<DirEntry 'b'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>]
[<DirEntry 'c'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>]
[<DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>, <DirEntry 'd'>]
[<DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>, <DirEntry 'e'>]
[<DirEntry 'f'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>]
[<DirEntry 'g'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>]
[<DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>, <DirEntry 'h'>]
[<DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>, <DirEntry 'i'>]
[<DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>, <DirEntry 'j'>]
[<DirEntry 'k'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>]
[<DirEntry 'l'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>]
[<DirEntry 'm'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>]
[<DirEntry 'n'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>]
[<DirEntry 'o'>, <DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>]
[<DirEntry 'ddddddddddddddddddddddddddddddddddddddddddddddddddd'>, <DirEntry 'p'>]
[<DirEntry 'lllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllllll
lllllllllllllllllllllllllllllllllllllllllllllllllllllllllll'>]

Also, the scandir argument is different:
3.9: <built-in function scandir>
3.11: <function Path._scandir at 0x7fed9bd923e0>
so I guess that those implementations behave differently (the 3.11 one returns much more entires), which causes the failure?

This is on Oracle Solaris.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

encukou commented Jun 4, 2025

Copy link

Copy Markdown

Member

Ah! We got the same test failure in 3.10, so I hotfixed it there. Try applying the hack to 3.11: dff62a1

It's just a test failure: if the filter is fully_trusted, this archive unpacks to a very interesting directory tree -- one that this version of Path.glob can't handle. And we call glob in the tests to get data for the assertions.

kulikjak reacted with hooray emoji

All reactions

🎉 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

kulikjak commented Jun 4, 2025 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Contributor

Oh, that fixed it. Thanks!

(i didn't realize that this difference between 3.9 and 3.11 patch is what causes the issue)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

DimNik9 commented Jun 5, 2025

Copy link

Copy Markdown

Hello, sorry to ask directly here but i haven't understood how 3.10 & 3.11 are affected by these CVEs, since the description of each CVE states that ONLY versions after 3.12 are affected, since the extraction filters were first introduced in 3.12
Thank you in advance

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

encukou commented Jun 5, 2025

Copy link

Copy Markdown

Member

yeah, looks like the CVE text needs an update :(
cc @sethmlarson
The feature was backported to 3.8.17, 3.9.17, 3.10.12, 3.11.4

sethmlarson and DimNik9 reacted with thumbs up emoji

All reactions

👍 2 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

sethmlarson commented Jun 5, 2025

Copy link

Copy Markdown

Contributor

@encukou Gotcha, I didn't realize that filtering had been backported, I'll update the prose description in each document. The affectedness of the CVEs is correct thankfully :)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

sethmlarson commented Jun 5, 2025

Copy link

Copy Markdown

Contributor

Updated the CVE records and sent a correction to [email protected].

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Yhg1s deleted the backport-c358142-3.11 branch

August 13, 2026 12:52

Reviewers

encukou

encukou approved these changes

ethanfurman

Awaiting requested review from ethanfurman

ethanfurman is a code owner

ambv

Awaiting requested review from ambv

pablogsal

Awaiting requested review from pablogsal

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

6 participants

Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

GeezerVison

Conversation

Yhg1s commented Jun 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kulikjak commented Jun 4, 2025

Uh oh!

encukou commented Jun 4, 2025

Uh oh!

kulikjak commented Jun 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DimNik9 commented Jun 5, 2025

Uh oh!

encukou commented Jun 5, 2025

Uh oh!

sethmlarson commented Jun 5, 2025

Uh oh!

sethmlarson commented Jun 5, 2025

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Yhg1s commented Jun 3, 2025 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

kulikjak commented Jun 4, 2025 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.