GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-28837: Fix lib2to3 handling of map/zip/filter by stuarteberg · Pull Request #24 · python/cpython

Conversation

@stuarteberg

@stuarteberg stuarteberg commented Feb 11, 2017 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

The 2to3 tool will automatically wrap calls to map, zip, and filter with list():

# Original:
zip('abc', '123')

# After 2to3:
list(zip('abc', '123'))

...but the tool misses some cases, if the call is followed by a "trailer" such as [0]:

# Skipped by 2to3:
zip('abc', '123')[0]

This PR augments the lib2to3 "fixers" for map, zip, and filter to correctly handle such cases.

http://bugs.python.org/issue28837

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@stuarteberg
Issue python#28837: Fix lib2to3 handling of map/zip/filter calls when…
5cdd2db
… followed with a 'trailer', e.g. zip()[x]
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label Feb 11, 2017
@codecov

codecov Bot commented Feb 11, 2017

Copy link
Copy Markdown

Codecov Report

Merging #24 into master will increase coverage by <.01%.

@@            Coverage Diff             @@
##           master      #24      +/-   ##
==========================================
+ Coverage   82.37%   82.37%   +<.01%     
==========================================
  Files        1427     1427              
  Lines      350948   351009      +61     
==========================================
+ Hits       289088   289144      +56     
- Misses      61860    61865       +5

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e7ffb99...5cdd2db. Read the comment docs.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@stuarteberg

stuarteberg commented Feb 15, 2017

Copy link
Copy Markdown
Contributor Author

@benjaminp: Are you the right person to ping about lib2to3 PRs?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@rhettinger
rhettinger requested a review from benjaminp April 5, 2017 05:48
benjaminp
benjaminp reviewed Apr 5, 2017
View reviewed changes
Comment thread Lib/lib2to3/fixes/fix_map.py
|
power<
'map' trailer< '(' [arglist=any] ')' >
'map' args=trailer< '(' [any] ')' >

@benjaminp benjaminp Apr 5, 2017

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the reason for this change?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@stuarteberg stuarteberg Apr 5, 2017

Copy link
Copy Markdown
Contributor Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the reason for this change?

On line 100 (below), I need to clone the argument list:

new = Node(syms.power, [Name("map"), args.clone()])

If I leave the above line unchanged, then I could have written this, instead:

new = Node(syms.power, [Name("map"), ArgList([args.clone()])])

...but there's a problem with that: It doesn't clone the ( and ) tokens -- it creates them from scratch. As a result, one of the tests fails, due to an unpreserved prefix before the ) token:

======================================================================
FAIL: test_prefix_preservation (Lib.lib2to3.tests.test_fixers.Test_map)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/bergs/Documents/workspace/cpython-git/Lib/lib2to3/tests/test_fixers.py", line 3036, in test_prefix_preservation
    self.check(b, a)
  File "/Users/bergs/Documents/workspace/cpython-git/Lib/lib2to3/tests/test_fixers.py", line 3031, in check
    super(Test_map, self).check(b, a)
  File "/Users/bergs/Documents/workspace/cpython-git/Lib/lib2to3/tests/test_fixers.py", line 36, in check
    tree = self._check(before, after)
  File "/Users/bergs/Documents/workspace/cpython-git/Lib/lib2to3/tests/test_fixers.py", line 32, in _check
    self.assertEqual(after, str(tree))
AssertionError: "x =    list(map(   f,    'abc'   ))\n\n" != "x =    list(map(   f,    'abc'))\n\n"
- x =    list(map(   f,    'abc'   ))
?                               ---
+ x =    list(map(   f,    'abc'))

Hence, I think it's best to capture and clone the whole trailer. But if I'm missing something, please let me know.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@benjaminp benjaminp Apr 6, 2017

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, thanks for the explanation.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@benjaminp
benjaminp merged commit 93b4b47 into python:master Apr 6, 2017
Mariatta referenced this pull request in Mariatta/cpython Jun 16, 2017
@stuarteberg @Mariatta
[3.6] bpo-28837: Fix lib2to3 handling of map/zip/filter calls when fo…
bac6964
…llowed with a 'trailer', e.g. zip()[x] (GH-24)

(cherry picked from commit 93b4b47)
@Mariatta Mariatta mentioned this pull request Jun 16, 2017
[3.6] bpo-28837: Fix lib2to3 handling of map/zip/filter calls when fo… #2235
Merged
Mariatta added a commit that referenced this pull request Jun 16, 2017
@Mariatta
bpo-28837: Fix lib2to3 handling of map/zip/filter calls when followed…
292b421
… with a 'trailer', e.g. zip()[x] (GH-24) (GH-2235)

(cherry picked from commit 93b4b47)
@pablogsal pablogsal mentioned this pull request Jun 12, 2020
bpo-40958: Avoid buffer overflow in the parser when indexing the current line #20842
Closed
@itachaaa itachaaa mentioned this pull request Aug 22, 2022
Python 3.10 hang at exit in drop_gil() (due to resource warning at exit?) #91414
Closed
@mdboom mdboom mentioned this pull request Aug 25, 2022
Assert and incorrect error message when loading source file containing invalid UTF-8 #96268
Closed
@mdboom mdboom mentioned this pull request Nov 22, 2022
Type punning (and strict aliasing) issue in Py_CLEAR() and Py_SETREF() macros: Python --enable-pystats is miscompiled #99701
Closed
@ziegenbalg ziegenbalg mentioned this pull request Nov 30, 2022
double free in io.TextIOWrapper #72573
Closed
jaraco pushed a commit that referenced this pull request Dec 2, 2022
@Mariatta
Delete a branch when PR is closed. (GH-24)
1f08664
jaraco pushed a commit to jaraco/cpython that referenced this pull request Feb 17, 2023
@warsaw
Support Python 2.7 (python#24)
64a9e9f
Refactor into Python 3 and Python 2.7 submodules.
nanjekyejoannah added a commit to nanjekyejoannah/cpython that referenced this pull request Mar 10, 2023
@bors @nanjekyejoannah
Merge python#24
1f12b38
24: Warn for exception: move warning to ceval r=ltratt a=nanjekyejoannah

This replaces the old PR: softdevteam#12
Moved the warning to ceval.

I removed the three component warning because it was committed in an earlier PR here: softdevteam#14

Co-authored-by: Joannah Nanjekye <[email protected]>
Fidget-Spinner referenced this pull request in pylbbv/pylbbv May 27, 2023
@Fidget-Spinner
Merge pull request #24 from JuliaPoo/types_metainterpreter
eeaa02f
Implemented type propagation
oraluben pushed a commit to oraluben/cpython that referenced this pull request Jun 25, 2023
@Fidget-Spinner
fix: bb_test_pop_if_false not setting flag (python#24)
7cab316
@gvanrossum gvanrossum mentioned this pull request Aug 22, 2023
heap-use-after-free in _PyFunction_LookupByVersion #108253
Closed
@LinanV LinanV mentioned this pull request Oct 23, 2023
Objects/typeobject.c: No such file or directory. #111203
Closed
@stasos24 stasos24 mentioned this pull request Oct 24, 2023
Modules/cjkcodecs/_codecs_iso2022.c - read out of bounds #101180
Closed
@williamhu020 williamhu020 mentioned this pull request Nov 5, 2023
Use the API C of 'Py-NewInterpreterFromConfig' to exit unexpectedly in multiple threads. #111751
Closed
@kcatss kcatss mentioned this pull request Nov 15, 2023
Use-after-free in unregister() of atexit module #112127
Closed
@kcatss kcatss mentioned this pull request Jan 14, 2024
crash in long_vectorcall in longobject.c #114050
Closed
@kcatss kcatss mentioned this pull request Feb 12, 2024
Segmentation Fault in pthread_getcpuclockid function in time module #115378
Closed
@kcatss kcatss mentioned this pull request Feb 20, 2024
Use After Free at _heapreplace_max #115706
Closed
@ngoldbaum ngoldbaum mentioned this pull request Sep 23, 2024
Crash running PyO3 tests with --test-threads=1000 #124375
Closed
@Dean6767 Dean6767 mentioned this pull request Feb 10, 2025
🚨 Segmentation Fault in Python 3.10.12 when using OpenSSL (libcrypto.so.3) #129974
Closed
This was referenced Feb 11, 2025
segmentfault when pip installing setuptools #129992
Closed
segmentfault when pip installing setuptools #129993
Closed
@DominiquePACCO DominiquePACCO mentioned this pull request Jun 3, 2025
PdfWriter add_page adds (re)writes old page instead of new #135062
Closed
@efimov-mikhail efimov-mikhail mentioned this pull request Nov 5, 2025
memory leak in threading stack size #141044
Open
StanFromIreland referenced this pull request in StanFromIreland/cpython Dec 6, 2025
@pablogsal
Merge pull request #24 from StanFromIreland/lazy-color
4cd4322
@sergey-miryanov sergey-miryanov mentioned this pull request Dec 18, 2025
Observed memory leak in ssl library: Python 3.14 GC issue #142516
Closed
@Qanux Qanux mentioned this pull request Feb 11, 2026
heap-buffer-overflow in functools.partial.__repr__() #144475
Closed
@ngoldbaum ngoldbaum mentioned this pull request Mar 6, 2026
Remove deprecated and undocumented function ctypes.SetPointerType #133866
Closed
Eclips4 pushed a commit to Eclips4/cpython that referenced this pull request Mar 10, 2026
@emmatyping
Implement the Unix build system refactor (python#24)
95753bb
This PR pretty much completely refactors the Rust build system. There are now two main ways Rust code is built:

# both

- The Rust triple is deduced based on the preprocessor run on Misc/platform_triplet.c
- A cpython-build-helper crate is used to pass the proper link arguments for each configuration
- The proper Rust toolchain is downloaded for iOS/Android/WASI for CI on those platforms
- cpython-sys is updated significantly to ensure bindgen properly generates the bindings for each platform

# shared

For shared builds, we build a crate into a cdylib and pass through link arguments from the makefile/configure and the linker executable so that the final link will match what is done for C programs.

# static

For static builds, a cpython-rust-staticlib crate is a built which depends on and re-exports the module initializers for each crate. This ensures there aren't duplicated Rust stdlib/core symbols.

Fixes python#23
raul-sq added a commit to raul-sq/cpython that referenced this pull request Jul 18, 2026
@raul-sq
Colon added on line python#24
73ef9bc
@johng johng mentioned this pull request Jul 19, 2026
use-after-free: functools partial #154189
Closed
@djarfluka djarfluka mentioned this pull request Sep 12, 2026
Heap UAF in io.TextIOWrapper read path via reentrant detach() #157363
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@benjaminp benjaminp benjaminp left review comments

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@stuarteberg @benjaminp @the-knights-who-say-ni
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.