GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Major slowdown on large listcomp · Issue #117108 · python/cpython

Major slowdown on large listcomp #117108

New issue
Copy link
New issue
Copy link
Closed
Closed
Major slowdown on large listcomp#117108
Copy link
Assignees
markshannon
Labels
interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)performancePerformance or resource usagePerformance or resource usagerelease-blockertype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

Description

@tim-one
tim-one
opened on Mar 21, 2024
Issue body actions

Bug report

Bug description:

That started within the last 48 hours, on the main branch. Here's test code; it's possible it could be simplified

if 1:
    class Obj:
        def __init__(self, i):
            self.val = i

    import sys
    print(sys.version)
    from time import perf_counter as now

    start = now()
    xs = [Obj(i) for i in range(2**21)]
    finish = now()
    print(finish - start)

Under 3.12.2 it takes about a second, but under current main about 21 seconds.

C:\Code\Python\PCbuild>py \MyPy\small.py
3.12.2 (tags/v3.12.2:6abddd9, Feb  6 2024, 21:26:36) [MSC v.1937 64 bit (AMD64)]
0.9562742999987677

C:\Code\Python\PCbuild>amd64\python.exe \MyPy\small.py
3.13.0a5+ (heads/main:f4cc77d494, Mar 20 2024, 20:28:50) [MSC v.1939 64 bit (AMD64)]
20.99507549998816

If the size of the list is increased, it appears to suffer much worse than linear-time slowdown.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Windows

Linked PRs

  • GH-117108: Change the size of the GC increment to about 1% of the total heap size. #117120
  • GH-117108: Set the "old space bit" to "visited" for all young objects  #117213

Activity

  1. tim-one
    added
    type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
    interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)
    on Mar 21, 2024
  2. hauntsaninja commented on Mar 21, 2024

    @hauntsaninja
    hauntsaninja
    on Mar 21, 2024
    Contributor
    More actions

    It looks like the regression is due to 1530932 / #116206 cc @markshannon

    Reacted by Tim Peters
  3. tim-one commented on Mar 21, 2024

    @tim-one
    tim-one
    on Mar 21, 2024
    MemberAuthor
    More actions

    Thanks - at least the cause is obvious 😆.,

  4. Eclips4
    added
    performancePerformance or resource usagePerformance or resource usage
    on Mar 21, 2024
  5. markshannon
    self-assigned this
    on Mar 21, 2024
  6. tim-one commented on Mar 21, 2024

    @tim-one
    tim-one
    on Mar 21, 2024
    MemberAuthor
    More actions

    Noting that, in the original program that triggered this, disabling gc restored its former speed.

  7. AlexWaygood
    added
    release-blocker
    and removed
    type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
    on Mar 21, 2024
  8. ezio-melotti
    added this to Release and Deferred blockers 🚫on Mar 21, 2024
  9. github-project-automation
    moved this to Todo in Release and Deferred blockers 🚫on Mar 21, 2024
  10. AlexWaygood
    added
    type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
    on Mar 22, 2024
  11. markshannon
    added a commit that references this issue on Mar 22, 2024

    GH-117108: Change the size of the GC increment to about 1% of the tot…

    e28477f
  12. gvanrossum
    added a commit that references this issue on Mar 23, 2024

    Revert "pythonGH-117108: Change the size of the GC increment to about…

    7420a55
  13. adorilson
    added a commit that references this issue on Mar 25, 2024

    pythonGH-117108: Change the size of the GC increment to about 1% of t…

    47f5202
  14. vstinner
    added a commit that references this issue on Mar 26, 2024

    GH-117108: Set the "old space bit" to "visited" for all young objects (…

    8bef34f
  15. pablogsal commented on Mar 26, 2024

    @pablogsal
    pablogsal
    on Mar 26, 2024
    Member
    More actions

    Has this been fixed by either #117120 or #117213?

  16. markshannon commented on Mar 26, 2024

    @markshannon
    markshannon
    on Mar 26, 2024
    Member
    More actions

    It was fixed by #117120

    Reacted by Victor Stinner and bluss
  17. tim-one
    closed this as completedon Mar 26, 2024
  18. github-project-automation
    moved this from Todo to Done in Release and Deferred blockers 🚫on Mar 26, 2024
  19. NGRsoftlab
    added 2 commits that reference this issue on Apr 2, 2024

    Squashed commit of the following:

    1b5fe4c

    Squashed commit of the following:

    a016a54
  20. diegorusso
    added 2 commits that reference this issue on Apr 17, 2024

    pythonGH-117108: Change the size of the GC increment to about 1% of t…

    6e59d30

    pythonGH-117108: Set the "old space bit" to "visited" for all young o…

    6ca8a3e
  21. Yhg1s
    added 2 commits that reference this issue on Sep 29, 2024

    Revert "pythonGH-117108: Set the "old space bit" to "visited" for all…

    2992289

    Revert "pythonGH-117108: Change the size of the GC increment to about…

    60a248b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

  • @markshannon
    markshannon

Labels

interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)performancePerformance or resource usagePerformance or resource usagerelease-blockertype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

Projects

  • Release and Deferred blockers 🚫
    Status
    Done
    Show more project fields

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.