GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-150942: Optimize stringlib split/splitlines with _PyList_AppendTakeRef by danielKim614 · Pull Request #155922 · python/cpython

Conversation

@danielKim614

@danielKim614 danielKim614 commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

Append result items to the output list with _PyList_AppendTakeRef instead of PyList_Append followed by Py_DECREF in the stringlib SPLIT_ADD (past the 12-slot prealloc) and SPLIT_APPEND (used by splitlines() for every line) macros, removing a reference-count round-trip per appended item (and a per-append lock on the free-threaded build). This covers split/rsplit/splitlines for str, bytes and bytearray. The result list is freshly allocated and stays local to the split function until it is returned.

Microbenchmarks

  • Plain release build, macOS arm64
  • Interleaved A/B runs (8 rounds alternating baseline/patched binaries), medians
Benchmark workload main this PR speedup
str_split_10k_pieces ("abc " * 10_000).split(" ") 165 us 152 us 1.09×
str_split_whitespace_10k ("word " * 10_000).split() 184 us 168 us 1.10×
str_splitlines_10k ("line\n" * 10_000).splitlines() 182 us 165 us 1.10×
bytes_split_10k_pieces (b"abc " * 10_000).split(b" ") 148 us 139 us 1.06×
geomean 1.09×

test_str and test_bytes pass, including -R 3:3 refleak runs.

Benchmark script
"""Microbenchmark str/bytes split / rsplit / splitlines (result-list building)."""
import pyperf

# Many small pieces (>12): exercises the append path after the 12-slot prealloc.
import pyperf

# Many small pieces (>12): exercises the append path after the 12-slot prealloc.
STR_MANY = "abc " * 10_000            # .split(" ") -> 10001 pieces
STR_WS = "word  " * 10_000            # .split() whitespace variant
BYTES_MANY = b"abc " * 10_000         # bytes.split
LINES = "line\n" * 10_000             # .splitlines(): SPLIT_APPEND for every line

# Controls
STR_FEW = "a b c d e f g h i j"       # 10 pieces: stays in the SET_ITEM prealloc path
STR_BIG = ("x" * 4096 + " ") * 250    # large pieces: copy-dominated


def bench_method(loops, obj, name, *args):
    method = getattr(obj, name)
    range_it = range(loops)
    t0 = pyperf.perf_counter()
    for _ in range_it:
        method(*args)
    return pyperf.perf_counter() - t0


runner = pyperf.Runner()
runner.bench_time_func("str_split_10k_pieces", bench_method, STR_MANY, "split", " ")
runner.bench_time_func("str_split_whitespace_10k", bench_method, STR_WS, "split")
runner.bench_time_func("str_splitlines_10k", bench_method, LINES, "splitlines")
runner.bench_time_func("bytes_split_10k_pieces", bench_method, BYTES_MANY, "split", b" ")
runner.bench_time_func("str_split_10_pieces_ctl", bench_method, STR_FEW, "split", " ")
runner.bench_time_func("str_split_large_pieces_ctl", bench_method, STR_BIG, "split", " ")
  • Issue: Improve performance by using reference stealing methods #150942

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@danielKim614
pythongh-150942: Optimize stringlib split/splitlines with _PyList_App…
1430948
…endTakeRef
@python-cla-bot

python-cla-bot Bot commented Aug 17, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

CLA signed

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot mentioned this pull request Aug 17, 2026
Improve performance by using reference stealing methods #150942
Open
@bedevere-app bedevere-app Bot added the awaiting review label Aug 17, 2026
@corona10 corona10 self-assigned this Aug 17, 2026
@corona10 corona10 added the skip news label Aug 17, 2026
@corona10
Delete Misc/NEWS.d/next/Core_and_Builtins/2026-08-17-14-00-00.gh-issu…
f3fd338
…e-150942.sTrSpL.rst
corona10
corona10 approved these changes Aug 17, 2026
View reviewed changes

@corona10 corona10 left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

danielKim614 reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@bedevere-app bedevere-app Bot added awaiting merge and removed awaiting review labels Aug 17, 2026
@corona10 corona10 added the sprint label Aug 17, 2026
@github-project-automation github-project-automation Bot added this to Sprint Aug 17, 2026
@github-project-automation github-project-automation Bot moved this to Todo in Sprint Aug 17, 2026
@github-project-automation github-project-automation Bot moved this from Todo to In Progress in Sprint Aug 17, 2026
@corona10 corona10 added the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Aug 17, 2026
@bedevere-bot

bedevere-bot commented Aug 17, 2026

Copy link
Copy Markdown

🤖 New build scheduled with the buildbot fleet by @corona10 for commit f3fd338 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F155922%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-refleak-buildbots label again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the 🔨 test-with-refleak-buildbots Test PR w/ refleak buildbots; report in status section label Aug 17, 2026
@corona10
corona10 merged commit 07a9427 into python:main Aug 17, 2026
61 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@github-project-automation github-project-automation Bot moved this from In Progress to Done in Sprint Aug 17, 2026
@bedevere-app bedevere-app Bot removed the awaiting merge label Aug 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@corona10 corona10 corona10 approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

@corona10 corona10

Labels

skip news sprint

Projects

Sprint
Archived in project

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@danielKim614 @bedevere-bot @corona10
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.