GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — [3.10] gh-98820: Fix quadratic time in csv.Sniffer for quoted fields (GH-154867) by serhiy-storchaka · Pull Request #155539 · python/cpython

Conversation

@serhiy-storchaka

@serhiy-storchaka serhiy-storchaka commented Aug 11, 2026

Copy link
Copy Markdown
Member

Manual backport of #154867, cherry-picked from the 3.12 backport (#155166), with one 3.10-only change in the second commit.

Possessive quantifiers were added in 3.11, so (?:(?P=quote){2}|(?!(?P=quote)).)*+ does not compile here. The body is matched as an unrolled loop instead; since the quote character is a backreference, the "not a quote" atom needs a lookahead:

other = r'(?:(?!(?P=quote)).)*'
body = r'%s(?:(?P=quote){2}%s)*' % (other, other)

Every position matches only one of the two alternatives, and each iteration of the outer loop consumes the two quote characters, so it does not backtrack.

Checked against the 3.11 version of the file: no difference in the sniffed dialect on 90000 samples, and the same timings — sniffing '"abcdefghijklmnopqrstuvwxyz"\n' * n with delimiters=',:|\t' takes 0.147, 0.298 and 0.602 s for n = 2500, 5000 and 10000, against 0.142, 0.286 and 0.569 s on 3.11.

  • Issue: csv.Sniffer().sniff() Hangs on Single-Column Data #98820

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
miss-islington and others added 2 commits August 11, 2026 09:44
@miss-islington @serhiy-storchaka
[3.10] pythongh-98820: Fix quadratic time in csv.Sniffer for quoted f…
2be8f83
…ields (pythonGH-154867)

(cherry picked from commit b30c7fa)
(cherry picked from commit 39dac55)
@serhiy-storchaka @claude
Match the body of a quoted field without possessive quantifiers
413b9d6
They were added in 3.11.  An unrolled loop is unambiguous at every
position, so it does not backtrack either.

Co-Authored-By: Claude Opus 5 (1M context) <[email protected]>
@bedevere-app bedevere-app Bot mentioned this pull request Aug 11, 2026
csv.Sniffer().sniff() Hangs on Single-Column Data #98820
Closed
@bedevere-app bedevere-app Bot added the type-security A security issue label Aug 11, 2026
@bedevere-app bedevere-app Bot mentioned this pull request Aug 11, 2026
[3.15] gh-98820: Fix quadratic time in csv.Sniffer for quoted fields #154867
Merged
@bedevere-app bedevere-app Bot added the awaiting core review label Aug 11, 2026
@pablogsal
pablogsal merged commit fd78b56 into python:3.10 Aug 11, 2026
15 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-app bedevere-app Bot removed the awaiting core review label Aug 11, 2026
@serhiy-storchaka
serhiy-storchaka deleted the backport-3.10-gh98820 branch August 27, 2026 12:40
icanhasmath pushed a commit to ActiveState/cpython that referenced this pull request Oct 28, 2026
@serhiy-storchaka @icanhasmath
[3.10] pythongh-98820: Fix quadratic time in csv.Sniffer for quoted f…
c1c23f0
…ields (pythonGH-154867) (python#155539)

(cherry picked from commit fd78b56)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

type-security A security issue

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@serhiy-storchaka @pablogsal @miss-islington
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.