GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — gh-157678: Allow csv.Sniffer to inspect large fields by GruffElixir · Pull Request #157689 · python/cpython

Conversation

@GruffElixir

@GruffElixir GruffElixir commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

gh-157678

csv.Sniffer.sniff() uses the CSV reader for trial parsing, so a sample containing a field larger than csv.field_size_limit() currently makes every candidate parse fail and raises csv.Error. Temporarily raising the parser limit to the sample length lets Sniffer determine the dialect, then restores the caller's configured limit in a finally block.

Tests:

  • python Lib/test/test_csv.py TestSniffer
  • focused regression for a 200,000-character field with a configured limit of 100
  • git diff --check

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
Copilot AI lite review requested due to automatic review settings September 17, 2026 16:00
@python-cla-bot

python-cla-bot Bot commented Sep 17, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

CLA signed
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@bedevere-app bedevere-app Bot mentioned this pull request Sep 17, 2026
csv.Sniffer.sniff() raises Could not determine delimiter when the sample contains a field larger than field_size_limit() #157678
Open
Copilot AI reviewed Sep 17, 2026

Copilot AI left a comment

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@bedevere-app bedevere-app Bot added the awaiting review label Sep 17, 2026
@GruffElixir
pythongh-157678: Allow Sniffer to inspect large fields
1bae00d
@GruffElixir
GruffElixir force-pushed the fix-csv-sniffer-field-limit branch from 9020c23 to 1bae00d Compare September 17, 2026 16:02
sylvesterkaczmarek
sylvesterkaczmarek reviewed Sep 17, 2026
View reviewed changes

@sylvesterkaczmarek sylvesterkaczmarek left a comment

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This temporarily changes the interpreter-wide CSV field limit, so concurrent readers can observe the raised limit, and a concurrent field_size_limit() update can also get overwritten by the finally restore. Could Sniffer avoid mutating the global limit for its trial parses, or otherwise isolate this so sniffing doesn't change another thread's parser behavior?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@GruffElixir

GruffElixir commented Sep 20, 2026

Copy link
Copy Markdown
Contributor Author

Good catch. The limit is process-wide, so restoring it afterward still leaves a race. I removed the temporary mutation; trial parses now skip an over-limit row and continue, so the reported sample still infers , without changing the configured limit. Pushed 23cd0b0. The focused reproducer and syntax check pass locally; the full test_csv suite is left to CI.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@GruffElixir
pythongh-157678: Avoid changing shared CSV field limit
b6006b7
@GruffElixir
GruffElixir force-pushed the fix-csv-sniffer-field-limit branch from 23cd0b0 to b6006b7 Compare September 20, 2026 15:03
@sylvesterkaczmarek

sylvesterkaczmarek commented Sep 21, 2026

Copy link
Copy Markdown

Rechecked b6006b7. Sniffer no longer mutates the process-wide field_size_limit at all; oversized trial rows are skipped locally while later rows can still contribute to dialect inference. The regression also pins that the configured global limit remains unchanged, and the full CI matrix is green. My concurrency concern is resolved.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@lpyu001

lpyu001 commented Sep 22, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

Please avoid using  amend  and  force‑push . Follow the CPython community commit guidelines ,https://devguide.python.org/getting-started/pull-request-lifecycle/
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot review effort
Applies to this pull request for everyone.Learn more about Copilot code review.
Copilot left review comments
+1 more reviewer
@sylvesterkaczmarek sylvesterkaczmarek sylvesterkaczmarek left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@GruffElixir @sylvesterkaczmarek @lpyu001
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.