GeezerVison is showing github.com — Feature request: sniff csv quoting and lineterminator · Issue #129374 · python/cpython
Feature request: sniff csv quoting and lineterminator #129374DescriptionIf you read a csv file, alter the data and then write it back, it would be very nice to recreate also the quoting behaviour. So e.g. if the file is quoted in style csv.QUOTE_NONNUMERIC, it would be very useful to have the quoting property set correctly in the dialect object returned by csv.Sniffer().sniff.
Also, I just figured out that the line terminator is also hard-coded to "\r\n". Would also be nice to be adaptive...
Linked PRs
Activitychanged the title [-]Feature request: sniff csv quoting[/-] [+]Feature request: sniff csv quoting and lineterminator[/+] on Jan 27, 2025ericvsmith commented on Jan 28, 2025 This probably belongs on discuss.python.org. But in any event, you'll need to specify more information. What specific changes would you like to see? Nikilauz commented on Feb 3, 2025 In general, I would expect the Dialect object returned by csv.Sniffer.sniff to have its properties set in a way that a writer object initialized with the dialect would reproduce a similar csv string. This seems to be a clear correctness constraint for the dialect functionality for me, so is this really something to put in a discussion forum? I'm right aware that this may be an issue with a questionable benefit-cost-ratio, though...
Anyhow, I'm not familiar with the feedback/issue conventions of the python project and may have misunderstood the purpose of dialects, so feel free to correct me (: ericvsmith commented on Feb 4, 2025 Could you provide sample code that shows what results you are getting, and explain how that differs from what expect to get? Nikilauz commented on Feb 5, 2025 Sure:
import csv
s = '42,"hello","world",-1\n' # e.g. via readline method from some csv file
d = csv.Sniffer().sniff(s)
d.quoting # 0
d.lineterminator # '\r\n'
x = list(csv.reader([s], d)) # [['42', 'hello', 'world', '-1']]
f = open("test.csv", "w", newline = "")
csv.writer(f, d).writerows(x)
f.close()
f = open("test.csv", "r+b")
f.read() # b'42,hello,world,-1\r\n'
f.close()
So instead of b'42,hello,world,-1\r\n' I would expect to get b'42,"hello","world",-1\n'.
More precisely, I would expect d.quoting to be 2 (i.e. csv.QUOTE_NONNUMERIC) and d.lineterminator to be '\n'.
Thanks for your time! aterrel commented on May 19, 2025 What would you expect if there are a mix of string quoted and not quoted? Nikilauz commented on May 19, 2025 Well then it's none of the possible values of Dialect.quoting and may be some fallback default quoting style I guess, i.e. 0 or so... Reacted by Andy R. Terrel aterrel commented on May 19, 2025 Nikilauz commented on May 20, 2025 Looks good, depending on the assertEquals and getvalue methods there could be some issue with comparing \n and \r\n. With actual files that was a problem, thats why I used bytestrings... aterrel commented on May 21, 2025 @Nikilauz I've added the PR for the feature. Would be great if you could pull down the dev version of Python and test it out a bit. Unfortunately there are some gotchas but this seems to be the beginning of a solution. serhiy-storchaka commented on Jul 21, 2026 There is an older issue for detecting lineterminator:
MetadataMetadataStatusShow more project fields
DevelopmentNo branches or pull requestsIssue actions
|