GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-43510: Implement PEP 597 opt-in EncodingWarning. by methane · Pull Request #19481 · python/cpython

Conversation

@methane

@methane methane commented Apr 12, 2020 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

See PEP 597.

  • Add -X warn_default_encoding and PYTHONWARNDEFAULTENCODING.
  • Add EncodingWarning
  • Add io.text_encoding()
  • open(), TextIOWrapper() emits EncodingWarning when encoding is omitted and warn_default_encoding is enabled.
  • _pyio.TextIOWrapper() uses UTF-8 as fallback default encoding used when failed to import locale module. (used during building Python)
  • bz2, configparser, gzip, lzma, pathlib, tempfile modules use io.text_encoding().
  • Add documents
  • Add what's new entry

bpo-43510

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

ZackerySpytz and graingert reacted with thumbs up emoji
All reactions
  • 👍 2 reactions
@methane methane added the DO-NOT-MERGE label Apr 12, 2020
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label Apr 12, 2020
@bedevere-bot bedevere-bot added the awaiting core review label Apr 12, 2020
@methane
methane force-pushed the open-encoding branch 4 times, most recently from c37295e to 6f72b38 Compare April 13, 2020 09:22
methane added 3 commits January 29, 2021 17:36
@methane
Raise a warning when encoding is omitted
a3c014b
The warning is raised only in dev mode.

pathlib uses the new `io.text_encoding` helper function.
Other libraries will follow.
@methane
add test
050bd1b
@methane
wrap encoding=None with text_encoding.
939f4a0
@methane
methane force-pushed the open-encoding branch from e44dc5b to 939f4a0 Compare January 29, 2021 08:37
@methane
Add io.LOCALE_ENCODING = "locale"
3c99777
@methane methane changed the title PEP 597: Raise a warning when encoding is omitted PEP 597: Emit EncodingWarning when encoding is omitted Jan 30, 2021
methane added 2 commits January 30, 2021 17:52
@methane
Add EncodingWarning.
4016278
Copied from python#13938
@methane
Add sys.warn_default_encoding
c5c556c
@methane
methane force-pushed the open-encoding branch from db537b9 to c5c556c Compare January 30, 2021 08:52
methane added 11 commits January 30, 2021 18:04
@methane
shorten option names
d9a08c2
@methane
EncodingWarning extends Warning
772648e
@methane
make clinic
1a8e305
@methane
fix test
20966cd
@methane
remove wrong test case
2b80f42
@methane
fix exception_hierarchy.txt
760308c
@methane
Make sys.flags.encoding_warning int
a95dff2
@methane
Fix text_embed.
31fb411
@methane
Fix test_pickle
096a0a3
@methane
configparser: use io.text_encoding()
99fc938
@methane
Rename option names
6fdbcbc
sys.flags.encoding_warning -> warn_default_encoding
-X warn_encoding -> warn_default_encoding
PYTHONWARNENCODING -> PYTHONWARNDEFAULTENCODING
@methane methane changed the title PEP 597: Emit EncodingWarning when encoding is omitted bpo-43510: Add encoding="locale" and optional EncodingWarning Mar 16, 2021
@methane methane removed the DO-NOT-MERGE label Mar 16, 2021
@methane
Merge remote-tracking branch 'upstream/master' into open-encoding
3f362bc
methane and others added 7 commits March 23, 2021 14:49
@methane @CAM-Gerlach
Update Doc/library/io.rst
5b2830b
Co-authored-by: CAM Gerlach <[email protected]>
@methane @CAM-Gerlach
Apply suggestions from code review
14f2a6e
Co-authored-by: CAM Gerlach <[email protected]>
@methane
Move EncodingWarnings
06e2a32
@methane
fix comment
27d49d2
@methane
fix text_encoding() docstring
80f4644
@methane
update what's new
6ad0e7f
@methane
fix doc build
73b27f1
CAM-Gerlach
CAM-Gerlach reviewed Mar 23, 2021
View reviewed changes
Comment thread Lib/_pyio.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@CAM-Gerlach

CAM-Gerlach commented Mar 23, 2021

Copy link
Copy Markdown
Member

Thanks @methane ! I've replied to your comments with updated suggestions to reflect your feedback.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@methane @CAM-Gerlach
Update Doc/library/io.rst
c149d65
Co-authored-by: CAM Gerlach <[email protected]>
methane
methane commented Mar 24, 2021
View reviewed changes
Comment thread Lib/_pyio.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@methane @CAM-Gerlach
Apply suggestions from code review
4eb7655
Co-authored-by: CAM Gerlach <[email protected]>
methane
methane commented Mar 24, 2021
View reviewed changes
Comment thread Doc/whatsnew/3.10.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

@methane @CAM-Gerlach
Apply suggestions from code review
e3bce76
Co-authored-by: CAM Gerlach <[email protected]>
CAM-Gerlach
CAM-Gerlach reviewed Mar 24, 2021
View reviewed changes
Comment thread Doc/library/io.rst Resolved

Uh oh!

There was an error while loading. Please reload this page.

@methane @CAM-Gerlach
Update Doc/library/io.rst
c089fd7
Co-authored-by: CAM Gerlach <[email protected]>
@methane

methane commented Mar 25, 2021

Copy link
Copy Markdown
Member Author

I will merge this PR next week if there are no objections.

CAM-Gerlach reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@methane
methane merged commit 4827483 into python:master Mar 29, 2021
@methane
methane deleted the open-encoding branch March 29, 2021 03:28
@bedevere-bot bedevere-bot removed the awaiting core review label Mar 29, 2021
@bedevere-bot

bedevere-bot commented Mar 29, 2021

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 RHEL7 LTO + PGO 3.x has failed when building commit 4827483.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/96/builds/910) and take a look at the build logs.
  4. Check if the failure is related to this commit (4827483) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/96/builds/910

Failed tests:

  • test_tools
  • test_ftplib

Failed subtests:

  • test_storlines - test.test_ftplib.TestFTPClass
  • test_reindent_file_with_bad_encoding - test.test_tools.test_reindent.ReindentTests

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

412 tests OK.

10 slowest tests:

  • test_concurrent_futures: 2 min 38 sec
  • test_shelve: 2 min 30 sec
  • test_mailbox: 1 min 57 sec
  • test_dbm: 1 min 52 sec
  • test_tarfile: 1 min 36 sec
  • test_multiprocessing_spawn: 1 min 34 sec
  • test_tokenize: 1 min 33 sec
  • test_dbm_gnu: 1 min 28 sec
  • test_unparse: 1 min 28 sec
  • test_asyncio: 1 min 17 sec

1 test failed:
test_tools

14 tests skipped:
test_devpoll test_gdb test_ioctl test_kqueue test_msilib
test_ossaudiodev test_startfile test_tix test_tk test_ttk_guionly
test_winconsoleio test_winreg test_winsound test_zipfile64

2 re-run tests:
test_ftplib test_tools

Total duration: 6 min 19 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/test/test_ftplib.py", line 623, in test_storlines
    self.check_data(self.server.handler_instance.last_received_data, RETR_DATA)
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/test/test_ftplib.py", line 494, in check_data
    self.assertEqual(len(received), len(expected))
AssertionError: 12019 != 12018


Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/test/test_tools/test_reindent.py", line 29, in test_reindent_file_with_bad_encoding
    rc, out, err = assert_python_ok(self.script, '-r', bad_coding_path)
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/test/support/script_helper.py", line 160, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/test/support/script_helper.py", line 145, in _assert_python
    res.fail(cmd_line)
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/test/support/script_helper.py", line 72, in fail
    raise AssertionError("Process return code is %d\n"
AssertionError: Process return code is 1
command line: ['/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/python', '-X', 'faulthandler', '-I', '/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Tools/scripts/reindent.py', '-r', '/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto-pgo/build/Lib/test/bad_coding.py']
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot

bedevere-bot commented Mar 29, 2021

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 RHEL7 LTO 3.x has failed when building commit 4827483.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/507/builds/896) and take a look at the build logs.
  4. Check if the failure is related to this commit (4827483) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/507/builds/896

Failed tests:

  • test_tools

Failed subtests:

  • test_reindent_file_with_bad_encoding - test.test_tools.test_reindent.ReindentTests

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

412 tests OK.

10 slowest tests:

  • test_mailbox: 4 min 33 sec
  • test_peg_generator: 3 min 37 sec
  • test_concurrent_futures: 2 min 56 sec
  • test_shelve: 2 min 27 sec
  • test_multiprocessing_spawn: 1 min 50 sec
  • test_tokenize: 1 min 49 sec
  • test_unparse: 1 min 37 sec
  • test_lib2to3: 1 min 25 sec
  • test_asyncio: 1 min 21 sec
  • test_multiprocessing_forkserver: 1 min 9 sec

1 test failed:
test_tools

14 tests skipped:
test_devpoll test_gdb test_ioctl test_kqueue test_msilib
test_ossaudiodev test_startfile test_tix test_tk test_ttk_guionly
test_winconsoleio test_winreg test_winsound test_zipfile64

1 re-run test:
test_tools

Total duration: 7 min 15 sec

Click to see traceback logs
Traceback (most recent call last):
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto/build/Lib/test/test_tools/test_reindent.py", line 29, in test_reindent_file_with_bad_encoding
    rc, out, err = assert_python_ok(self.script, '-r', bad_coding_path)
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto/build/Lib/test/support/script_helper.py", line 160, in assert_python_ok
    return _assert_python(True, *args, **env_vars)
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto/build/Lib/test/support/script_helper.py", line 145, in _assert_python
    res.fail(cmd_line)
  File "/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto/build/Lib/test/support/script_helper.py", line 72, in fail
    raise AssertionError("Process return code is %d\n"
AssertionError: Process return code is 1
command line: ['/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto/build/python', '-X', 'faulthandler', '-I', '/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto/build/Tools/scripts/reindent.py', '-r', '/home/buildbot/buildarea/3.x.cstratak-RHEL7-x86_64.lto/build/Lib/test/bad_coding.py']
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot

bedevere-bot commented Mar 29, 2021

Copy link
Copy Markdown

⚠️⚠️⚠️ Buildbot failure ⚠️⚠️⚠️

Hi! The buildbot AMD64 Windows10 3.x has failed when building commit 4827483.

What do you need to do:

  1. Don't panic.
  2. Check the buildbot page in the devguide if you don't know what the buildbots are or how they work.
  3. Go to the page of the buildbot that failed (https://buildbot.python.org/all/#builders/146/builds/1014) and take a look at the build logs.
  4. Check if the failure is related to this commit (4827483) or if it is a false positive.
  5. If the failure is related to this commit, please, reflect that on the issue and make a new Pull Request with a fix.

You can take a look at the buildbot page here:

https://buildbot.python.org/all/#builders/146/builds/1014

Failed tests:

  • test_locale

Failed subtests:

  • test_getsetlocale_issue1813 - test.test_locale.TestMiscellaneous

Summary of the results of the build (if available):

== Tests result: FAILURE then FAILURE ==

395 tests OK.

10 slowest tests:

  • test_mmap: 9 min 37 sec
  • test_largefile: 5 min 57 sec
  • test_io: 5 min 39 sec
  • test_unparse: 3 min 12 sec
  • test_multiprocessing_spawn: 2 min 16 sec
  • test_concurrent_futures: 2 min 3 sec
  • test_tokenize: 1 min 46 sec
  • test_capi: 1 min 46 sec
  • test_peg_generator: 1 min 39 sec
  • test_asyncio: 1 min 31 sec

1 test failed:
test_locale

31 tests skipped:
test_curses test_dbm_gnu test_dbm_ndbm test_devpoll test_epoll
test_fcntl test_fork1 test_gdb test_grp test_ioctl test_kqueue
test_multiprocessing_fork test_multiprocessing_forkserver test_nis
test_openpty test_ossaudiodev test_pipes test_poll test_posix
test_pty test_pwd test_readline test_resource test_spwd
test_syslog test_threadsignals test_wait3 test_wait4
test_xxlimited test_xxtestfuzz test_zipfile64

1 re-run test:
test_locale

Total duration: 19 min 35 sec

Click to see traceback logs
Traceback (most recent call last):
  File "D:\buildarea\3.x.bolen-windows10\build\lib\test\test_locale.py", line 567, in test_getsetlocale_issue1813
    locale.setlocale(locale.LC_CTYPE, loc)
  File "D:\buildarea\3.x.bolen-windows10\build\lib\locale.py", line 610, in setlocale
    return _setlocale(category, locale)
locale.Error: unsupported locale setting
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@methane

methane commented Mar 29, 2021 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member Author

test_locale fail is unrelating to this PR. See bpo-37945.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@CAM-Gerlach

CAM-Gerlach commented Mar 29, 2021

Copy link
Copy Markdown
Member

Thanks @methane ! Excited to see this in Python 3.10.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@alexdelorenzo alexdelorenzo mentioned this pull request Apr 16, 2021
Python 3.10 alpha 7 introduces new changes to pathlib alexdelorenzo/aiopath#4
Open
chris-allan added a commit to chris-allan/omero-web that referenced this pull request Oct 10, 2022
@chris-allan
Update to a suitable version of the concurrent-log-handler
5dbc09c
We have a vendored version of ConcurrentLogHandler 0.9.1 [1] from
omero-py that we use.  This version is from 2013, has a few bugs, and
doesn't work properly with the encoding changes [2,3] made in Python
3.10.

A new version [4] whose lineage is that of the now unmaintained
ConcurrentLogHandler is now available which resolves several of the
aforementioned bugs, supports Python 3.10, and is drop in replacement
API compatible with the previous version.  This commit adds a dependency
for this new version and swaps the default logger class for it in
configuration.

  1. https://pypi.org/project/ConcurrentLogHandler/
  2. https://peps.python.org/pep-0597/
  3. python/cpython#19481
  4. https://pypi.org/project/concurrent-log-handler/
@chris-allan chris-allan mentioned this pull request Oct 10, 2022
Update to a suitable version of the concurrent-log-handler ome/omero-web#415
Merged
@AlexWaygood AlexWaygood mentioned this pull request Oct 26, 2023
text_encoding() is missing from io.__all__ #111356
Closed
@coderabbitai coderabbitai Bot mentioned this pull request Aug 16, 2026
Three Windows platform gaps, and a cpython_tests run that tested nothing recorded as PASS pyre-lang/pyre#1257
Merged
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vstinner vstinner vstinner left review comments
@CAM-Gerlach CAM-Gerlach CAM-Gerlach left review comments
@gpshead gpshead Awaiting requested review from gpshead

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@methane @CAM-Gerlach @bedevere-bot @vstinner @the-knights-who-say-ni
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.