Description

ncoghlan

opened on Jan 4, 2017

Issue body actions

Notes from the linux-sig thread at: https://mail.python.org/pipermail/linux-sig/2017-January/000014.html

• change the sense of the environment variable to be an explicit off-switch for the coercion behaviour (PYTHONCOERCECLOCALE=0) rather than being an on-switch for more permissive behaviour
• add a "--with[out]-c-locale-coercion" configure flag and a related PY_COERCE_C_LOCALE preprocessor definition (for the command line app coercion behaviour)
• add a "--with[out]-c-locale-warning" configure flag and a related PY_WARN_ON_C_LOCALE preprocessor definition (for the runtime library warning behaviour)
• in the command line app, check the result of the C.UTF-8 locale coercion, complain if it doesn't work, and then try the en_US.UTF-8 locale. If that still doesn't work, complain again, then continue with the C locale
• simplify the library warning to:

Python runtime initialized with LC_CTYPE=C (a locale with default ASCII encoding), which may cause Unicode compatibility problems. Using C.UTF-8 as an alternative Unicode-compatible locale is recommended.

Reacted by Barry Warsaw

Activity

1. ncoghlan commented on Jan 5, 2017

   

   ncoghlan

   on Jan 5, 2017

   ContributorAuthor

   More actions

   Additional notes based on further discussion:

   • PY_COERCE_C_LOCALE and PY_WARN_ON_C_LOCALE should both be unset when building on Mac OS X (since this problem doesn't occur there)
   • PY_COERCE_C_LOCALE and PY_WARN_ON_C_LOCALE should both be unset when building on Windows (while this will happen by default, and the entire code blocks where these occur are skipped in the Windows builds anyway, there's no harm in being explicit about it in the PEP)
2. ncoghlan commented on Jan 6, 2017

   

   ncoghlan

   on Jan 6, 2017

   ContributorAuthor

   More actions

   Additional notes based on https://www.python.org/dev/peps/pep-0540/:

   • if not already set, PYTHONIOENCODING should be set to UTF-8:surrogateescape if the locale is successfully coerced to one based on UTF-8 (NOT done: added a section explaining the use of "strict" instead)

   • emphasise that the proposed approach matches what people already do to get Python 3 working reliably in Docker containers and other environments where the system locale information isn't readily available (and/or isn't relevant)

   • emphasise that the proposed solution can be implemented as a wrapper script for earlier Python 3.x versions:

     #!/bin/sh
     LC_ALL=C.UTF-8 LANG=C.UTF-8 python3 $@
3. ncoghlan commented on Jan 7, 2017

   

   ncoghlan

   on Jan 7, 2017

   ContributorAuthor

   More actions

   Changes incorporated via 221099d
4. ncoghlan

   closed this as completedon Jan 7, 2017

Issue actions