GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — locale.getlocale fails if locale is set · Issue #87281 · python/cpython

locale.getlocale fails if locale is set #87281

New issue
Copy link
New issue
Copy link
Open
Open
locale.getlocale fails if locale is set#87281
Copy link
Labels
OS-windowsstdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

Description

AndersMunch
mannequin
opened on Feb 3, 2021
Issue body actions
BPO 43115
Nosy @malemburg, @pfmoore, @tjguk, @zware, @eryksun, @zooba, @AndersMunch

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields:

assignee = None
closed_at = None
created_at = <Date 2021-02-03.13:42:50.296>
labels = ['3.8', '3.10', 'library', '3.9', 'OS-windows']
title = 'locale.getlocale fails if locale is set'
updated_at = <Date 2021-02-23.17:59:57.570>
user = 'https://github.com/AndersMunch'

bugs.python.org fields:

activity = <Date 2021-02-23.17:59:57.570>
actor = 'eryksun'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Library (Lib)', 'Windows']
creation = <Date 2021-02-03.13:42:50.296>
creator = 'AndersMunch'
dependencies = []
files = []
hgrepos = []
issue_num = 43115
keywords = []
message_count = 15.0
messages = ['386203', '387136', '387138', '387147', '387149', '387157', '387164', '387195', '387205', '387208', '387209', '387256', '387273', '387558', '387588']
nosy_count = 8.0
nosy_names = ['lemburg', 'paul.moore', 'tim.golden', 'zach.ware', 'eryksun', 'steve.dower', 'swt2c', 'AndersMunch']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = None
url = 'https://bugs.python.org/issue43115'
versions = ['Python 3.8', 'Python 3.9', 'Python 3.10']

Linked PRs

  • gh-137729: Fix support for locales with @-modifiers #137253
  • gh-87281: Improve documentation for locale.setlocale() and locale.getlocale() #137313
  • [3.14] gh-87281: Improve documentation for locale.setlocale() and locale.getlocale() (GH-137313) #137722
  • [3.13] gh-87281: Improve documentation for locale.setlocale() and locale.getlocale() (GH-137313) #137723

Activity

Next
  1. AndersMunch commented on Feb 3, 2021

    @AndersMunch
    AndersMunchmannequin
    on Feb 3, 2021
    MannequinAuthor
    More actions

    getlocale fails with an exception when the string returned by _setlocale is already an RFC 1766 language tag.

    Example:

    Python 3.10.0a4 (tags/v3.10.0a4:445f7f5, Jan  4 2021, 19:55:53) [MSC v.1928 64 bit (AMD64)] on win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import locale
    >>> locale.setlocale(locale.LC_ALL, 'en-US')
    'en-US'
    >>> locale.getlocale()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "C:\flonidan\env\Python310\lib\locale.py", line 593, in getlocale
        return _parse_localename(localename)
      File "C:\flonidan\env\Python310\lib\locale.py", line 501, in _parse_localename
        raise ValueError('unknown locale: %s' % localename)
    ValueError: unknown locale: en-US

    Expected result:
    ('en-US', None)

    See wxWidgets/Phoenix#1637 for an example of the ensuing problems. wx.Locale calls C setlocale directly, but, as far as I can see, correctly, using dashes in the language code which is consistent with not only RFC 1766 but also the examples in Microsoft's documentation (https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/setlocale-wsetlocale?view=msvc-160). CPython seems to assume underscored names such as 'en_US' instead, as shown by getdefaultlocale inserting an underscore.

  2. AndersMunch
    added
    3.8 (EOL)end of lifeend of life
    3.9 (EOL)end of lifeend of life
    3.10 (EOL)end of lifeend of life
    stdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directory
    OS-windows
    on Feb 3, 2021
  3. AndersMunch commented on Feb 17, 2021

    @AndersMunch
    AndersMunchmannequin
    on Feb 17, 2021
    MannequinAuthor
    More actions

    I discovered that this can happen with underscores as well:

    Python 3.8.7 (tags/v3.8.7:6503f05, Dec 21 2020, 17:59:51) [MSC v.1928 64 bit (AMD64)] on win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import locale
    >>> locale.setlocale(locale.LC_ALL, 'en_DE')
    'en_DE'
    >>> locale.getlocale()
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File "C:\flonidan\env\Python38-64\lib\locale.py", line 591, in getlocale
        return _parse_localename(localename)
      File "C:\flonidan\env\Python38-64\lib\locale.py", line 499, in _parse_localename
        raise ValueError('unknown locale: %s' % localename)
    ValueError: unknown locale: en_DE

    locale.setlocale does validate input - if you write nonsense in the second argument then you get an exception, "locale.Error: unsupported locale setting". So I'm guessing the en_DE locale is actually being set here, and the problem is solely about getlocale making unfounded assumptions about the format.

    Same thing happens in 3.10.0a4.

  4. malemburg commented on Feb 17, 2021

    @malemburg
    malemburg
    on Feb 17, 2021
    Member
    More actions
    On 17.02.2021 10:55, Anders Munch wrote:
    >>>> import locale
    >>>> locale.setlocale(locale.LC_ALL, 'en_DE')
    > 'en_DE'
    >>>> locale.getlocale()
    > Traceback (most recent call last):
    >   File "<stdin>", line 1, in <module>
    >   File "C:\flonidan\env\Python38-64\lib\locale.py", line 591, in getlocale
    >     return _parse_localename(localename)
    >   File "C:\flonidan\env\Python38-64\lib\locale.py", line 499, in _parse_localename
    >     raise ValueError('unknown locale: %s' % localename)
    > ValueError: unknown locale: en_DE

    The locale module does not know this encoding, so cannot
    guess the encoding. Since getlocale() returns the language
    code and encoding, this fails.

    If you add the encoding, you should be fine:

    >>> locale.setlocale(locale.LC_ALL, 'en_DE.UTF-8')
    'en_DE.UTF-8'
    >>> locale.getlocale()
    ('en_DE', 'UTF-8')
  5. AndersMunch commented on Feb 17, 2021

    @AndersMunch
    AndersMunchmannequin
    on Feb 17, 2021
    MannequinAuthor
    More actions

    getlocale is documented to return None for the encoding if no encoding can be determined. There's no need to guess.

    I can't change the locale.setlocale call, because where I'm actually having the problem, I'm not even calling locale.setlocale: wxWidgets is calling C setlocale, that's where it comes from.

    wxWidgets aside, it doesn't seem right that getlocale fails when setlocale succeeded.

  6. malemburg commented on Feb 17, 2021

    @malemburg
    malemburg
    on Feb 17, 2021
    Member
    More actions

    On 17.02.2021 12:36, Anders Munch wrote:

    getlocale is documented to return None for the encoding if no encoding can be determined. There's no need to guess.

    Well, not quite... the documentation says that None can be returned,
    not that it will return None in such cases.

    What happens is that the value returned by the C lib's setlocale()
    is normalized and parsed. If it doesn't include an encoding,
    the table locale_alias is used to provide an encoding. If this
    table does not include the locale, you get the ValueError.

    We could enhance this to return None for the encoding instead
    of raising an exception, but would this really help ?

    Alternatively, we could add "en_DE" to the alias table and set
    a default encoding to use. I assume this would have to be
    ISO8859-15. OTOH, Windows supports UTF-8 for all locales in
    more recent Windows versions, so perhaps we should use UTF-8
    instead.

    BTW: What is wxWidgets doing with the returned values ?

    Resources:

    • setlocale() in Visual C++ on Windows:

    https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/setlocale-wsetlocale?view=msvc-160

    • locale name formats supported by Windows CRT:

    https://docs.microsoft.com/en-us/cpp/c-runtime-library/locale-names-languages-and-country-region-strings?view=msvc-160

    • Language locale tags known by Windows:

    https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-lcid/a9eac961-e77d-41a6-90a5-ce1a8b0cdb9c

    • UTF8 support in Windows CRT:

    https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/setlocale-wsetlocale?view=msvc-160#utf-8-support

  7. 25 remaining items

  8. serhiy-storchaka
    added a commit that references this issue on Aug 1, 2025

    pythongh-87281: Improve documentation for locale.setlocale() and loca…

    60a26f7
  9. StanFromIreland
    added
    type-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error
    and removed
    3.10 (EOL)end of lifeend of life
    3.9 (EOL)end of lifeend of life
    3.8 (EOL)end of lifeend of life
    on Aug 1, 2025
  10. serhiy-storchaka
    added a commit that references this issue on Aug 13, 2025

    gh-87281: Improve documentation for locale.setlocale() and locale.get…

    15ab457
  11. miss-islington
    added 2 commits that reference this issue on Aug 13, 2025

    pythongh-87281: Improve documentation for locale.setlocale() and loca…

    caea563

    pythongh-87281: Improve documentation for locale.setlocale() and loca…

    c85f0d7
  12. serhiy-storchaka
    added a commit that references this issue on Aug 13, 2025

    [3.13] gh-87281: Improve documentation for locale.setlocale() and loc…

    9417ea5
  13. Agent-Hellboy
    added a commit that references this issue on Aug 19, 2025

    pythongh-87281: Improve documentation for locale.setlocale() and loca…

    4059f2d
  14. hugovk
    added a commit that references this issue on Sep 11, 2025

    [3.14] gh-87281: Improve documentation for locale.setlocale() and loc…

    374b242
  15. StanFromIreland commented on Sep 17, 2025

    @StanFromIreland
    StanFromIreland
    on Sep 17, 2025
    Member
    More actions

    Triage: Is there anything left to do here? All backports are complete.

  16. StanFromIreland
    added
    pendingThe issue will be closed if no feedback is providedThe issue will be closed if no feedback is provided
    on Sep 17, 2025
  17. serhiy-storchaka commented on Sep 17, 2025

    @serhiy-storchaka
    serhiy-storchaka
    on Sep 17, 2025
    Member
    More actions

    Yes, this is a large and complex issue. It needs a lot of work.

  18. StanFromIreland
    removed
    pendingThe issue will be closed if no feedback is providedThe issue will be closed if no feedback is provided
    on Sep 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    OS-windowsstdlibStandard Library Python modules in the Lib/ directoryStandard Library Python modules in the Lib/ directorytype-bugAn unexpected behavior, bug, or errorAn unexpected behavior, bug, or error

    Projects

    • Locale issues 🗺
      Status
      No status
      Show more project fields

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

    You are visitor 01402503 — GeezerVison does not store the pages you look up.
    Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.