Description

serhiy-storchaka

opened on Mar 15, 2015

Issue body actions

BPO	23676
Nosy	@malemburg, @doerwalter, @ncoghlan, @vstinner, @vadmium, @serhiy-storchaka
Files	translate_error_handlers.patch translate_error_handlers_2.patch

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/serhiy-storchaka'
closed_at = None
created_at = <Date 2015-03-15.21:59:08.104>
labels = ['interpreter-core', 'type-feature']
title = 'Add support of UnicodeTranslateError in standard error handlers'
updated_at = <Date 2015-03-26.22:45:41.845>
user = 'https://github.com/serhiy-storchaka'

bugs.python.org fields:

activity = <Date 2015-03-26.22:45:41.845>
actor = 'serhiy.storchaka'
assignee = 'serhiy.storchaka'
closed = False
closed_date = None
closer = None
components = ['Interpreter Core']
creation = <Date 2015-03-15.21:59:08.104>
creator = 'serhiy.storchaka'
dependencies = []
files = ['38502', '38504']
hgrepos = []
issue_num = 23676
keywords = ['patch']
message_count = 8.0
messages = ['238163', '238180', '238973', '239018', '239353', '239355', '239357', '239358']
nosy_count = 6.0
nosy_names = ['lemburg', 'doerwalter', 'ncoghlan', 'vstinner', 'martin.panter', 'serhiy.storchaka']
pr_nums = []
priority = 'normal'
resolution = None
stage = 'patch review'
status = 'open'
superseder = None
type = 'enhancement'
url = 'https://bugs.python.org/issue23676'
versions = ['Python 3.5']

Linked PRs

• gh-67864: Add support of UnicodeTranslateError in standard error handlers #121212

Activity

1. serhiy-storchaka commented on Mar 15, 2015

   

   serhiy-storchaka

   on Mar 15, 2015

   MemberAuthor

   More actions

   Proposed patch adds support of UnicodeTranslateError in standard error handlers "xmlcharrefreplace", "namereplace" and "surrogatepass". Support in "backslashreplace" was added in bpo-22286, support in "strict", "ignore" and "replace" was always, support in "surrogateescape" is unlikely possible.

   This can be used with bpo-18814.
2. serhiy-storchaka

   added

   interpreter-core(Objects, Python, Grammar, and Parser dirs)(Objects, Python, Grammar, and Parser dirs)

   type-featureA feature request or enhancementA feature request or enhancement

   on Mar 15, 2015
3. serhiy-storchaka commented on Mar 16, 2015

   

   serhiy-storchaka

   on Mar 16, 2015

   MemberAuthor

   More actions

   Fixed a bug in "surrogatepass" with translating and added the versionchanged directive.
4. vadmium commented on Mar 23, 2015

   

   vadmium

   on Mar 23, 2015

   Member

   More actions

   I think I saw your patch for bpo-18814 proposes to use UnicodeTranslateError. Is there any other case where it is used, either currently or in the past? All I know of it is the documentation, which says it is raised “during translating”.

   Experimenting with the constructor reveals that the “object” attribute is only allowed to be a text string (not bytes). So perhaps “translating” actually means converting from text strings to text strings, like “rot-13”. It would be nice if this were documented somewhere, rather than just saying translating is now supported.
5. serhiy-storchaka commented on Mar 23, 2015

   

   serhiy-storchaka

   on Mar 23, 2015

   MemberAuthor

   More actions

   No, currently UnicodeTranslateError is not used in the stdlib in 3.x. But it is documented and supported by some error handlers. I think it should be wider used in text-to-text translations similar to proposed in bpo-18814.
6. serhiy-storchaka

   self-assigned this

   on Mar 23, 2015
7. vstinner commented on Mar 26, 2015

   

   vstinner

   on Mar 26, 2015

   Member

   More actions

   I'm sorry, I don't understand this issue. Could you please elaborate the use case? Why do you want to support more error handlers? str.translate() calls _PyUnicode_TranslateCharmap() with errors="ignore", it's not possible to choose the error handler.

   Many codecs are implemented in Python and some of them are implemented with "charmap". Does this issue enhance the codecs implemented with "charmap"?

   "a\udc80".encode("latin9", "surrogatepass") raises UnicodeEncodeError with and without the patch, b"\x81".decode("cp1252", "surrogatepass") raises UnicodeDecodeError with and without the patch.

   Hum, I'm not sure that codecs.charmap_build() is related str.translate().
8. serhiy-storchaka commented on Mar 26, 2015

   

   serhiy-storchaka

   on Mar 26, 2015

   MemberAuthor

   More actions

   str.encode, bytes.decode and str.translate are unrelated to UnicodeTranslateError. But str.transform could be.
9. vstinner commented on Mar 26, 2015

   

   vstinner

   on Mar 26, 2015

   Member

   More actions

   Serhiy Storchaka added the comment:

   str.encode, bytes.decode and str.translate are unrelated to UnicodeTranslateError. But str.transform could be.

   Can you please give an example of Python code to show your change?
10. serhiy-storchaka commented on Mar 26, 2015

    

    serhiy-storchaka

    on Mar 26, 2015

    MemberAuthor

    More actions

    bpo-18814
11. ezio-melotti

    transferred this issue fromon Apr 10, 2022
12. serhiy-storchaka

    added this to Codecs and encodings issueson May 22, 2022
13. serhiy-storchaka

    added a commit that references this issue on Jul 1, 2024

    pythongh-67864: Add support of UnicodeTranslateError in standard erro…

    a7a84fe

Issue actions