GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — bpo-24665: double-width CJK chars support for textwrap by fgallaire · Pull Request #89 · python/cpython Conversation
|
|
Hello, and thanks for your contribution! I'm a bot set up to make sure that the project can legally accept your contribution by verifying you have signed the PSF contributor agreement (CLA). Unfortunately we couldn't find an account corresponding to your GitHub username on bugs.python.org (b.p.o) to verify you have signed the CLA. This is necessary for legal reasons before we can look at your contribution. Please follow these steps to help rectify the issue:
Thanks again to your contribution and we look forward to looking at it! |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
I now have listed my GitHub username at bpo and i had already signed the CLA. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
|
||
| def __init__(self, | ||
| width=70, | ||
| cjk=False, |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
These arguments can be passed as positional argument.
So new argument should be added last.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I agree
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| i = 1 | ||
| # <= and i-1 to catch the last double length char of odd line | ||
| while cjklen(text[:i]) <= index: | ||
| i = i + 1 |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I don't like this O(n^2) algorithm.
w = 0
for i, c in enumerate(text):
w += cjkwide(c) + 1
if w > index:
break
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Very relevant point.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
e52db22 to
5c4b2af
Compare
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
You need to document the name option in Doc/library/textwrap.rst, don't forget ".. versionchanged:: 3.7" markup.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I suggest to make it private and use a different name than len: self._text_width() for example.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Exposing functions mean that you must document them, write unit tests, and that someone (maybe not you) have to maintain them. I don't think that it's worth it. Let's try with something simple, make these functions private.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I agree
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
There is not need to repeat cjk here, there is already a generic **kwargs. Same for other functions below.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
This is the same need as width: to be visible so easily usable.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
You can write {'F', 'W'}, it's optimized as a constant frozenset.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
It's really faster than a tuple ?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Please add _ in the name: cjk_wide().
Note sure about the name: is_cjk_wide_char()?
Do we really need to make the function?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Rename to cfk_width()?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I don't understand this case. Why do you pass a non-string to the function?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
This is for people who want to do:
from textwrap import cjklen as len
and use cjklen transparently
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
IMHO it's better to make all these new functions private: add _ prefix. Rename to _cjk_slices().
"index": is it a number of character or a width? Maybe rename to width?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
No, I really want this functions to be exported, they are useful
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
By email I proposed a different design. Make existing TextWrapper extensible: add text_width() and truncate() methods. Then add a new TextWrapperCJK which overrides these methods. It would allow to reuse the code for other cases than just CJK. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
One import per line please.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I agree
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Exposing functions mean that you must document them, write unit tests, and that someone (maybe not you) have to maintain them. I don't think that it's worth it. Let's try with something simple, make these functions private.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Your change breaks Python build: Python requires optparse to compile modules like unicodedata, optparse imports textwrap which now always requires unicodedata. Using two different classes, it would be simpler to only import unicodedata when the TextWrapperCJK class is instanciated, "on demand", and so fix the bootstrap issue. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
What about a depreciation warning to inform that cjk default will switch to |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
optparse is deprecated since Python 3.2 so it should not drive this work, but a port to argparse will not solve this problem because of the same cycling dependency. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Codecov Report
@@ Coverage Diff @@ ## master #89 +/- ## ========================================= Coverage ? 82.38% ========================================= Files ? 1428 Lines ? 351193 Branches ? 0 ========================================= Hits ? 289333 Misses ? 61860 Partials ? 0 Continue to review full report at Codecov.
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
While I agree supporting east_asian_wdith is preferable, I don't like current API.
It's OK for third party library. But for standard library, I prefer more generic API and implementation.
While "practical beats purity", I want glibc's wcwidth for UTF-8 at least. (bonus: option like ambiwidth=double in vim)
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
|
||
| .. function:: cjk_len(text) | ||
|
|
||
| Return the real width of *text* (its len if not a string). |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I don't like wording real.
How about zero-width space? How about combining character sequence? variation selector?
EMOJI MODIFIER?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
You're probably right. The real purpose is to help to understand that width (cjk_len) and len are different. What do you think about visual instead ?
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Disagreed. I guess text editors and terminals are top two applications in textwrap. I known little about text editors. In terminals, things are quite interesting. Different terminals use different versions of
Different Unicode versions can lead to different results for In CPython, handling multiple Unicode versions sounds impractical. IMO chasing the latest Unicode 9.0 is a good idea. If a terminal emulator is not compatible with Unicode 9.0, it should be fixed. [1] https://sourceware.org/bugzilla/show_bug.cgi?id=20313 |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
Also, I would recommend rename cjk_foobar stuffs to unicode_foobar. CJK characters occupy a large portion on Unicode tables but not the whole. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
@yan12125 sorry, my wording was wrong. (I'm not good at English). BTW, pull request is not place for discussion like this. |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| Fullwidth and Wide CJK chars are double-width. | ||
| """ | ||
| import unicodedata | ||
| return unicodedata.east_asian_width(char) in ('F', 'W') |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| def cjk_len(text): | ||
| """Return the real width of text (its len if not a string). | ||
| """ | ||
| if not isinstance(text, str): |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Strange case handling, maybe we should expect only string type text argument in this function...
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Again: it's for an handy replacement of the built-in len():
from textwrap import cjk_len as len
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| Santiago Gala | ||
| Yitzchak Gale | ||
| Matthew Gallagher | ||
| Florent Gallaire |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Not sure if that makes lots of sense for such a small change (even though such changes are very welcome of course!)
There are commits from loads of people in this repo, but way less entries in this file (adding every contributor would quickly make the file pretty chaotic).
IMO it'd be better to only add yourself to this file when you contribute e.g. a major fix, new feature, etc
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I strongly disagree, if persons are missing they should be added.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
Florent definitely belongs in ACKS. We have been fairly liberal in adding people, some for as little as a well-crafted sentence or line of code.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
| """Return True if char is Fullwidth or Wide, False otherwise. | ||
| Fullwidth and Wide CJK chars are double-width. | ||
| """ | ||
| import unicodedata |
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
I think importing unicodedata, when wanted, for each char, is wrong. If not imported at the top, it could be imported as a global in TextWrapper.init when cjk is true. (I am assuming that the convenience functions all instantiate TextWrapper.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
…er shutdown code Fix the implementation to work as documented if a thread dies. Now Stackless kills only tasklets with a nesting level > 0. During interpreter shutdown stackless additionally kills daemon threads, if they execute Python code or switch tasklets. This prevents access violations after clearing the thread states. Add a 1ms sleep for each daemon thread. This way the thread gets a better chance to run. Thanks to Kristján Valur Jónsson for suggesting this improvement. Add a test case for a C assertion violation during thread shutdown. Add some explanatory comments to the shutdown test case. Thanks to Christian Tismer for the sugestion. https://bitbucket.org/stackless-dev/stackless/issues/81 https://bitbucket.org/stackless-dev/stackless/issues/89 (grafted from 2cc41427a347a1ebec4eedc3db06a3664e67f798, 1388a2003957, 6140f5aaca2c and edc9b92ec457)
|
To try and help move older pull requests forward, we are going through and backfilling 'awaiting' labels on pull requests that are lacking the label. Based on the current reviews, the best we can tell in an automated fashion is that a core developer requested changes to be made to this pull request. If/when the requested changes have been made, please leave a comment that says, |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
|
I close this because I don't like APIs this PR has and the author seems satisfied by |
Sorry, something went wrong.
There was an error while loading. Please reload this page.
Per supervisor [chat L2626] + L2742 + librarian L2737 (close pythia python#87 python#3 ephemera-gap durably). Captures empirical W26 push 84 incident (95c9f9b → 1553c14 fix): - Both cinder_opcode_ids.h + Include/opcode.h use Py_OPCODE_H guard - Include order silently shadows whichever loads second - W26 case: BINARY_OP_ADD_INT undefined → attr_probe HIR regression caught by W21 golden trip-wire Three resolution options enumerated: - (A) rename cinder_opcode_ids.h guard to CINDERX_OPCODE_IDS_H — recommended, eliminates collision class - (B) static_assert in cinder_opcode.h to detect collision at compile — hardens detection without fixing - (C) header-comment only — current state, NOT recommended (chat- ephemera proven insufficient) Related W29 candidate noted: PHX_PRIM_OP_* / PHX_PRIM_UOP_* hard-coded in builder_emit_c.c lines 3727-3746 with no static_assert binding to classloader.h authoritative #defines (per pythia python#89 python#3 re-issue). DEFERRED per supervisor — schedule post-Batch-2 burndown, before any upstream sync touching Include/opcode.h or classloader.h.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.