GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-24665: Add CJK support in textwrap by default. by JulienPalard · Pull Request #5649 · python/cpython

Conversation

@JulienPalard

@JulienPalard JulienPalard commented Feb 13, 2018 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Related to:

  • Patch: new method get_wch for ncurses bindings: accept wide characters (unicode)
  • textwrap.wrap: new argument for more pleasing output
  • textwrap.wrap: add control for fonts with different character widths
  • Add functions to get the width in columns of a character
  • CJK support for textwrap
  • bpo-24665: double-width CJK chars support for textwrap #89

https://bugs.python.org/issue24665

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label Feb 13, 2018
@bedevere-bot bedevere-bot added the awaiting merge label Feb 13, 2018
@ned-deily
ned-deily requested a review from larryhastings February 13, 2018 03:19
fgallaire
fgallaire reviewed Feb 14, 2018
View reviewed changes
Comment thread Lib/textwrap.py Outdated

@fgallaire fgallaire Feb 14, 2018

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why inlining _len(), I don't have seen performance issues and it's less readable (less pythonic)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@JulienPalard JulienPalard Feb 15, 2018

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do you know where to break once you have the whole value?

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@fgallaire fgallaire Feb 15, 2018

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello I was reading too fast. In my version there's the _wide boolean function.
So here width += _wide(char) + 1

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@fgallaire fgallaire Feb 15, 2018

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And _len is just return sum(2 if _wide(char) else 1 for char in text) with no performance issues

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@fgallaire fgallaire Feb 15, 2018

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

More pythonic, DRY.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@JulienPalard JulienPalard Mar 6, 2018

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I won't bet on the performances, calling _len from _slice adds two functions calls per character (one to _len and one to sum). In one case I'm doing it on a character, and in the other case in a whole string. Yes I could also factorize this ternary to a third function, but I don't find it more readable.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
fgallaire
fgallaire reviewed Feb 14, 2018
View reviewed changes
Comment thread Lib/textwrap.py Outdated

@fgallaire fgallaire Feb 14, 2018

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why note use enumerate(), it's less readable (less pythonic)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@JulienPalard JulienPalard Feb 15, 2018

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because it does not works with enumerate as the last incrementation were not done. I do not remember which case exactly but if you run the unit test you'll spot it easily, it was failing, I'll do if needed but can't right now.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@fgallaire fgallaire Feb 15, 2018 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interested in that, the code was strongly tested for txt2tags and don't catch this problem.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@JulienPalard JulienPalard Mar 6, 2018

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Your initial implementation was working thanks to your if cjk_len(text) <= index: return text, '' fixing the special case explicitly, I may have tried to avoid it.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions

@fgallaire fgallaire Mar 6, 2018

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Explicit is better than implicit." but the more important is that both solutions are correct.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@fgallaire

fgallaire commented Feb 14, 2018

Copy link
Copy Markdown

Don't see my author credit
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@fgallaire

fgallaire commented Feb 15, 2018

Copy link
Copy Markdown

And you miss the if self.width <= 0: bug fixed in #89
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@JulienPalard
JulienPalard force-pushed the textwrap-cjk branch 2 times, most recently from 45fd84d to 4623375 Compare March 6, 2018 22:42
@JulienPalard @fgallaire
bpo-24665: Add CJK support in textwrap by default.
57b2882 
Co-authored-by: Florent Gallaire <[email protected]>
@JulienPalard
JulienPalard force-pushed the textwrap-cjk branch from 4623375 to 57b2882 Compare March 6, 2018 22:43
@JulienPalard

JulienPalard commented Mar 6, 2018

Copy link
Copy Markdown
Member Author

And you miss the if self.width <= 0: bug fixed in #89

You're right! And trying to split a wide character yield to an infinite loop.

Don't see my author credit

Gladly fixed and co-authored you.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@fgallaire

fgallaire commented Mar 6, 2018

Copy link
Copy Markdown

Thanks @JulienPalard, I'm so happy ! I had almost lost hope to see this issue fixed.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
fgallaire
fgallaire reviewed Mar 6, 2018
View reviewed changes
Comment thread Lib/textwrap.py
if self.width <= 0:
raise ValueError("invalid width %r (must be > 0)" % self.width)
elif self.width == 1 and _width(text) > len(text):
raise ValueError("invalid width 1 (must be > 1 when CJK chars)")

@fgallaire fgallaire Mar 6, 2018

Copy link
Copy Markdown

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have done a more complex solution:

elif self.width == 1 and (sum(self._width(chunk) for chunk in chunks) >
                              sum(len(chunk) for chunk in chunks)):

It throws the exception earlier, but it's probably not absolutely necessary.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@JulienPalard
JulienPalard requested a review from vstinner March 28, 2018 21:16
terryjreedy
terryjreedy requested changes Jul 8, 2018
View reviewed changes

@terryjreedy terryjreedy left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The change I request is that this be closed because it is conceptually wrong. Textwrap works in terms of abstract 'characters' (codepoint), not physical units. I will explain this on the issue.

Aside from that, 2 is the wrong number to add, as 'double-width' characters are not actually twice as wide as fixed-pitch Ascii chars of the same height. See the issue for this as well.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@bedevere-bot bedevere-bot added awaiting changes and removed awaiting merge labels Jul 8, 2018
@bedevere-bot

bedevere-bot commented Jul 8, 2018

Copy link
Copy Markdown

When you're done making the requested changes, leave the comment: I have made the requested changes; please review again.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
@methane methane closed this Jul 11, 2018
@JulienPalard
JulienPalard deleted the textwrap-cjk branch June 16, 2019 14:07
@fgallaire fgallaire mannequin mentioned this pull request Apr 10, 2022
CJK support for textwrap #68853
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@terryjreedy terryjreedy terryjreedy requested changes
@larryhastings larryhastings Awaiting requested review from larryhastings
@vstinner vstinner Awaiting requested review from vstinner
+1 more reviewer
@fgallaire fgallaire fgallaire left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

awaiting changes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

6 participants

@JulienPalard @fgallaire @bedevere-bot @terryjreedy @methane @the-knights-who-say-ni
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.