GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — bpo-31793: Documentation: Specialize smart-quotes for japanese. by JulienPalard · Pull Request #4006 · python/cpython

Conversation

@JulienPalard

@JulienPalard JulienPalard commented Oct 15, 2017 •
edited by bedevere-bot
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

Specializing and by doing so showing the way on how to specialize smart quoting of documentation in various languages.

I ran a test build using:

rm -f docutils.conf build/
sphinx-build -j8 -b html -d build/doctrees -D latex_elements.papersize= -D locale_dirs=/tmp/mdk/locales/ -D language=fr -D gettext_compact=0 -Ea -A daily=1 -A switchers=1 . build/html
mv build build_fr
sphinx-build -j8 -b html -d build/doctrees -D latex_elements.papersize= -D locale_dirs=/tmp/mdk/locales/ -D language=ja -D gettext_compact=0 -Ea -A daily=1 -A switchers=1 . build/html
mv build build_ja
sphinx-build -j8 -b html -d build/doctrees -D latex_elements.papersize= -Ea -A daily=1 -A switchers=1 . build/html
mv build build_en

cp /tmp/docutils.conf docutils.conf
sphinx-build -j8 -b html -d build/doctrees -D latex_elements.papersize= -D locale_dirs=/tmp/mdk/locales/ -D language=fr -D gettext_compact=0 -Ea -A daily=1 -A switchers=1 . build/html
mv build build_fr_docutils
sphinx-build -j8 -b html -d build/doctrees -D latex_elements.papersize= -D locale_dirs=/tmp/mdk/locales/ -D language=ja -D gettext_compact=0 -Ea -A daily=1 -A switchers=1 . build/html
mv build build_ja_docutils
sphinx-build -j8 -b html -d build/doctrees -D latex_elements.papersize= -Ea -A daily=1 -A switchers=1 . build/html
mv build build_en_docutils
$ cat docutils.conf 
[restructuredtext parser]
smartquotes-locales: ja: ""''

fr and en yielded not a single difference, but ja differed as expected, like:

< インタープリタ名は 「シェバン」 行としてアーカイブの起点に書込まれます。
---
> インタープリタ名は &quot;シェバン&quot; 行としてアーカイブの起点に書込まれます。
206,207c206,207
< <em>main</em> 引数は 「pkg.module:callable」 の形式を取り、
< アーカイブは 「pkg.module」 をインポートして実行され、

(6431 lines of diff on the generated HTML).

https://bugs.python.org/issue31793

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@JulienPalard
bpo-31793: Documentation: Specialize smart-quotes for japanese.
e7035d3
@the-knights-who-say-ni the-knights-who-say-ni added the CLA signed label Oct 15, 2017
@bedevere-bot bedevere-bot added the awaiting review label Oct 15, 2017
This was referenced Oct 15, 2017
smart quotes breaks API reference python/docsbuild-scripts#32
Closed
引用符から鉤括弧への変換に不具合がある python-doc-ja/python-doc-ja#774
Closed
vstinner
vstinner approved these changes Nov 6, 2017
View reviewed changes

@vstinner vstinner left a comment

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

It seems like @methane likes it as well:
python/docsbuild-scripts#32 (comment)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@bedevere-bot bedevere-bot added awaiting merge and removed awaiting review labels Nov 6, 2017
@vstinner vstinner added the skip news label Nov 6, 2017
@vstinner
vstinner requested review from merwok, methane and terryjreedy November 7, 2017 00:00
@methane

methane commented Nov 7, 2017

Copy link
Copy Markdown
Member

LGTM. But how about completely disable smart-quote?

When building English document for French, converting "foo" to «foo» will be good idea.
But we "translating" document. When «» is better than "", translators can use «» directly.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Nov 7, 2017

Copy link
Copy Markdown
Member

French: 6.1.3. Fichiers Python « compilés »
https://docs.python.org/fr/dev/tutorial/modules.html#compiled-python-files

Original (english): https://docs.python.org/dev/tutorial/modules.html#compiled-python-files
https://docs.python.org/dev/tutorial/modules.html#compiled-python-files

Do « and » come from the translation, or from smart quotes?

If it comes from the translation, where are the smart quotes used?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@JulienPalard

JulienPalard commented Nov 7, 2017

Copy link
Copy Markdown
Member Author

« and » come from the smart quotes, here's the po file:

#: ../Doc/tutorial/modules.rst:193
msgid "\"Compiled\" Python files"
msgstr "Fichiers Python \"compilés\""
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@methane

methane commented Nov 7, 2017

Copy link
Copy Markdown
Member

Then, without smart-quote, translators can choose quote symbols.
With smart-quote, translators can't.

merwok reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Nov 7, 2017

Copy link
Copy Markdown
Member

« and » come from the smart quotes, here's the po file: (...)

Ok. So I undertand that all quotes would have to be modified in the french PO file if we disable smart quotes for all languages. I dislike requesting the french translators to modify so many translations :-/

What is the impact of this change for the japanese PO translation? Does it break anything?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Nov 7, 2017

Copy link
Copy Markdown
Member

@methane: "With smart-quote, translators can't."

Hum. I don't understand something. The bpo title is "Allow to specialize smart quotes in documentation translations". What is the purpose of this PR in this case?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Nov 7, 2017

Copy link
Copy Markdown
Member

@methane: "With smart-quote, translators can't."

Sorry, I don't understand well the consequences of smart quotes.

Let's say that this PR is merge.

  • Original text: msgid ""Compiled" Python files"
  • French translation: msgstr "Fichiers Python "compilés""
  • (Hypothetical) Japanese translation: msgstr "Fichiers Python "compilés""
  • French rendered as: Fichiers Python « compilés »
  • Japanese rendered as: Fichiers Python "compilés"
  • (Hypothetical) Japanese translation 2: msgstr "Fichiers Python [compilés]"
  • (Hypothetical) Japanese translation 2 rendered as: Fichiers Python [compilés]

Smart quotes only changed the rendered text if the translation keeps "..." quotes, no? But with this PR, "..." is left unchanged. Japanese translators are free to replace "..." quotes with whatever else in the PO file, no?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@JulienPalard

JulienPalard commented Nov 7, 2017

Copy link
Copy Markdown
Member Author

A question is "Did smart quote been introduced just to fix translation of lazy translators, or is it a really interesting feature even for good translators". If it's "just to fix lazyness" I would be OK to drop it, let's do things properly in the first place.

I searched an answer to this question and found http://docutils.sourceforge.net/docs/user/smartquotes.html

Looks like SmartQuotes is also used to transform --- into — so it look like they're used by the english documentation too, not only translations, some can be seen around: https://docs.python.org/3/distutils/apiref.html#module-distutils.filelist

So completly removing smartquotes will also ask for english documents to be reviewed (actually a single sed is probably enough).

The smartquotes documentat also say it may be better to do it right in the first place than relying on smartquotes:

Even if you do care about accurate typography, you still might want to think twice before "auto-educating" the quote characters in your documents. As there is always a chance that the algorithm gets it wrong, you may instead prefer to use the compose key or some other means to insert the correct Unicode characters into the source.

If we're going to remove smartquotes, I'm OK to review french translation myself to fix quotes, let's not make this a blocking issue.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@vstinner

vstinner commented Nov 7, 2017

Copy link
Copy Markdown
Member

--- hyphen is very common in the Python documentation:

haypo@selma$ grep -R ' --- ' Doc|wc -l
469
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@methane

methane commented Nov 7, 2017

Copy link
Copy Markdown
Member

We use 「おはよう」 for Japanese sentence, and both of 「Good morning!」and "Good morning!" are OK.
But in technical document, we don't use 「options」. We use "options" instead.

So this pull request is OK for Japanese document.
I just wonder if French translators use « when they want «, and use " when they want it.
But I don't know much about French. If you're OK for smart-quote, let's merge it.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

methane
methane approved these changes Nov 7, 2017
View reviewed changes
@JulienPalard

JulienPalard commented Nov 7, 2017

Copy link
Copy Markdown
Member Author

I searched a a few minutes in the french documentation and didn't found a paragraph where " …. " would be better than the imposed-by-smartquotes « …», and never encontered one in the past, but we only translated 1/4 of the whole, so we may miss some smartquote induced problems in the future, I can't know.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@merwok

merwok commented Nov 7, 2017 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

It seems to me that the automatic replacement of --- with — and straight quotes to correct French quotes is a work-around for poor Unicode support in input methods and displays. Maybe it was or is useful for docs extracted from docstrings that are ASCII-only, like the rule for the stdlib. But for translations in 2017, I would not rely on these automatic replacements but use the real characters directly in po files: accents, chevron quotes, ™ symbols, long dashes and all. It really isn’t fun to read or edit source full of \uXXX or &xxx; escapes.

(Same rule for non-breaking spaces if the Python French translation uses the France variant; in Québec or Switzerland there are no spaces.)

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@merwok

merwok commented Nov 7, 2017 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

After reading most recent replies: if having straight quotes in source and using the automated replacement works for Julien, then +1! And if desired by the translation team, the source could be moved to correct quotes over time, removing the need for the transform.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@methane
methane merged commit 5a66c8a into python:master Nov 7, 2017
@bedevere-bot bedevere-bot removed the awaiting merge label Nov 7, 2017
@methane methane added needs backport to 2.7 labels Nov 7, 2017
@miss-islington

miss-islington commented Nov 7, 2017

Copy link
Copy Markdown
Contributor

Thanks @JulienPalard for the PR, and @methane for merging it 🌮🎉.. I'm working now to backport this PR to: 2.7.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@miss-islington

miss-islington commented Nov 7, 2017

Copy link
Copy Markdown
Contributor

Thanks @JulienPalard for the PR, and @methane for merging it 🌮🎉.. I'm working now to backport this PR to: 3.6.
🐍🍒⛏🤖

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot

bedevere-bot commented Nov 7, 2017

Copy link
Copy Markdown

GH-4324 is a backport of this pull request to the 2.7 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the needs backport to 2.7 label Nov 7, 2017
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Nov 7, 2017
@JulienPalard @miss-islington
bpo-31793: Doc: Specialize smart-quotes for Japanese (pythonGH-4006)
5d3dbbb
(cherry picked from commit 5a66c8a)
miss-islington pushed a commit to miss-islington/cpython that referenced this pull request Nov 7, 2017
@JulienPalard @miss-islington
bpo-31793: Doc: Specialize smart-quotes for Japanese (pythonGH-4006)
b96a984
(cherry picked from commit 5a66c8a)
@bedevere-bot

bedevere-bot commented Nov 7, 2017

Copy link
Copy Markdown

GH-4325 is a backport of this pull request to the 3.6 branch.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@bedevere-bot bedevere-bot removed the needs backport to 3.6 label Nov 7, 2017
methane pushed a commit that referenced this pull request Nov 7, 2017
@miss-islington @methane
bpo-31793: Doc: Specialize smart-quotes for Japanese (GH-4006)
47eaaa5
(cherry picked from commit 5a66c8a)
methane pushed a commit that referenced this pull request Nov 7, 2017
@miss-islington @methane
bpo-31793: Doc: Specialize smart-quotes for Japanese (GH-4006)
64f1049
(cherry picked from commit 5a66c8a)
embray pushed a commit to embray/cpython that referenced this pull request Nov 9, 2017
@JulienPalard @embray
bpo-31793: Doc: Specialize smart-quotes for Japanese (pythonGH-4006)
094a803
@adrianliaw adrianliaw mentioned this pull request Aug 16, 2018
3.7 translation of tutorial/venv.po python/python-docs-zh-tw#17
Merged
JulienPalard added a commit to JulienPalard/cpython that referenced this pull request Sep 15, 2018
@JulienPalard
Doc: Disable smartquotes for zh-tw translations.
050c38b
See: https://mail.python.org/pipermail/doc-sig/2018-August/004079.html
and: python#4006
This was referenced Sep 15, 2018
Doc: Disable smartquotes for zh-tw and fr translations. #9337
Closed
Smartquotes hitting us. python/python-docs-fr#303
Closed
@adrianliaw adrianliaw mentioned this pull request Sep 19, 2018
Doc: Disable smartquotes for zh-tw, fr and ja translations #9423
Merged
adrianliaw added a commit to adrianliaw/cpython that referenced this pull request Oct 11, 2018
@adrianliaw
Doc: Disable smartquotes for ja, fr and zh_TW translations
9c176a9
This is a replacement for python#9337.

* Using conf.py to disable smartquotes through Sphinx instead of using docutils.conf
* Disable smartquotes for Japanese, French and Traditional Chinese translations

See:
* Suggesting to use conf.py: https://mail.python.org/pipermail/doc-sig/2018-September/004084.html
* Original smartquotes issue in ja translations: https://bugs.python.org/issue31793
* Disabling for ja: python#4006
* Smartquotes issue in fr: python/python-docs-fr#303
* Smartquotes issue in zh_TW: https://mail.python.org/pipermail/doc-sig/2018-August/004079.html
@JulienPalard
JulienPalard deleted the issue-31793-smart-quotes branch June 16, 2019 14:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vstinner vstinner vstinner approved these changes
@methane methane methane approved these changes
@terryjreedy terryjreedy Awaiting requested review from terryjreedy
@merwok merwok Awaiting requested review from merwok

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

skip news

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

7 participants

@JulienPalard @methane @vstinner @merwok @miss-islington @bedevere-bot @the-knights-who-say-ni
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.