GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Edit Task
  • Edit Related Tasks...
  • Create Subtask
  • Edit Parent Tasks
  • Edit Subtasks
  • Merge Duplicates In
  • Close As Duplicate
  • Edit Commits
  • Edit Mocks
  • Mute Notifications
  • Protect as security issue
  • I18n
  • (Script & term conversion)
  • Chinese-Sites

This view

  • Original page
  • New address

GeezerVison is showing phabricator.wikimedia.org — T302158 Language conversion fixes in extensions using LanguageConverter are leaking conversion rules outside extension tags

Page MenuHomePhabricator
Log In
Create Task
Maniphest T302158

Language conversion fixes in extensions using LanguageConverter are leaking conversion rules outside extension tags
Open, Needs TriagePublicBUG REPORT
Actions

  • Edit Task
  • Edit Related Tasks...
  • Create Subtask
  • Edit Parent Tasks
  • Edit Subtasks
  • Merge Duplicates In
  • Close As Duplicate
  • Edit Related Objects...
  • Edit Commits
  • Edit Mocks
  • Mute Notifications
  • Protect as security issue
Assigned To
None
Authored By
Winston_Sung
Feb 19 2022, 4:33 PM2022-02-19 16:33:43 (UTC+0)
Project Tags
  • I18n (Script & term conversion)
  • Chinese-Sites (Extensions/Skins)
  • MediaWiki-Language-converter (Extensions/Skins)
  • MediaWiki-extensions-InputBox (Backlog)
  • Patch-Needs-Improvement
Referenced Files
None
Subscribers
Aklapper
Arthur2e5
cscott
Func
jayvdb
Liuxinyu970226
Nemo_bis
View All 14 Subscribers

Description

List of steps to reproduce (step by step, including full links if applicable):

  • Adding conversion rules with texts expected to be applied by these rules in extension tag(s)
  • Adding texts covered by the conversion rule in tag(s), which should not be affected (especially before the tag)

What happens?:
Conversion rules are applied to texts outside the extension tag(s), both before-tag texts and after-tag texts

What should have happened instead?:
Conversion rules should only applied in the extension tag.

Software version (if not a Wikimedia wiki), browser information, screenshots, other information, etc.:
(Listed below, see Special:Version)

Links:

  • 1.38.0-alpha (c413c5b), Inputbox (merged) & TemplateData (merged) & SyntaxHighlight (testing) https://patchdemo.wmflabs.org/wikis/f39fb53fde/wiki/首页?variant=zh-tw
  • 1.33.3, PortableInfobox (merged) https://ucptest.fandom.com/zh/wiki/LangConv?variant=zh-tw

Details

Related Changes in Gerrit:
SubjectAuthorRepoBranchLines +/-
LanguageConverter: Make reloadTables() method public for purging out manual conversion rulesFuncmediawiki/coremaster+10 -18
Purge out manual conversion rules after conversion appliedFuncmediawiki/extensions/InputBoxmaster+12 -10
Customize query in gerrit

Related Objects
Search...

  • Task Graph
  • Mentions
StatusSubtypeAssignedTask
OpenFeatureNoneT34943 Auto-conversion between zh-hans & zh-hant in <syntaxhighlight> elements
ResolvedBawolffT180485 Inputbox does not support variants conversion on Chinese sites
OpenBUG REPORTNoneT302158 Language conversion fixes in extensions using LanguageConverter are leaking conversion rules outside extension tags
Mentioned In
T263082: Add <langconvert> parser tag and implement language converter for Balinese (ban)
Mentioned Here
T484: RFC: Scoped language converter
T263082: Add <langconvert> parser tag and implement language converter for Balinese (ban)
rMWc413c5b0ca9f: Merge "api: Check possible null on TitleFactory::makeTitleSafe for incontinue="

Event Timeline

Winston_Sung created this task.Feb 19 2022, 4:33 PM2022-02-19 16:33:43 (UTC+0)
Winston_Sung added a project: MediaWiki-Language-converter.
Winston_Sung added a parent task: T180485: Inputbox does not support variants conversion on Chinese sites.Feb 19 2022, 4:45 PM2022-02-19 16:45:18 (UTC+0)
Winston_Sung updated the task description. (Show Details)
SunAfterRain subscribed.Edited · Feb 19 2022, 5:13 PM2022-02-19 17:13:52 (UTC+0)
Comment Actions

Difference between pre, inputbox, templatedata and syntaxhighlight with similar code

<pre>:
https://patchdemo.wmflabs.org/wikis/f39fb53fde/wiki/首页/pre?variant=zh-tw

<inputbox>:
https://patchdemo.wmflabs.org/wikis/f39fb53fde/wiki/首页/inputbox?variant=zh-tw

<templatedata>:
https://patchdemo.wmflabs.org/wikis/f39fb53fde/wiki/首页/templatedata?variant=zh-tw

<syntaxhighlight>:
https://patchdemo.wmflabs.org/wikis/f39fb53fde/wiki/首页/syntaxhighlight?variant=zh-tw

Func added a comment.Feb 20 2022, 2:40 AM2022-02-20 02:40:45 (UTC+0)
Comment Actions

Note that conversion after extension tag (only) is matching the normal behavior, but it's impossible to achieve.
The thing is, extension tags are processed during parsing, but language conversion on the whole content is done at the very end, after tidy or something.
And language converter doesn't support disabling some type of conversion rules, for cases mentioned above, any rules that can affect global states should be disabled.

Winston_Sung added a comment.Edited · Feb 20 2022, 2:51 AM2022-02-20 02:51:26 (UTC+0)
Comment Actions

It currently act like this:

1. outer text (unexpected: got converted caused by 4., should not be converted)

2. tag inner text without conversion enabled (expected)

3. outer text (unexpected: got converted caused by 4., should not be converted)

4. tag inner text with tag-wide conversion rules added and conversion enabled (expected)

5. outer text (unexpected: got converted caused by 4., should not be converted)

6. conversion rules removed page-wide

7. outer text (not converted caused by 4. added + 6. removed conversion rules)

8. tag inner text without conversion enabled (expected)

9. outer text (not converted caused by 4. added + 6. removed conversion rules)

10. tag inner text with tag-wide conversion rules added and conversion enabled (expected)

11. outer text (not converted caused by 4. added + 6. removed conversion rules but didn't affected by 10.)
gerritbot added a comment.Feb 20 2022, 12:53 PM2022-02-20 12:53:59 (UTC+0)
Comment Actions

Change 763939 had a related patch set uploaded (by Func; author: Func):

[mediawiki/core@master] LanguageConverter: Make reloadTables() method public for purging out manual conversion rules

https://gerrit.wikimedia.org/r/763939

gerritbot added a project: Patch-For-Review.Feb 20 2022, 12:53 PM2022-02-20 12:53:59 (UTC+0)
Func added a project: MediaWiki-extensions-InputBox.Feb 20 2022, 1:11 PM2022-02-20 13:11:29 (UTC+0)
Comment Actions

The behavior of TemplateData is fine, and PortableInfobox doesn't have a project tag, right?

Winston_Sung added a comment.Feb 20 2022, 1:14 PM2022-02-20 13:14:16 (UTC+0)
Comment Actions

PortableInfobox is an extension made by Fandom with ported versions, so it's outside the scope of Wikimedia Phabricator.

gerritbot added a comment.Feb 20 2022, 1:43 PM2022-02-20 13:43:22 (UTC+0)
Comment Actions

Change 763943 had a related patch set uploaded (by Func; author: Func):

[mediawiki/extensions/InputBox@master] Purge out manual conversion rules

https://gerrit.wikimedia.org/r/763943

Shizhao moved this task from Backlog to Extensions/Skins on the Chinese-Sites board.Feb 21 2022, 3:27 AM2022-02-21 03:27:24 (UTC+0)
SunAfterRain mentioned this in T263082: Add <langconvert> parser tag and implement language converter for Balinese (ban).Feb 28 2022, 2:46 PM2022-02-28 14:46:33 (UTC+0)
cscott subscribed.Aug 17 2022, 6:36 PM2022-08-17 18:36:10 (UTC+0)
Comment Actions

See T484: RFC: Scoped language converter as well, and https://wikimedia.slack.com/archives/C024Z8K9CAU/p1660748317850709 for those with access to WMF slack. tl;dr the fundamental issue here is that LanguageConverter for a given language is a global singleton with persistent state. I don't think just making reloadTables public is a good fix, because it doesn't solve the root cause: now all the rules will be wiped after the extension content is processed, which isn't any better than adding rules after the extension content is processed. (T263082 is mentioned here, but it probably has the same issue with language converter rules "leaking" out, although that might be a Feature rather than a Bug in that case.)

Restricted Application added a subscriber: Ericliu1912. · View Herald TranscriptAug 17 2022, 6:36 PM2022-08-17 18:36:11 (UTC+0)
Ericliu1912 unsubscribed.Sep 16 2022, 6:59 AM2022-09-16 06:59:59 (UTC+0)
Winston_Sung moved this task from Extensions/Skins to MediaWiki core on the Chinese-Sites board.Mar 10 2023, 8:43 AM2023-03-10 08:43:15 (UTC+0)
Shizhao moved this task from MediaWiki core to Extensions/Skins on the Chinese-Sites board.Mar 13 2023, 2:12 AM2023-03-13 02:12:14 (UTC+0)
Winston_Sung moved this task from Backlog to Extensions/Skins on the MediaWiki-Language-converter board.Mar 18 2023, 3:48 AM2023-03-18 03:48:52 (UTC+0)
Winston_Sung moved this task from Untriaged to Script & term conversion on the I18n board.Aug 9 2023, 3:14 PM2023-08-09 15:14:18 (UTC+0)
Pppery edited projects, added: Patch-Needs-Improvement; removed: Patch-For-Review.Dec 22 2024, 5:23 PM2024-12-22 17:23:00 (UTC+0)
gerritbot added a comment.Mar 30 2026, 1:09 PM2026-03-30 13:09:41 (UTC+0)
Comment Actions

Change #763939 abandoned by Hashar:

[mediawiki/core@master] LanguageConverter: Make reloadTables() method public for purging out manual conversion rules

https://gerrit.wikimedia.org/r/763939

gerritbot added a comment.Mar 30 2026, 11:17 PM2026-03-30 23:17:35 (UTC+0)
Comment Actions

Change #763939 restored by Thcipriani:

[mediawiki/core@master] LanguageConverter: Make reloadTables() method public for purging out manual conversion rules

https://gerrit.wikimedia.org/r/763939

gerritbot added a project: Patch-For-Review.Mar 30 2026, 11:17 PM2026-03-30 23:17:36 (UTC+0)
Restricted Application removed a project: Patch-Needs-Improvement. · View Herald TranscriptMar 30 2026, 11:17 PM2026-03-30 23:17:37 (UTC+0)
Pppery edited projects, added: Patch-Needs-Improvement; removed: Patch-For-Review.Mar 31 2026, 1:56 AM2026-03-31 01:56:55 (UTC+0)
Log In to Comment
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL · Credits
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.