GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Prepare `CharacterTracker` for advanced font features by QuLogic · Pull Request #30608 · matplotlib/matplotlib

Conversation

@QuLogic

@QuLogic QuLogic commented Sep 27, 2025 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

PR summary

The original code split fonts into subsets based on the character modulo the subset size (determined by font type limits). This had some limitations:

  1. If characters were used from across various font blocks (especially for type 3, whose blocks were only 256 characters), then there would be many, possibly sparsely-populated, font subsets.
  2. Sometimes a single character code may map to multiple glyphs. This is the case if you mix languages (i.e., Add language parameter to Text objects #29794), but the naive tracking would only produce one glyph.
  3. Sometimes multiple characters can map to a single glyph. This is the case for ligatures, and also with complex text shaping (such as Arabic), and this would just fail calling ord on a multi-char string.

To fix this, CharacterTracker now tracks characters and glyphs more closely. Specifically,

  1. for each font, a (character code(s), glyph index)-pair is mapped to a (subset index, subset character code)-pair. This ensures that point 2 above is handled.
  2. If the above map doesn't exist yet, then a subset index/character code is calculated:
    1. if the (singular) character code is in the first block (255 for type 3, or 64k for type 42), then keep the character code the same and put it in subset 0; this preserves the text in those lower ranges if you happen to be looking at a PDF directly
    2. if the (singular) character code is already in subset 0, then bump it to the next available spot; a conflict here means the character is being used with multiple glyphs (i.e., another case for part 2 above)
    3. if the character code is in fact multiple character codes, then also bump to the next available spot as it could never be in the subset 0 (this is part 3 above)
    4. the next available spot is the next character code in the next subset block, if necessary; by filling as needed, this takes care of point 1 above

With these changes, the complex/font features/languages tests in #30607 produce correct results.

PR checklist

  • [n/a] "closes #0000" is in the body of the PR description to link the related issue
  • new and changed code is tested
  • [n/a] Plotting related features are demonstrated in an example
  • [n/a] New Features and API Changes are noted with a directive and release note
  • [n/a] Documentation complies with general and docstring guidelines

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@QuLogic QuLogic added this to the v3.11.0 milestone Sep 27, 2025
@QuLogic QuLogic added this to Font and text overhaul Sep 27, 2025
@github-project-automation github-project-automation Bot moved this to Waiting for other PR in Font and text overhaul Sep 27, 2025
@QuLogic QuLogic moved this from Waiting for other PR to Ready for Review in Font and text overhaul Sep 27, 2025
@github-actions github-actions Bot added backend: ps backend: pdf labels Sep 27, 2025
@QuLogic
QuLogic force-pushed the simpler-track branch from 58874b6 to 5afd71b Compare September 27, 2025 03:21
@QuLogic QuLogic changed the title Deduplicate CharacterTracker.track implementation Prepare CharacterTracker for advanced font features Sep 27, 2025
@QuLogic

QuLogic commented Sep 27, 2025

Copy link
Copy Markdown
Member Author

Note, I think the original commit was small, and the remaining ended up small enough, that I just put them all in this PR.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@QuLogic QuLogic mentioned this pull request Sep 27, 2025
Implement libraqm for vector outputs #30607
Merged
1 of 5 tasks
anntzer
anntzer reviewed Sep 29, 2025
View reviewed changes
Comment thread lib/matplotlib/backends/_backend_pdf_ps.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

anntzer
anntzer reviewed Sep 29, 2025
View reviewed changes
Comment thread lib/matplotlib/backends/_backend_pdf_ps.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

anntzer
anntzer reviewed Sep 29, 2025
View reviewed changes
Comment thread lib/matplotlib/backends/_backend_pdf_ps.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

anntzer
anntzer reviewed Sep 29, 2025
View reviewed changes
Comment thread lib/matplotlib/backends/_backend_pdf_ps.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

anntzer
anntzer reviewed Sep 29, 2025
View reviewed changes
Comment thread lib/matplotlib/backends/_backend_pdf_ps.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

anntzer
anntzer reviewed Sep 29, 2025
View reviewed changes
Comment thread lib/matplotlib/backends/backend_pdf.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

anntzer
anntzer reviewed Sep 29, 2025
View reviewed changes
Comment thread lib/matplotlib/backends/backend_ps.py Outdated
print("%%BeginProlog", file=fh)
if not mpl.rcParams['ps.useafm']:
Ndict += len(ps_renderer._character_tracker.used)
Ndict += sum(map(len, ps_renderer._character_tracker.used.values()), 0)

@anntzer anntzer Sep 29, 2025

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 0 at the end is unneeded.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
anntzer
anntzer reviewed Sep 29, 2025
View reviewed changes
Comment thread lib/matplotlib/backends/backend_ps.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

@QuLogic
Deduplicate CharacterTracker.track implementation
50f76ff
No need to repeat the calculation of subset blocks, but instead offload
it to `track_glyph`.
@QuLogic
QuLogic force-pushed the simpler-track branch from 5e89363 to 662ac58 Compare September 30, 2025 00:27
QuLogic added 2 commits September 30, 2025 00:57
@QuLogic
pdf/ps: Compress subsetted font blocks
8274e17
Instead of splitting fonts into `subset_size` blocks and writing text as
character code modulo `subset_size`, compress the blocks by doing two
things:

1. Preserve the character code if it lies in the first block. This keeps
   ASCII (for Type 3) and the Basic Multilingual Plane (for Type 42) as
   their normal codes.
2. Push everything else into the next spot in the next block, splitting
   by `subset_size` as necessary.

This should reduce the number of additional font subsets to embed.
@QuLogic
pdf: Fix first-block characters using multiple glyph representations
70dc388
If mixing languages, sometimes a single character may use different
glyphs in one document. In that case, we need to give it a new character
code in the next subset, since subset 0 is preserving character codes.
@QuLogic
QuLogic force-pushed the simpler-track branch from 662ac58 to c21a0b0 Compare September 30, 2025 05:18
@QuLogic

QuLogic commented Sep 30, 2025

Copy link
Copy Markdown
Member Author

OK, I've handled all your comments, I think. I also fixed subsetting in the PostScript backend, noted above.

There are 3 test image changes:

  • 2 PDF tests think that the fourth emoji moved slightly; I think this is because the characters on the line are now from the same font instead of split between emoji/not. But this seems very small, as viewing it at 100% doesn't really seem to show any difference.
  • 1 EPS test change; this is because subsetting is now "real", so the switch from type 3 to type 42 no longer happens. Ghostscript seems to convert those to raster a little different, but on the upside, now the Computer Modern and DejaVu fonts look the same.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

anntzer
anntzer reviewed Sep 30, 2025
View reviewed changes
Comment thread lib/matplotlib/backends/backend_pdf.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

anntzer
anntzer approved these changes Sep 30, 2025
View reviewed changes

@anntzer anntzer left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a minor point regarding a comment.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
QuLogic added 2 commits September 30, 2025 03:23
@QuLogic
pdf: Support multi-character glyphs when subsetting
df670cf
For ligatures or complex shapings, multiple characters may map to a
single glyph. In this case, we still want to output a single character
code for the string using the font subset, but the `ToUnicode` map
should give back all the characters.
@QuLogic
ps: Fix font subset handling
ed5e074
Previously, this was supposed to "upgrade" type 3 to type 42 if the
number of glyphs overflowed. However, as `CharacterTracker` can suggest
a new subset for other reasons (i.e., multiple glyphs for the same
character or a glyph for multiple characters may go to a second subset),
we do need proper subset handling here as well.

Since that is now done, we can drop the "promotion" from type 3 to type
42, as we don't get too many glyphs in each embedded font.
@QuLogic
QuLogic force-pushed the simpler-track branch from c21a0b0 to d781040 Compare September 30, 2025 07:23
ksunden
ksunden approved these changes Oct 2, 2025
View reviewed changes
@QuLogic
QuLogic force-pushed the simpler-track branch from d781040 to ed5e074 Compare October 2, 2025 19:42
@QuLogic

QuLogic commented Oct 2, 2025

Copy link
Copy Markdown
Member Author

Removed the image changes (and moved them to the text-overhaul-figures branch) in preparation for merging.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@QuLogic

QuLogic commented Oct 2, 2025

Copy link
Copy Markdown
Member Author

Linting issues are known (#30626), so merging over those.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@QuLogic
QuLogic merged commit ed4ca6c into matplotlib:text-overhaul Oct 2, 2025
3 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@github-project-automation github-project-automation Bot moved this from Ready for Review to Done in Font and text overhaul Oct 2, 2025
@QuLogic
QuLogic deleted the simpler-track branch October 2, 2025 23:00
wavebyrd pushed a commit to wavebyrd/matplotlib that referenced this pull request Mar 13, 2026
@QuLogic
Merge pull request matplotlib#30608 from QuLogic/simpler-track
b1b6ed0
Prepare `CharacterTracker` for advanced font features
QuLogic added a commit to QuLogic/matplotlib that referenced this pull request Apr 10, 2026
@QuLogic
Update test images for font/text overhaul
12d2a4d
This includes images changes for the following pull requests / commits:

* [Fix center of rotation with
  rotation_mode='anchor'](matplotlib#29199)
  (c44db77)
* [Remove ttconv backwards-compatibility
  code](matplotlib#30145)
  (8caff88)
* [Remove kerning_factor from
  tests](matplotlib#29816)
  (7b4d725)
* [Set text hinting to
  defaults](matplotlib#29816)
  (8255ae2)
* [Update FreeType to
  2.13.3](matplotlib#29816)
  (89c054d)
* [Implement text shaping with
  libraqm](matplotlib#30000)
  (b0ded3a,
  9813523)
* [Add language parameter to Text
  objects](matplotlib#29794)
  (7ce8eae)
* [Fix auto-sized glyphs with BaKoMa
  fonts](matplotlib#29936)
  (3ba2c13)
* [pdf: Improve text with characters outside embedded font
  limits](matplotlib#30512)
  (b70fb88,
  6cedcf7)
* [Prepare `CharacterTracker` for advanced font
  features](matplotlib#30608)
  (8274e17,
  70dc388,
  df670cf,
  ed5e074)
* [Add font feature API to
  Text](matplotlib#29695)
  (972a688)
* [Fix spacing in r"$\max
  f$"](matplotlib#30715)
  (4a99a83)
* [Implement libraqm for vector
  outputs](matplotlib#30607)
  (bd17cd4)
* [Drop the FT2Font intermediate
  buffer](matplotlib#30059)
  (9d7d7b4)
* [Rasterize dvi files without
  dvipng](matplotlib#30039)
  (7627118)
* [Update bundled FreeType and HarfBuzz
  libraries](matplotlib#30938)
  (a161658,
  9619bcc)
* [Fix positioning of wide mathtext
  accents](matplotlib#31069)
  (c2fa7ba)
* [Refactor RendererAgg.draw_{mathtext,text,tex} to use same base
  algorithm](matplotlib#31085)
  (931bcf3)
* [Implement TeX's fraction and script
  alignment](matplotlib#31046)
  (94ff452,
  4bfa0f9,
  1cd8510)
* [Fix confusion between text height and ascent in metrics
  calculations](matplotlib#31107)
  (60f2310)
* [mathtext: Fetch quad width & axis height from font
  metrics](matplotlib#31110)
  (692df3f,
  383028b)
* [mathtext: add mathnormal and distinguish between normal and italic
  family](matplotlib#31121)
  (a6913f3)
* [ENH: Ignore empty text for
  tightbbox](matplotlib#31285)
  (d772043)
* [Drop axis_artist tickdir image compat, due to text-overhaul
  merge](matplotlib#31281)
  (2057583)
* [text: Use font metrics to determine line
  heights](matplotlib#31291)
  (3ab6a27,
  d961462,
  97f4943)
* [ps/pdf: Override font height metrics to support AFM
  files](matplotlib#31371)
  (e0913d4)
* [TST: Cleanup back-compat code in tests touched by text
  overhaul](matplotlib#31295)
  (7c33379)
* [TST: Set tests touched by text overhaul to mpl20
  style](matplotlib#31300)
  (41c4d8d)
@QuLogic QuLogic mentioned this pull request Apr 10, 2026
Font and text overhaul #30161
Merged
4 tasks done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@anntzer anntzer anntzer approved these changes
@ksunden ksunden ksunden approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

backend: pdf backend: ps

Projects

Font and text overhaul
Status: Done

Milestone

v3.11.0

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@QuLogic @anntzer @ksunden
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.