GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — WIP: New FreeType wrappers by mdboom · Pull Request #5414 · matplotlib/matplotlib

Conversation

@mdboom

@mdboom mdboom commented Nov 5, 2015 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

This isn't done yet, but thought I'd share at this point in the interest of transparency. This is related to the (really old at this point) MEP14.

This deletes the somewhat messy and inefficient FT2Font freetype wrapper with a new one at http://github.com/matplotlib/freetypy, which, being an independent project, should hopefully see more use and eyes from the Python graphics/viz community at large. http://github.com/rougier/freetype-py is the only other project I'm aware of that comes close in scope, but being that it's ctypes-based, it's a non-starter for matplotlib which needs to run in restricted environments. Also, the memory management in FreeType is really tricky, and I don't think it's possible to mange it correctly from ctypes.

This is mostly a refactor and should have no impact on most end users. However, it enables some cool things already:

  • It supports fonts that don't have "legacy" Macintosh metadata. This includes most of the new fonts that have shipped with MS-Windows since Vista or so, which only have UTF-16 metadata.
  • It handles Unicode better. It prefers to use Unicode mappings if available in a font, where the master uses the default mapping of the font which may be in some obscure code page resulting in the wrong glyphs. If there is no unicode mapping in the font, the new wrappers will correctly map unicode to that encoding using the extensive collection of codecs that come with Python.
  • Having a clean path of all the various transformations along the way (including the handling of fixed-point numbers correctly), this finally fixes the "wiggly" baselines problem that has plagued matplotlib for years.

BEFORE:

mathfont_cm_06-expected

AFTER:

mathfont_cm_06

Steps yet to do:

- [ ] Handle AFM file parsing in FreeType, rather than our own Python code (afm.py). This should provide a consistent interface and remove all the blocks that say "if afm... if ttf..."

  • Handle all text layout through a consistent interface. This will remove all the blocks that say "if math... if regular text..."
  • Ditch ttconv in favor of Python code that uses FreeType behind it. This will allow any font to be used in PDF and PS, not just TTF fonts.
  • Ditch font_manager.py and matplotlib's notorious font cache in favor of fontconfig (though that will require writing wrappers to fontconfig).
  • Handle Type 1 fonts in FreeType, rather than our own Python code (type1.py)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@mdboom mdboom added the status: needs review label Nov 5, 2015
@tacaswell tacaswell added this to the proposed next point release (2.1) milestone Nov 5, 2015
@Tillsten

Tillsten commented Nov 6, 2015

Copy link
Copy Markdown
Contributor

Awesome to see work on the text-stuff!
Some questions:
a) Could you expand the point why ctypes or cffi wouldn't work? From an maintenance pov these are much easier to handle.
b) After looking at the code this seem to be a perfect match for cython, or am i wrong here?
c) I toyed around with idea of using the internal freetype-cache system for matplotlib, would that be supported?

Thanks for all the and sorry if this sounds too critical.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mdboom

mdboom commented Nov 6, 2015

Copy link
Copy Markdown
Member Author

a) Could you expand the point why ctypes or cffi wouldn't work? From an maintenance pov these are much easier to handle.

ctypes and cffi both create a major security hole since one can basically open any library and call any address in it, so they are not available in restricted execution environments such as Google App Engine. With the whole world going to containers, I suppose that will become less of an issue over time, but in the meantime, we can't really decrease the number of platforms that matplotlib can run in.

b) After looking at the code this seem to be a perfect match for cython, or am i wrong here?

I actually tried and failed with Cython initially for these wrappers. The very careful memory management required for freetype and the stuff like the FT_OpenArgs with its C callbacks etc. really is far more cumbersome in Cython than in direct C. Plus, Cython requires copying all of the information in the FreeType headers over which is a real pain when trying to support multiple versions of FreeType. For example, FreeType explicitly says to not rely on the numeric values of error codes -- but in Cython we would have to copy those numbers over and then keep them in sync with each revision of FreeType. The FreeType API also provides a lot of things through direct access in structs. Again, as those structures change, a Cython wrapper would potentially have to account for changes in those structs over time. With C, a simple recompile will do.

Above all that, I find Cython really hard to deal with when things go wrong. Trying to run gdb or valgrind over that stuff is a nightmare.

Of course, all of this comes from a bias of being very comfortable with C.

c) I toyed around with idea of using the internal freetype-cache system for matplotlib, would that be supported?

I haven't wrapped any of FreeType's cache stuff. It certainly could be added in this scheme. It doesn't look super useful in a Python context since it's so easy to implement cache-like behavior, and it's reference counted etc. However, it might be the case that FreeType's cache code has less overhead etc. I think I would start by seeing just how much time is spent re-doing work to get a theoretical "best case" for using caching before devoting the considerable effort to use this stuff.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mdboom
mdboom force-pushed the freetypy branch from fb5c645 to 89083cd Compare November 18, 2015 16:23
@mdboom mdboom mentioned this pull request Nov 27, 2015
Incorrect selection of Times New Roman Bold #5574
Closed
@QuLogic

QuLogic commented Dec 3, 2015

Copy link
Copy Markdown
Member

Will this also obsolete #3912?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mdboom

mdboom commented Dec 3, 2015

Copy link
Copy Markdown
Member Author

Will this also obsolete #3912?

This will provide an avenue for handling TTC files in the correct and general way. Probably will do that as a follow-on to this PR, however.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mdboom mdboom mentioned this pull request Dec 9, 2015
Warning message prior to fc-list command #5640
Merged
@jenshnielsen jenshnielsen mentioned this pull request Jan 12, 2016
Repeated warning about fc-list #5836
Closed
@mdboom mdboom mentioned this pull request Jan 18, 2016
Some Microsoft fonts produce unreadable EPS #5862
Closed
@mdboom
mdboom force-pushed the freetypy branch from 89083cd to a60e127 Compare January 22, 2016 14:42
@mdboom

mdboom commented Jan 22, 2016

Copy link
Copy Markdown
Member Author

I've punted on checkbox #1 in the original topic. FreeType can only manage the metrics in an AFM file if you have an associated PFB file -- which we don't ship or need to ship. I think the status quo (using afm.py) is fine there for now. It supports what is now a sort of niche use case (generating PostScript files using only the core 14 fonts).

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mdboom mdboom mentioned this pull request Apr 11, 2016
Fix #5998: Support fallback font correctly #6203
Merged
@mdboom
mdboom force-pushed the freetypy branch from 27aa785 to 068279c Compare April 11, 2016 23:41
mdboom added 15 commits April 11, 2016 19:41
@mdboom
Remove ft2font
038cd9c
@mdboom
Install freetypy from git
5e4c36e
@mdboom
Remove tests and examples that are now obsolete
0cc61e9
...due to the removal of FT2Font
@mdboom
Use freetypy in font_manager.py
d97def9
@mdboom
Move from FT2Font to freetypy for all uses
23c2f07
@mdboom
Update comment
c91d988
@mdboom
Move local freetype building resp. to freetypy
440fc51
@mdboom
Bugfix for AFM fonts
632a6f6
@mdboom
Remove usage of ttconv, in favor of freetypy.subset
72cedd3
@mdboom
Use font config in place of font_manager
4eaa209
@mdboom
Font layout details
04c14db
@mdboom
Bug fixes in new font universe
d8ca0d5
@mdboom
Build freetypy with specific version of freetype
28fb1dc
@mdboom
Install fcpy on Travis
b7d73af
@mdboom
Minor fixes and consistency cleanups
d45129f
mdboom added 2 commits April 29, 2016 14:22
@mdboom
Use new layout API
0d10ec1
@mdboom
Update test images
9d6632f
@howthebodyworks howthebodyworks mentioned this pull request May 11, 2016
do not always import matplotlib librosa/librosa#343
Closed
@jkseppan jkseppan mentioned this pull request Feb 27, 2017
Version 2.0.0_1 breaks OpenType font character spacing on PDF save #8050
Closed
@tacaswell tacaswell mentioned this pull request May 12, 2017
support ttc fonts #8612
Closed
@tacaswell tacaswell modified the milestones: 2.1 (next point release), 2.2 (next next feature release) Aug 29, 2017
@anntzer anntzer mentioned this pull request Nov 13, 2017
New FreeType wrapper. #9763
Closed
@anntzer

anntzer commented Nov 15, 2017 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

For the record FreeType is already using a unicode charmap by default (http://git.savannah.gnu.org/cgit/freetype/freetype2.git/tree/src/base/ftobjs.c#n1370); since FreeType 2.8.1 such a charmap will be automatically generated if the font doesn't include it (https://sourceforge.net/projects/freetype/files/freetype2/2.8.1/). So the second point will become moot once we switch the vendored FT version to >=2.8.1.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@jklymak

jklymak commented Mar 7, 2018

Copy link
Copy Markdown
Member

Closing in lieu of #9763

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@jklymak jklymak closed this Mar 7, 2018
@anntzer

anntzer commented Mar 8, 2018

Copy link
Copy Markdown
Contributor

Actually there's (at least) still one thing that this PR does which #9763 doesn't, which is to replace ttconv (the font subsetter for pdf/ps output). That's actually a task fairly independent from wrapping freetype (which is what #9763 focuses on).

jklymak reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@anntzer anntzer reopened this Mar 8, 2018
@jklymak jklymak added status: needs rebase and removed status: needs review labels May 17, 2018
@jklymak jklymak modified the milestones: needs sorting, v3.0 May 17, 2018
@tacaswell tacaswell modified the milestones: v3.0, v3.1 Jul 9, 2018
@jklymak jklymak mentioned this pull request Oct 6, 2018
replace ttconv for ps/pdf #12418
Closed
@jklymak

jklymak commented Oct 6, 2018

Copy link
Copy Markdown
Member

Opened issue about ttconv and closing this again.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@jklymak jklymak closed this Oct 6, 2018
@anntzer anntzer mentioned this pull request Jan 4, 2020
Please add support for ttc font files (PDF/PS output) #3135
Closed
@anntzer anntzer mentioned this pull request Aug 24, 2020
Created a parameter fontset that can be used in each Text element #18145
Merged
4 tasks done
@anntzer anntzer mentioned this pull request Jan 13, 2021
Add machinery for png-only, single-font mathtext tests. #19261
Merged
7 tasks
@anntzer anntzer mentioned this pull request Feb 7, 2021
Fix over/under mathtext symbols #19314
Merged
5 of 7 tasks
@anntzer anntzer mentioned this pull request May 24, 2021
a bug that latex expression with subscripts exceeds bbox #14177
Closed
@QuLogic QuLogic mentioned this pull request Apr 1, 2025
Update FreeType to 2.13.3 #29816
Merged
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

status: needs rebase

Projects

None yet

Milestone

v3.1.0

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

6 participants

@mdboom @Tillsten @QuLogic @anntzer @jklymak @tacaswell
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.