GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — FIX: pyplot auto-backend detection case-sensitivity fixup by jayaddison · Pull Request #29721 · matplotlib/matplotlib

Conversation

jayaddison commented Mar 9, 2025 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Contributor

PR summary

There seems to be a case-sensitivity bug in the matplotlib.pyplot module code that decides whether to configure backend autodetection, and I think that this is the cause of the bug reported in #29713.

The bug means that a current interactive environment value of TkAgg is not detected as one of the possible values from a set that includes the string tkagg -- and so an auto-backend is not configured, as it would have been previously in v3.8.3 of this library.

A minimal repro example is provided below, for use in a container environment with no graphical/GUI environment available:

>>> import matplotlib.pyplot
>>> matplotlib.pyplot.new_figure_manager()
Traceback (most recent call last):
  File "<python-input-1>", line 1, in <module>
    matplotlib.pyplot.new_figure_manager()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3/dist-packages/matplotlib/pyplot.py", line 551, in new_figure_manager
    _warn_if_gui_out_of_main_thread()
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3/dist-packages/matplotlib/pyplot.py", line 528, in _warn_if_gui_out_of_main_thread
    canvas_class = cast(type[FigureCanvasBase], _get_backend_mod().FigureCanvas)
                                                ~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3/dist-packages/matplotlib/pyplot.py", line 369, in _get_backend_mod
    switch_backend(rcParams._get("backend"))
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/matplotlib/pyplot.py", line 433, in switch_backend
    raise ImportError(
    ...<2 lines>...
            newbackend, required_framework, current_framework))
ImportError: Cannot load backend 'TkAgg' which requires the 'tk' interactive framework, as 'headless' is currently running

PR checklist

"closes #0000" is in the body of the PR description to link the related issue
new and changed code is tested
Plotting related features are demonstrated in an example N/A
New Features and API Changes are noted with a directive and release note N/A
~~Documentation complies with general and docstring guidelines~~ N/A

Closes #29713.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions


          FIX: pyplot auto-backend detection case-sensitivity fixup

f45707d

github-actions Bot added the topic: pyplot API label

Mar 9, 2025

jayaddison mentioned this pull request

Mar 9, 2025

[Bug]: Matplotlib selects TkAgg backend on LXC containers #29713

Closed

jayaddison commented Mar 10, 2025

Copy link

Copy Markdown

Contributor Author

Thinking about how to add test coverage here: I think that modifying/extending the test_backend_fallback_headless and test_backend_fallback_headful test cases might be a reasonable way to demonstrate the problem and/or fix:

matplotlib/lib/matplotlib/tests/test_rcparams.py

Lines 523 to 563 in b37dbe9

	@pytest.mark.skipif(sys.platform != "linux", reason="Linux only")
	def test_backend_fallback_headless(tmp_path):
	env = {**os.environ,
	"DISPLAY": "", "WAYLAND_DISPLAY": "",
	"MPLBACKEND": "", "MPLCONFIGDIR": str(tmp_path)}
	with pytest.raises(subprocess.CalledProcessError):
	subprocess_run_for_testing(
	[sys.executable, "-c",
	"import matplotlib;"
	"matplotlib.use('tkagg');"
	"import matplotlib.pyplot;"
	"matplotlib.pyplot.plot(42);"
	],
	env=env, check=True, stderr=subprocess.DEVNULL)


	@pytest.mark.skipif(
	sys.platform == "linux" and not _c_internal_utils.xdisplay_is_valid(),
	reason="headless")
	def test_backend_fallback_headful(tmp_path):
	if parse_version(pytest.__version__) >= parse_version('8.2.0'):
	pytest_kwargs = dict(exc_type=ImportError)
	else:
	pytest_kwargs = {}

	pytest.importorskip("tkinter", **pytest_kwargs)
	env = {**os.environ, "MPLBACKEND": "", "MPLCONFIGDIR": str(tmp_path)}
	backend = subprocess_run_for_testing(
	[sys.executable, "-c",
	"import matplotlib as mpl; "
	"sentinel = mpl.rcsetup._auto_backend_sentinel; "
	# Check that access on another instance does not resolve the sentinel.
	"assert mpl.RcParams({'backend': sentinel})['backend'] == sentinel; "
	"assert mpl.rcParams._get('backend') == sentinel; "
	"assert mpl.get_backend(auto_select=False) is None; "
	"import matplotlib.pyplot; "
	"print(matplotlib.get_backend())"],
	env=env, text=True, check=True, capture_output=True).stdout
	# The actual backend will depend on what's installed, but at least tkagg is
	# present.
	assert backend.strip().lower() != "agg"

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

jayaddison commented Mar 10, 2025

Copy link

Copy Markdown

Contributor Author

It's possible to re-order some of the import and get_backend code lines in test_backend_fallback_headful to get the test to fail without this fix and pass with it in place -- but I'm not completely certain what the expected logic should be, so I'll open this PR for review/feedback.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

jayaddison marked this pull request as ready for review

March 10, 2025 11:29

timhoffm approved these changes

Mar 10, 2025

View reviewed changes

timhoffm left a comment

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: I was wondering whether this is the right case to handle the case. Alternatives could be

validate_backend() coercing to lower-case. See

matplotlib/lib/matplotlib/rcsetup.py

Line 280 in b37dbe9

def validate_backend(s):

While we could do this, the BackendRegistry generally accepts arbitrary casing and coerces internally. Therefore, it may be slightly more consistent to keep the name as is in rcParams, so that the user always sees the name as given: They may use matplotlib.use("TkAgg") and set "TkAgg" in their matplotlibrc file. Then it's slightly awkward to return the lower-case variant for rcParams["backend_fallback"] - though one could still argue that eager normalization is still the way to go. I have no strong opinion here.
Push everything down to the BackendRegistry. While this would be nice in principle, we have a very specific request here: "Is the given backend name an available interactive backend, but not webagg or nbagg" and that's not easy to express as a high-level BackendRegistry API that could support arbitrary casing.

I therefore think the present solution is right.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

jayaddison commented Mar 10, 2025

Copy link

Copy Markdown

Contributor Author

@nileshpatra can you confirm that this fixup (f45707d) resolves the problems you encountered during unit tests for matplotlib-dependent packages? I think it should do, but it would be good to make sure.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

ianthomas23 commented Mar 11, 2025

Copy link

Copy Markdown

Member

Thanks @jayaddison, this is a bug that I introduced and the fix here looks good to me. I think you are looking in the right place for a test, but it might be a little awkward due to the subprocess approach.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

jayaddison commented Mar 11, 2025

Copy link

Copy Markdown

Contributor Author

Thank you @ianthomas23 @timhoffm for the code reviews. Also, I think I've figured out what could be some viable test coverage:

Given that test_backend_fallback_headless tests a negative/failure case (the ability to open a tkagg output when headless mode is active is expected to cause a failure), I think that test coverage for this change should add a second headless test that presents a successful rendering case. And to do that, it should be possible I think to import matplotlib.pyplot followed by creating a plot directly (no matplotlib.use / other imports required).

I've tested that theory in a shell with and without the fix in place and it seems to behave as expected; I'll try adding it to the test suite in a few moments.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

github-actions Bot removed the topic: pyplot API label

Mar 11, 2025

This comment was marked as outdated.

This comment was marked as resolved.

jayaddison marked this pull request as draft

March 11, 2025 13:08

This comment was marked as outdated.

jayaddison mentioned this pull request

Mar 11, 2025

[Bug]: Unit tests: MacOS 14 failures: gi-invoke-error-quark #29732

Open

This comment was marked as outdated.

github-actions Bot added the topic: pyplot API label

Mar 11, 2025

This comment was marked as resolved.

github-actions Bot removed the topic: pyplot API label

Mar 11, 2025

jayaddison commented

Mar 11, 2025

View reviewed changes

Comment thread

lib/matplotlib/tests/test_rcparams.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

github-actions Bot added the topic: pyplot API label

Mar 12, 2025

jayaddison marked this pull request as ready for review

March 12, 2025 19:35

tacaswell added this to the v3.10.2 milestone

Mar 12, 2025

tacaswell commented Mar 12, 2025

Copy link

Copy Markdown

Member

Can we squash this to one or two commits?

jayaddison reacted with thumbs up emoji

All reactions

👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.


          TST: coverage of backend fallback in headless mode

2ed08c1

Co-authored-by: Ian Thomas <[email protected]>

nileshpatra commented Mar 13, 2025 via email

Copy link

Copy Markdown

Now that all comments are addressed, could this be merged?

On 13 March 2025 2:18:25 am IST, Thomas A Caswell ***@***.***> wrote: tacaswell left a comment (matplotlib/matplotlib#29721) Can we squash this to one or two commits? -- Reply to this email directly or view it on GitHub: #29721 (comment) You are receiving this because you were mentioned. Message ID: ***@***.***>

Best, Nilesh

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

ianthomas23 approved these changes

Mar 14, 2025

View reviewed changes

ianthomas23 left a comment

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jayaddison. CI failure is unrelated.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

ianthomas23 merged commit 309d380 into matplotlib:main

Mar 14, 2025

meeseeksmachine pushed a commit to meeseeksmachine/matplotlib that referenced this pull request

Mar 14, 2025


          Backport PR matplotlib#29721: FIX: pyplot auto-backend detection case…

5090b26

…-sensitivity fixup

meeseeksmachine mentioned this pull request

Mar 14, 2025

Backport PR #29721 on branch v3.10.x (FIX: pyplot auto-backend detection case-sensitivity fixup) #29754

Merged

jayaddison commented Mar 14, 2025

Copy link

Copy Markdown

Contributor Author

Thank you @ianthomas23!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

jayaddison deleted the issue-29713/auto-backend-case-sensitivity-fixup branch

March 14, 2025 09:31

dstansby added a commit that referenced this pull request

Mar 21, 2025


          Merge pull request #29754 from meeseeksmachine/auto-backport-of-pr-29…

a56911e

…721-on-v3.10.x

Backport PR #29721 on branch v3.10.x (FIX: pyplot auto-backend detection case-sensitivity fixup)

ksunden mentioned this pull request

May 9, 2025

merge up 3.10.3 #30031

Closed

5 tasks

ksunden mentioned this pull request

Jul 25, 2025

Manual Backport PR #30195 on branch v3.10.x (ci: Enable wheel builds on Python 3.14) #30356

Merged

5 tasks

ksunden mentioned this pull request

Aug 7, 2025

merge up v3.10.5 #30401

Merged

5 tasks

Reviewers

ianthomas23

ianthomas23 approved these changes

timhoffm

timhoffm approved these changes

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

topic: pyplot API

Projects

None yet

Milestone

v3.10.3

Development

Successfully merging this pull request may close these issues.

[Bug]: Matplotlib selects TkAgg backend on LXC containers

Uh oh!

There was an error while loading. Please reload this page.

5 participants

Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.

GeezerVison

Conversation

jayaddison commented Mar 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR summary

PR checklist

Uh oh!

jayaddison commented Mar 10, 2025

Uh oh!

jayaddison commented Mar 10, 2025

Uh oh!

timhoffm left a comment

Choose a reason for hiding this comment

Uh oh!

jayaddison commented Mar 10, 2025

Uh oh!

ianthomas23 commented Mar 11, 2025

Uh oh!

jayaddison commented Mar 11, 2025

Uh oh!

This comment was marked as outdated.

This comment was marked as resolved.

This comment was marked as outdated.

This comment was marked as outdated.

This comment was marked as resolved.

Uh oh!

tacaswell commented Mar 12, 2025

Uh oh!

nileshpatra commented Mar 13, 2025 via email

Uh oh!

ianthomas23 left a comment

Choose a reason for hiding this comment

Uh oh!

jayaddison commented Mar 14, 2025

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

jayaddison commented Mar 9, 2025 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.