GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — scrub kwarg and use one figure for all of the tests by ivanov · Pull Request #56 · matplotlib/matplotlib

Conversation

@ivanov

@ivanov ivanov commented Mar 24, 2011

Copy link
Copy Markdown
Member

The changes here fix a substantial memory leak in the test suite which was caused by generating a new figure for every test but never closing any of them. The memory usage for the test suite went from being monotonically increasing ramp that goes up to ~800MB with each test, to being a much more reasonable ~100MB.

To implement this, all of the tests which created new figures now just use one particular figure -- figure(1) -- which is also cleared just before the test.

Previously, clearing a figure removed all of the artists but kept around the state of the subplotpars, which caused problems for all tests that ran after a change to subplot parameters was made in the test suite, so I added a 'scrub' keyword argument to clf to also reset the subplot parameters, and also added a rcParam['figure.autoscrub'] to control the behavior of when clf is called without explicitly specifying 'scrub'.

These new tests were documented in both the CHANGELOG and in docs/api/api_changes, and the docs built fine (with all of the links and format working as intended)

ok, i'm closing this one in favor of #96

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
ivanov added 4 commits March 22, 2011 17:09
@ivanov
always use figure(1)
0009f1b
@ivanov
added 'scrub' keyword to fig.clf()
3494bba
@ivanov
more updates to the scrub, fixed some tests
1532b7e
@ivanov
documented new scrub kwarg for clf
aecc1e2
@WeatherGod

WeatherGod commented Mar 24, 2011

Copy link
Copy Markdown
Member

All my computers thank you greatly (especially my poor EeePC). I haven't tested this yet, though, but this has been a thorn in my side for a long time. Tests take a long time for me to run because my computer starts thrashing the swap-space, and then cleaning out the virtual memory at the end of the tests takes a long time.

Thanks!

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

jkseppan
jkseppan reviewed Mar 24, 2011
View reviewed changes
Comment thread lib/matplotlib/figure.py

@jkseppan jkseppan Mar 24, 2011

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IMHO the rc parameter is overkill. We could default to False and let users who like the scrubbing clear define an alias def scrub(): clf(scrub=True), or we could even define it for them.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

@ivanov ivanov Mar 29, 2011

Copy link
Copy Markdown
Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the reason I went with the kwarg is because I think users would have a reasonable expectation to have the clear figure command to clear the figure completely, as opposed to having to call a separate command for doing that. But given that others might want to keep the old behavior, we can leave as a user configurable option. The new default behavior (figure.autoscrub = True) allows for the plt.clf() commands to stay as is in the rest of the pull request (though I realize you don't like the code duplication).

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
ivanov added 2 commits March 24, 2011 12:55
@ivanov
added a test for scrub.
5805de2
@ivanov
also scrub hspace and wspace
500a96e
@ivanov

ivanov commented Apr 21, 2011

Copy link
Copy Markdown
Member Author

any objections to merging this?

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ivanov ivanov closed this Apr 21, 2011
@ivanov ivanov reopened this Apr 22, 2011
@ivanov

ivanov commented Apr 22, 2011

Copy link
Copy Markdown
Member Author

apologies, i pushed the comment and close button instead of simply comment

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@efiring

efiring commented Apr 24, 2011

Copy link
Copy Markdown
Member

Paul, I think this has gotten too complicated, so I don't recommend merging this changeset. Instead, I recommend that your close-fig branch, or some variation of it, be applied to v1.0.x and merged into master ASAP. Failing to close the figures in the tests is a real bug, and should be fixed in the maintenance branch. There is another major bug in testing in master that I would like to fix, but to avoid more complicated merges, I would like to see the close patch applied first.
Resetting the subplot parameters is a separate issue, and should not get scrambled up with the basic bug fix.
Regarding that, I agree with Jouni that an rcParam is overkill; the rc system should be used sparingly. I get the impression that all we need may be a figure method that resets subplot params.
The value of reusing a figure in testing sounds marginal at best; but if it is to be done, it should be a separate changeset in master.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@efiring

efiring commented Apr 24, 2011

Copy link
Copy Markdown
Member

More miscellaneous comments:
It is not at all clear to me that a user would expect or want clf to reset the subplot params; and if a user did, it would not be obvious that that is what a kwarg called "scrub" is for. My guess is that in most cases where a user really wants to reuse a figure, the user will not want to have the subplot params reset. Better to leave clf() simple, let it just continue to delete artists. If one really wants a clean slate, one can start with a new figure--and even then one has to be careful because there are other ways that global state can be altered. Not just via the rc system, but via units support, and probably other ways I haven't thought of. (The units support is currently fouling up a subsequent test; this is what I want to get fixed as soon as the close/clf issue is resolved.)
The reason close() is slow is that it is calling gc.collect() to ensure cyclic references are cleaned up immediately. Maybe this is overkill, but I suspect it is better left in place than removed or made optional. This might be worth some investigation.
We can probably save 30-40 seconds on the tests merely by modifying the hexbin test; again, I want to do that after close/clf is resolved.
I suspect it would also be possible to speed up the tests a bit by modifying the image_comparison decorator so that it takes a function returning a figure, and then uses the savefig method on that figure to generate the various file types. That would cut the number of figure generations down by a factor of 3. That decorator is horrendously complicated, though.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mdboom

mdboom commented Apr 25, 2011

Copy link
Copy Markdown
Member

After thinking on this for a while, I think I agree. I think fixing the
memory consumption in the tests is important, but the public API should be
driven by the needs of general usage. In that latter context, the "scrub"
kwarg doesn't seem terribly useful.

But let's get all those figures closing in the tests come hell or high water
-- the difference is staggering!

On Sun, Apr 24, 2011 at 7:10 PM, efiring <
[email protected]>wrote:

I suspect it would also be possible to speed up the tests a bit by
modifying the image_comparison decorator so that it takes a function
returning a figure, and then uses the savefig method on that figure to
generate the various file types. That would cut the number of figure
generations down by a factor of 3. That decorator is horrendously
complicated, though.

That's a good idea.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@mdboom

mdboom commented Apr 26, 2011

Copy link
Copy Markdown
Member

I have some time today -- I'm going to look into cleaning up that decorator and see if we can save the figure regeneration time -- it's also a good place to automatically close figures when done which probably addresses the problem this pull request addresses.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ivanov

ivanov commented Apr 26, 2011

Copy link
Copy Markdown
Member Author

ok, i'm closing this one in favor of #96

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ivanov ivanov closed this Apr 26, 2011
@richbwood richbwood mentioned this pull request Dec 19, 2012
test_pcolormesh hangs #1609
Closed
fariza added a commit to fariza/matplotlib that referenced this pull request Sep 22, 2016
@fariza
Merge pull request matplotlib#56 from fgoudreault/starting_process_doc
cb69a06
Starting doc
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@ivanov @WeatherGod @efiring @mdboom @jkseppan
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.