GeezerVison is showing github.com — simplify screenshot testing by kushalkolar · Pull Request #447 · fastplotlib/fastplotlib
Conversation
kushalkolar
commented
Mar 23, 2024
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
use ideas from pygfx/pygfx#684
closes #446
A threshold of 0.2 seemed like it was too lenient, I was able to change the color of one of the circles in the line collection examples and the RMSE was 0.09
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
kushalkolar
commented
Mar 23, 2024
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
|
This seems to work quite well. The ground truths were generated on my RX 570 at home, the largest diff was with the scatter_size example, due to the text. The diff on github actions was slightly over 0.02. I also tested on the GPUs on the Flatiron cluster, the diff for that example is 0.020826.
There seems to be no difference between the diffs on Nvidia Tesla V100 and Nvidia A100 on the cluster, at least Nvidia is consistent about one think I guess 😂
EDIT: Just tested on the Nvidia H100 and I get a max diff of 0.02077 on the scatter_size example. The diffs are a bit difference than on the V100 and A100 so I guess Nvidia actually isn't consistent.
diffs for all screenshots on a Tesla V100 on the Flatiron cluster for reference:
This is the diff between the Rx 570 and the overpowered Nvidia GPUs
examples/tests/test_examples.py::test_that_we_are_on_lavapipe DiscreteGPU Vulkan
examples/tests/test_examples.py::test_example_screenshots[gridplot] 0.0025418061938043534
examples/tests/test_examples.py::test_example_screenshots[gridplot_non_square] 0.0017253794384335269
examples/tests/test_examples.py::test_example_screenshots[heatmap] 0.0026005766798322047
examples/tests/test_examples.py::test_example_screenshots[heatmap_cmap] 0.002662877302239744
examples/tests/test_examples.py::test_example_screenshots[heatmap_data] 0.0020797810741957933
examples/tests/test_examples.py::test_example_screenshots[heatmap_vmin_vmax] 0.002914066889570422
examples/tests/test_examples.py::test_example_screenshots[image_cmap] 0.0019563093643064503
examples/tests/test_examples.py::test_example_screenshots[image_rgb] 0.0014489276911573798
examples/tests/test_examples.py::test_example_screenshots[image_rgbvminvmax] 0.0005612692156754262
examples/tests/test_examples.py::test_example_screenshots[image_simple] 0.0016309417498936698
examples/tests/test_examples.py::test_example_screenshots[image_vminvmax] 0.0005612692156754262
examples/tests/test_examples.py::test_example_screenshots[line] 0.0029302534153462386
examples/tests/test_examples.py::test_example_screenshots[line_cmap] 0.003085553070868819
examples/tests/test_examples.py::test_example_screenshots[line_colorslice] 0.0031247686572509034
examples/tests/test_examples.py::test_example_screenshots[line_dataslice] 0.0023580007672577286
examples/tests/test_examples.py::test_example_screenshots[line_present_scaling] 0.0027367853208208497
examples/tests/test_examples.py::test_example_screenshots[line_collection] 0.003779845416274792
examples/tests/test_examples.py::test_example_screenshots[line_collection_cmap_values] 0.0023366316256708897
examples/tests/test_examples.py::test_example_screenshots[line_collection_cmap_values_qualitative] 0.002891149865417338
examples/tests/test_examples.py::test_example_screenshots[line_collection_colors] 0.0`022398405285301453
examples/tests/test_examples.py::test_example_screenshots[line_stack] 0.005911900712291164
examples/tests/test_examples.py::test_example_screenshots[scatter] 0.012353324462660456
examples/tests/test_examples.py::test_example_screenshots[scatter_cmap] 0.0030825225024386587
examples/tests/test_examples.py::test_example_screenshots[scatter_colorslice] 0.004020251842124368
examples/tests/test_examples.py::test_example_screenshots[scatter_dataslice] 0.013067512988075274
examples/tests/test_examples.py::test_example_screenshots[scatter_present] 0.013810446602977897
examples/tests/test_examples.py::test_example_screenshots[scatter_size] 0.020826465412837777
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
kushalkolar
commented
Mar 23, 2024
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
|
I think that we should figure out how to make the heatmap test screenshot have more of the actual heatmap, not sure why it's small in the top left corner.
Or maybe think about getting rid of heatmap since the Texture dimension limit seems to no longer apply with WGPU for the past while, need to investigate this 🤔
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Reviewers
clewis7
Awaiting requested review from clewis7
clewis7 is a code owner
almarklein
Awaiting requested review from almarklein
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Milestone
No milestone
Development
Successfully merging this pull request may close these issues.
better image comparison for tests
Uh oh!
There was an error while loading. Please reload this page.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
