GeezerVison

These look great! I'll take a closer look soon, but here's some info that I talked with folks at SciPy2024 about:

• We have a 2D colour map implementation in swiftsimio: https://swiftsimio.readthedocs.io/en/latest/visualisation/tools.html#d-color-maps that is maybe worth looking at
• A key component of this is the ability to load a 2D colour map from file, which makes reproducibility much easier and allows for easier tuning in image editors.
• There's a very famous 2D colour map (shown there) that was used for the Millennium Simulation: https://wwwmpa.mpa-garching.mpg.de/galform/virgo/millennium/

These look great! I'll take a closer look soon, but here's some info that I talked with folks at SciPy2024 about:

😄

I think we try to save the discussion on specific colormaps for a later PR, but the Millennium colormap sounds to me like the kind of thing to include.

For now, there is colors.BivarColormapFromImage(name, np.array) that lets you make colormaps from numpy arrays of shape (n,m,3) or (n,m,4). Based on this, it is easy to make a function that reads from an image file, and it becomes more a question of what the public API should look like. I'll try to keep it in mind and get back to it when it in a later PR, as I have tried to keep this PR limited to just the new colormap classes. If we need additional ways to access them we can easily add them later, for example when working on tutorials/examples.

This is a really nice PR. In addition to the above, I think the only thing that I would say should be super clarified down the line is exactly how two 'clashing' 1D colour maps can be smashed together to always produce in-range values. For instance A maps to [0, 0, 199] and B maps to [0, 0, 205] leading to [0, 0, 404] in 'Add' mode. I'm a little concerned that the existing implementation here can lead to low-contrast and low-brightness images (instead of using brightness-preserving blend modes).

This is a really nice PR. In addition to the above, I think the only thing that I would say should be super clarified down the line is exactly how two 'clashing' 1D colour maps can be smashed together to always produce in-range values. For instance A maps to [0, 0, 199] and B maps to [0, 0, 205] leading to [0, 0, 404] in 'Add' mode. I'm a little concerned that the existing implementation here can lead to low-contrast and low-brightness images (instead of using brightness-preserving blend modes).

This is certainly an issue 😅, but it quickly becomes complicated. While it is perhaps not ideal to simply truncate the colors, it is not clear to me what the alternative is.

That said, if the colormaps are designed with this in mind, it becomes less of an issue.
Consider the following colormaps:

(Example data from: https://arxiv.org/abs/1812.10366)
You will notice that the this combination of colormaps never leaves sRGB space, thus there is no issue. However, as you say, the color palette is perhaps a bit drab (low in saturation).
This colormap is designed with the following design criteria:

1. Each component by itself is perceptually uniform
2. Each component has similar saturation and lightness
3. Equal mixing of the components provides a grayscale
4. No combination leaves sRGB space.

In perceptually linear colorspace (CAM02-LCD) the "rhombohedron" formed by this colormap looks like this:

(3D view of colorspace)

I find that if we want to have colormaps with greater saturation, we have to compromise on the points above and/or represent colors in a different colorspace. I wold really like to do the color mixing in CAM02-LCD directly, but the transformation $sRGB\leftrightarrow CAM02-LCD$ is both complicated and numerically unstable. OkLab¹ is an alternative, but last time I tried I found that it was not suitable for mixing colors around the extremes (black/white).

Alternatively, if we are willing to compromise on point 4. (No combination leaves sRGB space) above, we can get higher saturation:

(3D view of colorspace)

If the data is sparse (i.e. each image is mostly black and the different images are bright in different areas), as is the case in the image above, this tradeoff might be worth it. Fluorescent microscopy (shown above) and element maps from X-ray fluorescence are the two main applications I am aware of where color mixing of 3+ channels is routine, and both of these techniques frequently produce sparse data. (Let me know if you know of another domain frequently uses 3+ color channels)

That said, I think perhaps we should rename the names of combination modes to clarify for the user what is happening.
i.e:

• combination_mode='Add' → combination_mode='sRGB_Add'
• combination_mode='Sub' → combination_mode='sRGB_Sub'

This also makes it easier to add additional combination_mode keywords later.
@JBorrow does this seem sensible to you?

¹ I would like to add an tutorial for how users may subclass MultivariateColormap to create multivariate colorbars that combine in interesting ways, and I think Oklab would make for a good example.

PS: let me know if there are any other way of color mixing that I have not thought of that you want me to try.

Yes, this is really tricky! I agree with some very carefully chosen colour maps, you can get around this problem, and you've done a great job doing that.

The specific blending mode that I would strongly suggest is 'lighten'. This compares n items, and keeps the colour that is the lightest. This is super useful in cases where you have many colour maps (here, if I remember correctly, I was using a perceptually uniform colour map segmented into 7 pieces, each which is assigned to a specific density value which scales with its brightness in a small range), and don't want to pay the 1/n brightness price as this is a sparse situation.

Individual density maps look something like this:

Using add:

Using lighten:

More information on this kind of visualisation is available in the yt docs (though I did not use yt for this...) https://yt-project.org/doc/visualizing/volume_rendering.html

Now, the fundamental problem with this is that it is at some level lossy. You lose information about the low-brightness items if they are 'hidden' behind high-brightness items. But I think this is an acceptable (and in many cases beneficial) choice that should be available as a core feature, and dramatically opens up the visualization possibilities.

@JBorrow

Like this?

(source code)

We can easily add this feature, and I think we should.
However this feature can easily added in a later PR and I think it is best if we do not include it here. (This PR is large enough as it is.)
Once this PR, #28428 and the following PRs that allows for the use of multivariate colormaps in the plotting functions are accepted, I'll make a PR implementing 'lightness' and tag you.

At that point we can discuss what the name should be¹, and if we need to ship the feature with additional colormaps.

¹I would like to have combination_mode = 'choose_max' because that is descriptive of the operation we perform, but lets not make any decisions now :)

Yes, that looks perfect! Thank you, definitely happy to defer this to a later PR. Just thought I would point this out as it's really a critical feature for sparse datasets.

One thing that I will say is that there are already well-understood names for all of these modes; I would strongly suggest sticking with them. They are used in all design, drawing, and video editing programs and are well-understood.

See e.g. https://www.clipstudio.net/how-to-draw/archives/154182

@QuLogic I rebased this, do you want to take another pass and possibly approve this?

@QuLogic thank you for looking at this again, I added a test for repr_png :)

@story645 all your comments should be resolved now :)
Thank you!

I can squash merge, but do you have a preference for the commit message. This is the default:

* MultivarColormap and BivarColormap

Creation and tests for classes containing multivariate and bivariate colormaps.

* __getitem__ for colors.BivarColormap

Adds support for __getitem__ on colors.BivarColormap, i.e.: BivarColormap[0] and BivarColormap[1], which returns (1D) Colormap objects along the selected axes

* Better descriptors for 'Add' and 'Sub' in colors.MultivarColormap

* minor fixes for MultivarColormap and BivarColormap

removal of ColormapBase
Addition of _repr_png_() for MultivarColormap
addition of _get_rgba_and_mask() for Colormap to clean up __call__()

* Allows one to not clip to 0...1 in colors.MultivarColormap.__call__()

Also adds a an improvement to colors.BIvarColormap.__getitem__() so that this returns a ListedColormap object instead of a Colormap object

* Multivariate and bivariate resampling of colormaps

Also removed all set_ functions, and replaced them with with_extremes()

* Corrected stubs for mutlivariate and bivariate colormaps

* minor fixes to colors.py

* Rename 'Add' to 'sRGB_add'

This renames the 'combination_mode' keywords from 'Add' and 'Sub' to  'sRGB_add' and 'sRGB_sub'. This change is intended to provide increased clarity if/when additional keywords are added.

* name as keyword variable and additional tests to multivariate colormaps

* Apply suggestions from code review

Co-authored-by: Elliott Sales de Andrade <[email protected]>

* Fixes based on feedback on review

* removed exposure of multivariate and bivariate colormaps in cm.globals()

* improved docstring for MultivarColormap

* Apply suggestions from code review

Co-authored-by: Elliott Sales de Andrade <[email protected]>

* test _repr_png_() for MultivarColormap

* improved docstring for SegmentedBivarColormap()

also allows SegmentedBivarColormap to take non-square patches.
This was easier that testing that the input is always square.

* updated default arguments in docs for multivar and bivar colormaps

based on code review by @story645

---------

Co-authored-by: Elliott Sales de Andrade <[email protected]>

I was thinking simply

* MultivarColormap and BivarColormap

Creation and tests for the classes MultivarColormap and BivarColormap.

But I have no strong preferences, so do what is right for the project :)