Description

eriknw

opened on Feb 5, 2024

Issue body actions

We have found that many applications want 3d arrays, and some may need arrays of even higher dimensions.

This is typically handled by stacking arrays such that a single dimension of a GraphBLAS Matrix encodes multiple logical dimensions of a tensor. Performing operations between such objects often requires reorganizing the data so the logical axes align. This is awkward to do today. Can we make it easier?

One option is to add something like np.moveaxis (https://numpy.org/doc/stable/reference/generated/numpy.moveaxis.html). To make this general, I think we would need to include how the dimensinos of the logical tensor are encoded in Vector or Matrix dimensions, and then how to reorganize it. Some operations--particularly in 3D--can probably be optimized pretty well, and we should try to make things fast when possible.

A quarter-baked proposal may look something like this: A.reorganize(src_shape=((2, 3), (4,2)), dst_axes=((0, 1), (3, 2))). This effectively does a transpose by swapping the last two axes. It may be more natural at times to think in terms of source shape and source indices, or in terms of destination shape and indices (or some mix of the two). This has the potential of being confusing in general, but should hopefully make sense when you actually have data that you need to manipulate.

How far would such an operation go to making python-graphblas more usable for tensors? What else do we need for common use cases?

Activity

1. eriknw

   added

   discussionDiscussing a topic with no specific actions yetDiscussing a topic with no specific actions yet

   featureSomething is missingSomething is missing

   on Feb 5, 2024
2. SultanOrazbayev commented on Feb 5, 2024

   

   SultanOrazbayev

   on Feb 5, 2024

   Member

   More actions

   How far can we go without breaking with the SuiteSparse and C API? My understanding is they support only 2d...
3. eriknw commented on Feb 5, 2024

   

   eriknw

   on Feb 5, 2024

   MemberAuthor

   More actions

   Heh, we'll find out. I should have mentioned dimsum that @jim22k experimented with a few years ago to provide a N-dimensional wrapper on top of GraphBLAS: https://github.com/jim22k/dimsum

   SuiteSparse:GraphBLAS provides some useful functions that get us partway there, and we expose these as:

   • gb.ss.concat
   • matrix.ss.split and vector.ss.split
   • matrix.ss.reshape and vector.ss.reshape
   • matrix.ss.flatten

   We can also export the data, transform the row and column indices, and then import the data. This may be the way we do this generally. I experimented a bit with this in https://github.com/eriknw/strided (where we used strides like how numpy does internally to encode dimension information). This code tests mimicking np.moveaxis, but it's a little dense to comprehend:

   • https://github.com/eriknw/strided/blob/main/strided/tests/test_nd.py
   • https://github.com/eriknw/strided/blob/main/strided/core.py

   Reacted by Sultan Orazbayev
4. eriknw commented on Feb 5, 2024

   

   eriknw

   on Feb 5, 2024

   MemberAuthor

   More actions

   Some places this has come up:

   • GraphBLAS backend Transurgeon/cvxpy#42
   • https://github.com/Nixtla/neuralforecast
   • GraphBLAS backend implementation of a model in https://github.com/huggingface/transformers
5. SultanOrazbayev commented on Feb 5, 2024

   

   SultanOrazbayev

   on Feb 5, 2024

   Member

   More actions

   Oooh, I am definitely interested in the last option. :)

   However, it does feel like something that should be a wrapper/companion package, rather than python-graphblas feature. (though again, it's not clear if that can be done without making internal changes to the coding of the graphblas logic)
6. eriknw commented on Feb 6, 2024

   

   eriknw

   on Feb 6, 2024

   MemberAuthor

   More actions

   However, it does feel like something that should be a wrapper/companion package, rather than python-graphblas feature.

   That's a good point. It probably makes more sense to create a new package to experiment with N-dimensional GraphBLAS (even if super-limited in scope). Once we learn a bit, we could push anything that makes sense to back into python-graphblas.

   Reacted by Sultan Orazbayev and William Zijie Zhang
7. Transurgeon commented on Nov 26, 2024

   

   Transurgeon

   on Nov 26, 2024

   Contributor

   More actions

   Hey @eriknw thank you for raising this issue!
   I just wanted to reference some recent work that is ongoing in the scipy repo: scipy/scipy#21923 related to this 3d (nd) sparse support, it could help you guys for future reference.

   On a side note, cvxpy is planning to add graphblas as an optional dependency for the next release! I need to do some cleaning up of the backend but the maintainer team has agreed that it would be good to have moving forward!

   Reacted by Erik Welch
8. eriknw commented on Dec 9, 2024

   

   eriknw

   on Dec 9, 2024

   MemberAuthor

   More actions

   Thanks for sharing @Transurgeon, and that's great to hear about cvxpy! Please don't be shy about asking for anything you might need. We're going to try to update python-graphblas by the end of the year to support the latest python, numpy, etc. 🤞. We've all been busy this past year, but we're still around ;)

   Reacted by William Zijie Zhang and Deepak Cherian

Issue actions