Description

sneakers-the-rat

opened on Apr 9, 2023

Issue body actions

Trying to understand the implementation of the API wrapper, and I'm wondering how the version of the graphBLAS API is handled? It looks like the API moves slowly, but it does move ( eg https://graphblas.org/ says v1.3.0 on September 25, 2019 and v2.0.0 on November 15, 2021 ).

I ask because it looks like the mappings are relatively manual - a lot of hardcoded names, etc. - rather than something where you're programmatically generating bindings directly from the API. Having just started taking a look at the implementation I am assuming there's a good reason for that, so completely fine, but is there some place where you specify that "this version of python-graphblas is compatible with x version of graphblas API?"

I see that a version of suitesparse-graphblas is pinned, so presumably that also pins some specific version of the API?

In any case, seems like something that would be worth having in the docs if not implemented in the code - there's a statement on python version here: https://python-graphblas.readthedocs.io/en/stable/getting_started/faq.html#what-is-the-version-support-policy but I don't find one for graphBLAS API version

edit: part of pyOpenSci/software-submission#81

Activity

1. eriknw commented on Apr 10, 2023

   

   eriknw

   on Apr 10, 2023

   Member

   More actions

   is there some place where you specify that "this version of python-graphblas is compatible with x version of graphblas API?"

   Not yet, but that is a good point: we should.

   We don't have a strict version policy for how many or how long we'll support any given GraphBLAS C API version, because it's hard to know how future versions will change and how compatible they will be. When new versions come out, we would like to support them. When this happens, I suppose we ought to have a discussion with implementers and users to assess their needs. So, I guess our policy is "we'll talk to stakeholders when this comes up". Version 2.1 is expected to be released this year, and it should be largely compatible, so we can probably support both 2.0 and 2.1.

   I see that a version of suitesparse-graphblas is pinned, so presumably that also pins some specific version of the API?

   Yeah, right now SuiteSparse:GraphBLAS is the main implementation, and it implements version 2.0.0 (and it's own extensions). We plan to support other implementations, and we expect each implementation to have differences and their own extensions.

   SuiteSparse:GraphBLAS has an extension API that lets you set configuration gb.ss.config and see information gb.ss.about. For example, to get the C API version: gb.ss.about["api_version"] (and I know these aren't super-well documented). This type of query is expected to be part of the GraphBLAS C API 2.1 spec.
2. sneakers-the-rat commented on Apr 10, 2023

   

   sneakers-the-rat

   on Apr 10, 2023

   Author

   More actions

   gotcha. I suppose this relates to my questions i'm having rn as i try and read the implementation of the wrapping, wondering how y'all will support multiple backends, as it seems like calls to suitesparse are mostly concentrated in the top level __init__ module, but also I see some special casing done in some utils functions and other parts of the package. I think probably that will eventually need to be split out into a specific backend handling system, which should also help sort out some of the more confusing parts in how the backend is initialized and the calls are constructed, but that seems like a "dev roadmap" kind of question to me
3. eriknw commented on Apr 10, 2023

   

   eriknw

   on Apr 10, 2023

   Member

   More actions

   wondering how y'all will support multiple backends [...] seems like a "dev roadmap" kind of question to me

   Yup, supporting multiple backends is definitely a roadmap item that will require some work. If another implementation is in C that is wrapped by cffi, then the current approach of having the lib and ffi from the cffi wrapping should "just work" for most things. If, however, another implementation uses something such as MLIR or numba, then we'll need to create a better abstraction layer. I expect/hope this will mostly be grunt work and not particularly complicated. We've already decoupled python-graphblas from relying too heavily on cffi compared to earlier.

   As the team has discussed, support for multiple backends will likely come in phases. In one phase, we can support multiple backends, but only one backend at a time (kind of like now with "suitesparse" and "suitesparse-vanilla" backends). In another phase, we'd like to allow use of multiple backends simultaneously in the same Python process. We have ideas of what this might look like, but it's still largely TBD. (cygraphblas was a PoC from years ago trying one (limited) way to do this, but I think using Cython would make the code even more difficult to develop and understand).

   The "suitesparse-vanilla" backend is not quite fully vanilla yet, but it's a long way there and has helped us from being too dependent on SuiteSparse:GraphBLAS implementation. I think a refactor of the operators will be in order to better support "suitesparse-vanilla" and other backends. Hopefully the refactor will make operators more comprehendible (such as expressible in JSON) and more easy to document.

   We're inching our way there with other changes such as making numba (and even suitesparse-graphblas) optional.

   I ask because it looks like the mappings are relatively manual - a lot of hardcoded names, etc.

   I'm curious what alternative you have in mind.
4. eriknw

   added

   documentationImprove or add to documentationImprove or add to documentation

   discussionDiscussing a topic with no specific actions yetDiscussing a topic with no specific actions yet

   on Apr 12, 2023
5. eriknw

   closed this as completedin #441on Apr 20, 2023

Issue actions