GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Type UDT ops like built-in ops; fix UDT literals, casts and comparisons by eriknw · Pull Request #591 · python-graphblas/python-graphblas

Conversation

@eriknw

@eriknw eriknw commented Aug 4, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Member

This PR settles how python-graphblas types an operation: a field or element of
a user-defined type (UDT) is typed as a built-in Vector of that dtype is, and a
Python number beside any operand is typed as numpy 2 types it (NEP 50). It began
as a check that two record UDTs nest the same way (records now pair by leaf paths
and field shapes; see "Compare record leaf paths and field shapes, not leaf
counts"), grew into the whole rule set, and carries #592-#595, whose content it
needs. UDT and UDF code moves into its own modules ("Move UDT and UDF logic out
of the general modules"): core/udt.py (type rules and store casts),
core/literals.py (literal typing), core/operator/udf.py (UDF wrappers) and
core/operator/udt_codegen.py (was udt_utils.py).

The message of each commit that changes behavior gives the before and after.

Changed for every user: Python numbers beside built-in dtypes (NEP 50)

A Python bool, int, float or complex used as an operand (v + 1,
A * 0.5, binary.plus(v, 1), v.select(">", 3), ewise_union defaults, UDF
arguments) is now weak, as in numpy 2: it takes the other operand's dtype when
that dtype's kind can hold it, else INT64, FP64 or FC64. This is the same on
numpy 1 and numpy 2. Before, python-graphblas typed it through numpy with a
value-0 array standing in for it, so the result depended on the installed numpy:
with numpy 2, every int made the result INT64 and every float FP64; with numpy 1,
a large literal could wrap silently. There is no deprecation period. From "Type
Python numbers beside built-in dtypes as numpy 2 does (NEP 50)", "Compare select
value thunks exactly; isequal compares arrays as np.array_equal" and "Compare a
Python int beyond a 64-bit element's range exactly". Checked on main and on this
branch with numpy 2.4:

Expression main (numpy 2) this PR
int8_vec + 1 INT64 INT8
int8_vec * 2, with a 100 INT64, 200 INT8, wraps to -56 (as numpy)
int8_vec ** 2, with a 100 INT64, 10000 INT8, 127 (GraphBLAS saturates pow; numpy wraps to 16)
int8_vec + 300, int8_vec % 300, apply(binary.minus, left=1000), ewise_union(..., 1000, 0), a UDF's apply(udf, right=1000) computed in INT64 OverflowError (as numpy)
uint8_vec - 1, with a 0 INT64, -1 UINT8, 255
uint64_vec + 1, at 2**63 + 1 FP64, rounded UINT64, exact
uint64_vec // 3 FP64 UINT64
fp32_vec * 0.5 FP64 FP32
fp32_vec * 1j FC64 FC32
fp32_vec + 1e300 FP64, 1e300 FP32, inf with a RuntimeWarning (as numpy; fp32_vec == 1e300 warns too)
fp32_vec == 0.1, at 0.1 False True (compared in FP32, as numpy)
Scalar(int8) + 1 INT64 INT8
int8_vec.apply(binary.atan2, right=1) FP64 FP32
int8_vec * 0.5, int8_vec / 300, int8_vec < 300 FP64, FP64, BOOL unchanged
int64_vec < 2**63, Scalar(int64) == 2**63 OverflowError all True, False (exact, as numpy 2)
uint64_vec[0] = 2**64 - 1, Scalar.from_value(2**63, "UINT64") OverflowError stored

To keep the old widening, make the operand strong: int8_vec + np.int64(1), a
Scalar (Scalar.from_value(1)), a typed operator
(int8_vec.apply(binary.plus["INT64"], right=1)), or convert first
(int8_vec.dup("INT64") + 1). numpy scalars, Scalars and typed operators were
strong before and still are. A comparison keeps no result type, so an int out of
an integer dtype's range compares exactly rather than raise, as does an op whose
result is another type anyway (int8_vec / 300 is FP64). to_dense(fill_value=...)
widens to hold a fill value that does not fit (fp32_vec.to_dense(fill_value=1e300)
is float64). dtypes.unify's is_left_scalar and is_right_scalar no longer change
anything (it used them to stand a value-0 array in for a Python number); this is
visible only on numpy 1.

Two more changes for built-in dtypes:

  • A built-in Scalar's == computes eq in the type binary.eq is typed for,
    instead of comparing Python values: Scalar(INT64 2**53 + 1) == 2.0**53 was
    False and is True, as in FP64 (isequal(check_dtype=False) still says False).
    A bad operand's error names Scalar.ewise_mult(op=binary.eq).
  • select with a value op and a thunk of another type compares in the values'
    own type, working around a SuiteSparse bug (Fix GrB_select with a VALUE* op whose type differs from A's DrTimothyAldenDavis/GraphBLAS#463,
    in 8.0.2 through 10.5.1) that made main keep nothing for
    int8_vec.select("==", 1.0); it keeps the 1. "Fix select on complex values
    with a thunk of another type" covers the complex case.

The UDT rules, in five sentences

  1. Each field or element is typed as the built-in op types that dtype, so no
    lifted op narrows (integer truediv is float64; int8 times float32 is float32).
  2. A Python number, or a tuple, list or dict of them laid out like an element,
    is weak field by field, as beside a built-in dtype; numpy scalars, numpy
    arrays and Scalars keep their own dtype.
  3. Two UDTs combine when numpy would combine them: records with the same field
    names, order, nesting and field shapes, and arrays whose shapes broadcast. The
    result is an operand's type when it holds the result, else the promoted
    layout.
  4. Storing a result into an object of another UDT type casts it as GraphBLAS
    casts built-in dtypes, because the object's type asks for it.
  5. Nothing makes a hidden copy of the data: a store cast is fused into the op or
    done in one pass, and what would need a copy raises DomainMismatch, naming
    the explicit spelling.

docs/user_guide/udt.rst has the rules as two tables ("How results are typed" and
"Storing results"); types.rst, udf.rst and operators.rst have the rest.

What changes for UDT users

From "Type UDT ops like built-in ops, cast UDT stores, broadcast array UDTs",
"Cast UDT stores without temporaries, type UDT literals, fold in #592-#595" and
the commits after them. Checked on main and on this branch:

  • Lifted ops type each leaf as built-in ops do. i8x3 * 2.5 was an INT8[3]
    result multiplied by 2, and is FP64[3]; i8x3 / 2 is FP64[3];
    i8x3 + (0.5, 1.5, 2.5) is FP64[3] (main added 0, 1, 2); rec8 * 0.5 has
    float64 fields (main gave (0, 0)); a bool field + 1 is INT64; i8x3 + np.int64(1) is INT64[3]; i8x3.apply(monoid.plus, right=0.5) is FP64[3]. A
    record with an int32 field .ewise_mult(int64_vec, binary.plus) has an int64
    field.
  • Mixed UDT pairs compute: INT8[3] with FP64[3] or FP64[1], records with the
    same field names and other field dtypes, INT8[3] with INT64[2, 3]. Main raised
    KeyError or DomainMismatch. r8.inner(r16) is R16 (main: R8).
  • ewise_add on Vectors and Matrices of mixed UDTs, or with truediv on an
    integer UDT, raises a DomainMismatch that says why: an entry present in one
    input alone would be copied into the result type, which GraphBLAS cannot do
    for a UDT. Main ran the truediv case and truncated the quotients. Infix +
    takes the union instead, with each operand's zero (-0.0 in float fields), and
    ewise_union and ewise_mult work.
  • Store casts: w << v, v.dup(dtype=udt2) and fused stores such as
    w << v.apply(...) work between UDTs whose layouts correspond (main:
    DomainMismatch). f64x3_vec[0] = Scalar(i8x3) and
    i8x3_vec[0].new(dtype=F64x3) read the int8 bytes as float64 on main (values
    like 9.7e-319, past the end of the element) and now cast. `i8x3_vec << i8x3_vec
    • 2.5computes in FP64 and stores with GraphBLAS's cast (truncate, saturate), andi8x3_vec += 0.5` truncates (main converted 0.5 to 0 and left it unchanged).
  • A literal that must become an element of a UDT (beside a user's op, an
    IndexUnaryOp thunk, an ewise_union default) must fit it by type. Main converted
    it and lost what did not fit: apply(udf, right=(0.5, 2, 3)) added 0 and now
    raises ValueError "does not fit". New DeprecationWarning: one that fits only
    by value, such as (1.0, 2, 3) for an int8 array or 0 for a bool field,
    still converts, with a DeprecationWarning; it will raise ValueError in a
    future version (MAINT note in literals._refuse_unless_exact). Test suites
    that run with -W error will see it.
  • Array UDT equality: isequal compares as np.array_equal and ==
    broadcasts. Scalar([1, 1, 1]).isequal(1) was True and is False;
    Scalar([0, 0, 0]) == 0.5 was True and is False; fp64x1.isequal(fp64x3) was
    a KeyError and is False; i8x3 == 300 was an OverflowError and is False.
  • Exception types: an op that does not lift, on a UDT
    (rec.ewise_mult(int_vec, binary.lt)), raises KeyError (main: numpy's
    DTypePromotionError); int_vec.select(select.valueeq, Scalar(rec)) likewise.
    monoid.plus[R8, R16] raises TypeError, where main returned monoid.plus[R16].
  • Two layouts registered under one name ("Give each UDT layout its own JIT C
    name; Scalar == without Numba; CI fixes"): the second gets JIT C name
    _gbudt_NNN, so their JIT kernels are not shared (they read and wrote the
    wrong element size). dtypes.ss.register_new refuses a name already used as a
    C type name. Scalar == on a UDT works without Numba.
  • Array UDTs broadcast, and the generated code loops over an array's elements
    instead of writing out a term for each, whose Numba typing time grew faster
    than linearly with the number of elements.
  • From Add a test fixture that pins the UDT operator execution path #592-Define UDT division on both execution paths #595: min, max, floordiv, division and complex multiplication
    give the same bits on the Numba and C JIT paths, matching numpy ("Spell out
    complex * and / for UDTs; Numba 0.68 follows Python 3.14" too). udt.rst lists
    the few edges where a built-in vector differs from numpy.

Code that may break: result dtypes of UDT ops with float literals or mixed types
change ((i8x3 * 2.5).new() is an anonymous FP64[3], so a UDF typed for the
INT8[3] UDT no longer takes it); ewise_add with truediv on an integer UDT
raises; UDF literals that do not fit raise or warn; monoid.plus[R8, R16]
raises; a Scalar of bytes == 1 raises KeyError. Pickles made on main load and
compute. Private names moved: graphblas.core.operator.udt_utils is now
udt_codegen, and scalar._dict_to_record, _udt_bytes and _padding_ranges
are in core/udt.py.

Not done, deliberately: broadcasting inside record fields; ewise_add on mixed
UDT Vectors without copies; store casts that would need a temporary; stores
between a UDT and a built-in dtype; C JIT kernels for mixed-type ops and casts;
a casting-policy setting.

Fixed after the #591 review round (2026-11-03)

  • "Leave a dtypes.ss.register_new name free when the registration fails": a
    failed register_new (an np_type Numba cannot type) kept the name, so a
    correct retry raised "name for dtype is unavailable". Main allowed the retry.
  • "Type a Python date or duration in a record literal as numpy does": a
    datetime with a time of day, a timedelta finer than the field, or a date
    as text was cut to the field's unit before ==, so noon equaled its day in a
    field of days (main too). It is now compared as numpy compares it.
  • "Compare a UDT of dates or durations with one in another unit as numpy does":
    the same for a UDT that is a bare datetime64 or timedelta64:
    day == np.datetime64("2020-01-01T12:00") was True and is False, and
    seconds == np.timedelta64(5000, "ms") was a KeyError and is True.
  • "Type a literal beside a typed op in the op's own input type": a literal
    beside an op typed op[A, INT64] became an element of A, whose bytes the op
    read as an int64 (silently wrong, main too); v.apply(iu[A, INT64], 2) now
    gives what Scalar.from_value(2) gives.
  • "Accumulate into a UDT Scalar as into a built-in Scalar": s += 0.5 on an
    INT8[3] Scalar of -1s gave 0s (the sum, truncated), where an INT8 Scalar gives
    -1. It now gives -1s, as main did.
  • "Pair an array literal with an array UDT when their shapes broadcast":
    i8x3 + np.array([[1, 2, 3], [4, 5, 6]]) raised numpy's "could not
    broadcast" and is INT64[2, 3], as with the same values in a UDT.
  • "Give IndexBinaryOp is_positional, so dtype in op and op[dtype] work" and
    "Refuse a bound IndexBinaryOp in reduce with the usual TypeError": R in ibo,
    ibo[R] and v.reduce(ibo(1)) raised AttributeError.

Added in the 2026-10-20 coverage round

CI: the workflow passes secrets.CODECOV_TOKEN to the codecov action. Codecov
refuses tokenless uploads from this repository's own branches with "Token required
because branch is protected", so this PR had no coverage report. (The input moved
here from #636, which now only pins the action's version in that step.)
.pre-commit-config.yaml listed the check-executables-have-shebangs hook twice; the
second is removed.

The CI cap that keeps an unpinned pandas below 3.1 (conda-forge's pandas
3.1.0 fails three test_formatting.py tests) is now on main, from #658.

Coverage: tests for lines this PR adds that CI never ran (literal typing,
select's thunk rewrite, UDT store and ewise_add casts, UDF return-type errors,
JIT layout checks); # pragma: no cover for Numba-compiled bodies and unreachable
guards; and removal of a few branches no caller can reach. Locally, coverage of
literals.py, udt.py, udt_codegen.py, udf.py and select.py went from 92-96% to
99-100%. A later commit adds tests for more literals a UDT refuses (a record
literal of the wrong length, parts numpy cannot make an array of, text a bytes
field cannot encode) and for the UdfParseError a lifted op gives when Numba
cannot compile it, and pragmas for two arms of the monoid identity that the
binary op's refusal keeps anything from reaching.

Test fix: test_dtype's test of two UDT layouts under one name compiled kernels with
the JIT forced on, so where the JIT cannot compile (no C compiler, or a broken one) it
failed with JitError instead of skipping; CI always has a compiler. It now asks the
one-compile probe the JIT tests use and skips only its kernel part, so its C name checks
still run everywhere.

Behavior fixes found along the way, each in its own commit with before and
after in its message:

  • select with a value op on a complex input and a thunk of another type kept
    the wrong elements (main too). The thunk now compares exactly.
  • ==, != and isequal on a UDT that is neither a record nor an array
    (bytes, dates) raised TypeError (main too; Scalar == was a regression in
    this PR). They now compare values as numpy does. Such a UDT pairs only with
    its own type, so a Scalar == with a number, which main answered, raises
    KeyError; the exceptions, under == and !=, are two text UDTs of one kind
    (bytes or str) at different widths, and two UDTs of dates, or of durations,
    in different units.
  • Matrix.power with a type-changing semiring on a UDT raised
    GrB_DOMAIN_MISMATCH for n >= 3 (a regression in this PR).
  • select with a float64 thunk too large for FP32 warned about overflow (a
    regression in this PR).
  • A literal value that a bytes, str, date or duration field (or a bare UDT of
    one) would not keep (a number, a finer duration, text that is too long)
    raises "does not fit" for a user op instead of converting with a loss (main
    converted silently). Text too long for a bare text UDT no longer compares
    equal after being cut short, as it did in main's Scalar ==.
  • Every leaf of a record literal is typed, numpy numbers, text and dates too,
    so (np.float64(1.5), 1) no longer becomes (1, 1) in an int8 record, as it
    did on main.
  • float16 leaves, which Numba cannot compute on: UDT ops raise a KeyError
    naming float16, and casting stores a DomainMismatch, where they raised
    Numba's bare NotImplementedError (main too; a bare float16 UDT's == raised
    TypeError there). A np.float16 in an int8 record's literal, which main cut
    to an integer, gets the same KeyError. Scalar == compares them in numpy, as
    on main.
  • A record's array field types its literal as an array UDT does, but keeps
    its shape: the literal's part for it must broadcast to the field's shape, or
    ValueError names both shapes. That part was converted into the field as
    numpy assigns it, silently, even beside a user's op: with an int8[3] field
    holding [1, 2, 3], r + (np.array([[1, 300, 3]]), 0) gave [2, 46, 6].
    Stores (r[0] = ...) still convert as numpy assigns.
  • == between leaves of different kinds (text and a number, for example)
    raises eq's KeyError. Visible against main: a record with a bytes field
    == 1 compared b"1" with 1, and nested == (b"x", np.float64(1)) was
    False; both now raise.
  • A UDF may return an operand of a UDT that is neither a record nor an array
    (bytes, str); it failed to compile (main too).
  • On numpy before 2.3, a NaT duration in another unit became a number when
    converted into a duration UDT or field: np.timedelta64("NaT", "ms") into
    seconds was 9223372036854774 s, and NaT minutes 0 s (main too). It now stays
    NaT, as on numpy 2.3 and later, and a literal holding one is typed as there
    rather than refused.
  • With the suitesparse backend, the exports of an iso Vector or Matrix of an
    array UDT such as INT64[4] kept the extra length-1 axis SuiteSparse gives
    its iso value (main too): Matrix.to_dense had shape (nrows, ncols, 1, 4),
    to_csr, to_csc, to_dcsr and to_dcsc values of shape (nvals, 1, 4),
    to_dicts each value as [[0, 1, 2, 3]], and Vector.to_dense raised
    TypeError. They now give (nrows, ncols, 4), (nvals, 4), [0, 1, 2, 3] and
    (size, 4). CI found it through Set the SuiteSparse JIT control from config, and JIT-compile lifted UDT ops on mixed operand types #652's JIT test, whose constant inputs
    SuiteSparse before 10.5 builds as iso.
  • register_anonymous refuses a dtype of size 0 (str, bytes, an empty
    record, a subarray of length 0) with ValueError "dtype must not have a
    size of 0" on every version. SuiteSparse 8 and later refused it with a
    bare InvalidValue, but SuiteSparse 7 takes a size of 0 as 1, so there
    lookup_dtype(str) registered the UDT "<U0" and Scalar + "x" raised a
    ValueError from cffi; it now raises the TypeError it raises on 8 and later.
    CI found it in its one job with python-suitesparse-graphblas 7.4.2.
  • Registering a UDT that holds an array of bool, datetime64, timedelta64, str
    or bytes, alone or as a record field, raised Numba's own error, such as
    AttributeError: 'Boolean' object has no attribute 'bitwidth' (main too).
    With Numba it now raises ValueError naming the array and what to use
    instead (an array of uint8 for bool), chained from Numba's error, and
    register_anonymous leaves no unused GrB_Type behind. Without Numba these
    still register.
  • Ops registered from C source (ss.register_new of unary, binary,
    indexunary, select and indexbinary) could run another op's kernel (main
    too). SuiteSparse finds a compiled kernel in memory by its op's name and the
    types in the kernel's encoding, and each op was named by its last name
    alone. So an op registered again under its name with another definition (as
    when a notebook cell runs again), or one of the same last name in another
    namespace (ss.f and ss.nested.f), ran the kernel compiled first in the
    process, and so did a JIT IndexBinaryOp's signature that differs from
    another only in theta's type, which is not in the encoding. Each op is now
    compiled under a C name made of its last name, its types for an
    IndexBinaryOp, and a digest of those with its full name and definition
    (add_theta_4_FP64_4_FP64_4_FP32_ and 12 hex digits); jit_c_name and
    jit_c_source show it.
  • Binding an empty Scalar as an IndexBinaryOp's theta crashed the interpreter
    (main too). It raises EmptyObject, as apply does for an empty IndexUnaryOp
    thunk.

Element stores still use numpy's assignment cast.

IndexBinaryOp typing. An untyped op(theta) took the op's type from theta
alone and cast the inputs to it. It now returns a BinaryOp typed each time it
runs, as any other BinaryOp is, at the type that holds both inputs and theta.
Theta is typed by its value, as apply types an IndexUnaryOp's thunk: a Python
int is INT64, a float FP64 and a bool BOOL, a numpy scalar or a Scalar keeps
its type, and a numpy array is typed as its array layout. An op with
is_udt=True is compiled for the types of x, y and theta as they are, and
SuiteSparse gets theta's type as the op's theta_type, so theta is never
converted; a JIT op takes the signature for the inputs' and theta's types, as
a JIT IndexUnaryOp takes its thunk. op[X, Y, T] names all three types;
op[X, Y] and op[T] give theta y's type, as before. IndexBinaryOp is on main
but not released, so nothing is deprecated. Before and after (the commit
message has the rest, including pickling and repr):

  • A.ewise_mult(B, op()) on INT64: a BOOL op, so every value became True ->
    INT64, the values op computes.
  • u.ewise_mult(w, op(2)) on FP64: computed in INT64 (1.5 + 2 * 2.25 gave 5)
    -> FP64 (6.0).
  • A Semiring from op(0.5) with INT64, FP32 or INT8 matrices: KeyError ->
    computed in FP64.
  • The SuiteSparse user guide's argmax (an is_udt multiplier that makes the
    pair (k, A(i, k))) on a matrix of pairs: iop(), iop(False), iop(0) and
    iop(0.0) failed to compile -> all four run.
  • A JIT op registered at FP64: op(10.0) on FP32 data ran in FP64 -> KeyError
    naming (FP32, FP32, FP64). op["FP64", "FP32"] is x and y, no longer x and
    theta.

Stack created with GitHub Stacks CLI • Give Feedback 💬

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@eriknw
eriknw marked this pull request as ready for review August 4, 2026 16:07
@eriknw
eriknw force-pushed the 11-udt-record-leaf-count branch from 7d55403 to d4c4481 Compare August 4, 2026 16:12
@eriknw
eriknw force-pushed the 11-udt-record-leaf-count branch from d4c4481 to 0bd9607 Compare August 5, 2026 00:06
@eriknw
eriknw force-pushed the 11-udt-record-leaf-count branch 2 times, most recently from 479c26c to 563b80a Compare August 5, 2026 17:44
@eriknw
eriknw force-pushed the 11-udt-record-leaf-count branch from 563b80a to 123b148 Compare August 5, 2026 18:03
@eriknw
eriknw force-pushed the 11-udt-record-leaf-count branch 2 times, most recently from 683dcc9 to ec062f1 Compare August 6, 2026 07:59
@eriknw
eriknw force-pushed the 11-udt-record-leaf-count branch from ec062f1 to b8e797f Compare August 6, 2026 15:39
@eriknw
eriknw force-pushed the 11-udt-record-leaf-count branch from b8e797f to 17bc219 Compare August 6, 2026 15:41
@eriknw
eriknw force-pushed the 11-udt-record-leaf-count branch from 17bc219 to da41e30 Compare August 6, 2026 20:36
@eriknw
eriknw force-pushed the 11-udt-record-leaf-count branch from da41e30 to 15b9955 Compare August 6, 2026 20:42
@eriknw
eriknw force-pushed the 11-udt-record-leaf-count branch from 15b9955 to 02464a8 Compare August 7, 2026 02:48
@eriknw
eriknw force-pushed the 11-udt-record-leaf-count branch from 02464a8 to 2f6fb4d Compare August 7, 2026 05:09
@eriknw
eriknw force-pushed the 11-udt-record-leaf-count branch from 2f6fb4d to cdefa0b Compare August 26, 2026 17:30
@eriknw
eriknw force-pushed the 11-udt-record-leaf-count branch from cdefa0b to dc5dbaf Compare September 18, 2026 03:09
@eriknw
eriknw force-pushed the 11-udt-record-leaf-count branch from dc5dbaf to 3b4489d Compare September 18, 2026 03:46
eriknw added 5 commits October 28, 2026 20:21
@eriknw
Convert a NaT duration in its leaf's unit, not as a NaT of no unit
f6d9129
"Keep a NaT duration NaT in another unit on numpy before 2.3" handed
numpy a NaT of no unit, np.timedelta64("NaT"), in place of each NaT
duration. numpy 2.5 deprecates that generic unit ("The 'generic' unit
for NumPy timedelta is deprecated"), so on numpy 2.5 every store or
literal of a NaT duration into a duration UDT or field warned with a
DeprecationWarning, which the test suite's filterwarnings makes an
error: test_udt_nat_duration_in_another_unit_stays_nat,
test_udt_sequence_and_array_literals_type_like_numbers and
test_udt_literal_converts_only_when_exact failed.

Each NaT duration is now converted into the unit of the leaf it fills
by converting it as an array (astype), which keeps NaT on every numpy,
before numpy assigns it. The value is walked beside the dtype: a tuple
of a record's length as its fields, a list (or a tuple for any other
dtype) as a sequence of elements, and an array UDT by its element type.
Nothing makes a generic unit.

numpy 2.5.4: DeprecationWarning before, NaT with no warning now. numpy
1.26, 2.2 and 2.4: NaT in the target unit before and after. New rows:
a nested record's duration field, and an array UDT of durations given
a list, where Numba is absent (Numba cannot type an array of durations,
so registering one fails with it).
@eriknw
Skip the JIT part of the one-name UDT layouts test without a C compiler
16f679c
test_udt_layouts_under_one_name_get_their_own_jit_c_names set
jit_c_control to 'on' and compiled kernels for both layouts, so on a host
whose JIT cannot compile (no C compiler, or a broken one) it failed with
JitError instead of skipping. CI always installs a compiler, so CI never
saw it.

It now asks _enable_jit_for_udt, which probes with a real compile once
per process, and skips the kernel part when the answer is no. The C name
checks and the register_new check need no compiler, so they come first
and run everywhere.
@eriknw
Expand an iso array UDT's value without an extra axis in exports
2a564c2
Behavior fix. SuiteSparse exports the iso value of an array UDT with
shape (1, *subarray shape), and normalize_values broadcast it to
shape + values.shape, keeping that axis of length 1. With the
suitesparse backend, for an iso Vector or Matrix of an array UDT such
as INT64[4] (as v[:] = [0, 1, 2, 3] makes):

- Vector.to_dense: TypeError "unsupported operand type(s) for +: 'int'
  and 'tuple'" on main and before this commit (it passed its size as
  an int); now an array of shape (size, 4).
- Matrix.to_dense: shape (nrows, ncols, 1, 4); now (nrows, ncols, 4).
- Matrix.to_csr, to_csc, to_dcsr and to_dcsc: values of shape
  (nvals, 1, 4); now (nvals, 4).
- Matrix.to_dicts, which uses to_dcsr or to_dcsc: each value as
  [[0, 1, 2, 3]]; now [0, 1, 2, 3].

Vector.to_coo, Matrix.to_coo, the vanilla backend and records were
already right. normalize_values now broadcasts to
shape + values.shape[1:], and Vector.to_dense passes (size,).

CI found it through #652's test_udt_mixed_jit_c_name_is_its_own (17
jobs, e.g. job 122633175811): SuiteSparse before 10.5 makes a vector
built from equal values iso, so that test's constant inputs gave an
iso result, and to_dense raised. With 10.5 the vectors are not iso.
@eriknw
Refuse a zero-size UDT, so a str literal raises TypeError on SuiteSpa…
8a109a5
…rse 7

Behavior fix. register_anonymous passed a dtype's itemsize to
GrB_Type_new without a check. SuiteSparse 8 and later refuse a size of
0, but SuiteSparse 7 takes it as 1. lookup_dtype auto-registers what it
does not know, so on SuiteSparse 7 a str became the UDT "<U0", which
holds nothing.

- register_anonymous of str, bytes, np.dtype("U"), an empty record or
  a subarray of length 0: InvalidValue with no message on SuiteSparse
  8 and later, a UDT on 7; now ValueError "dtype must not have a size
  of 0" on every version.
- lookup_dtype(str): ValueError "Unknown dtype" on 8 and later, the
  "<U0" UDT on 7; now ValueError "Unknown dtype" on every version.
- Scalar.from_value("x"), Scalar + "x" and v.apply(binary.plus,
  right="x"): on SuiteSparse 7, ValueError "got more than 0 values to
  unpack" from cffi; now the TypeError they raise on 8 and later.

CI found it in wave 2 through #654's test_bad_ewise case (s1, "x"),
which expects the TypeError, in its one job with
python-suitesparse-graphblas 7.4.2 (122632718294).
@eriknw
Note that SuiteSparse:GraphBLAS 10.5.2 fixed select with a value op
7efd40b
SuiteSparse:GraphBLAS 10.5.2 fixed the bug _value_select_in_own_type
works around (DrTimothyAldenDavis/GraphBLAS#463): select with a VALUE*
op of another type than the input's now casts both to the op's type.
Its MAINT note said to delete the function once every supported version
has the fix, but that would change two results that
test_select_value_thunk_of_another_type pins: with the function
bypassed on 10.5.2, an INT64 thunk of 2**53 + 1 beside FP32 or FC32
values of 2**53 rounds to 2**53 in the op's type and keeps that element.
The other 61 cases of the test match. The note now says so, and gives
the versions with the bug as 8.0.2 through 10.5.1. Comments only.
@eriknw
eriknw force-pushed the 11-udt-record-leaf-count branch from c045eea to 7efd40b Compare October 29, 2026 02:01
eriknw added 7 commits October 28, 2026 23:11
@eriknw
Say which array Numba cannot type when registering a UDT
1034473
Behavior change. Registering a UDT that holds an array of bool,
datetime64, timedelta64, str or bytes (as the UDT, or as a field of a
record) raised Numba's own error, which does not say what to change:

    dtypes.register_anonymous(np.dtype((np.bool_, (2,))))
    # was AttributeError: 'Boolean' object has no attribute 'bitwidth'
    # now ValueError: Numba, which python-graphblas uses for UDTs, cannot
    #     type an array of bool, as in ('?', (2,)); use an array of uint8
    #     instead

The ValueError, the type of the other errors register_anonymous raises
for a dtype it refuses, is chained from Numba's AttributeError. The
np_type of dtypes.ss.register_new gets the same error. Without Numba
these dtypes still register, as before.

Numba now types the dtype before GrB_Type_new, so a dtype it cannot type
no longer leaves an unused GrB_Type behind.

lookup_dtype, which registers a numpy dtype it has not seen, still says
"Unknown dtype" for these, as for any dtype it cannot register.
@eriknw
Type an IndexBinaryOp bound to theta when it runs, with theta of its …
0a6c4b5
…own type

Binding theta to an IndexBinaryOp of no type, op(theta), took the op's type
from theta alone, and the inputs were cast to it. Now op(theta) returns a
BinaryOp whose type is chosen each time it runs, as any other BinaryOp's is.
Theta is typed by its value when it is given, as apply types an IndexUnaryOp's
thunk: a Python int is INT64, a float FP64 and a bool BOOL, and a numpy scalar
or a Scalar keeps its type. A numpy array is typed as its array layout, as
other array literals are. The default theta, False, changes no type.

An op registered without is_udt runs at the type that holds both inputs and
theta. One with is_udt=True is compiled for the types of x, y and theta as
they are, and SuiteSparse gets theta's type as the op's theta_type, which
GxB_IndexBinaryOp_new takes apart from x's and y's. So theta is never
converted. The compiled ops are cached per (x, y, theta) types, and
op[X, Y, T] names all three; op[X, Y] and op[T] give theta y's type, as
before. A Semiring built from op(theta) is polymorphic in the same way.
op[T](theta) and op(theta, dtype=T) still bind at T. IndexBinaryOp has not
been released, so nothing is deprecated.

Behavior changes (before -> after):

- A.ewise_mult(B, op()) on INT64: a BOOL op, so every value became True ->
  INT64, the values op computes.
- u.ewise_mult(w, op(2)) on FP64: computed in INT64 (1.5 + 2 * 2.25 gave 5)
  -> FP64 (6.0). op(np.int8(2)): computed in INT8, where 300.0 became 127
  and wrapped -> FP64.
- A Semiring from op(0.5) with INT64, FP32 or INT8 matrices: KeyError ->
  computed in FP64, with or without is_udt.
- The SuiteSparse user guide's argmax (an is_udt multiplier that makes the
  pair (k, A(i, k)), theta unused) on a matrix of pairs: iop(), iop(False),
  iop(0) and iop(0.0) were ops on BOOL, INT64 or FP64 values, which failed
  to compile for the pairs -> all four run, compiled for (pair, BOOL, BOOL),
  (pair, BOOL, INT64) and (pair, BOOL, FP64).
- An is_udt op on INT64 data: op(0.5) or an FP32 Scalar theta made the op
  FP64 or FP32 and cast the inputs to it -> the inputs stay INT64, theta keeps
  its type, and Numba types the result (FP64 for x + y + theta).
- An is_udt op on int32[3] data: op(np.array([10, 20, 30], np.int32)) raised
  "Cannot infer a dtype for theta" -> theta is an int32[3] value; an int64
  array is an int64[3] one. op(5) gave GrB_DOMAIN_MISMATCH -> theta is an
  INT64. op((10, 20, 30)) still raises TypeError, now saying to give theta
  as a Scalar of its UDT; an op without is_udt raises KeyError for an array
  theta, as for a UDT Scalar.
- A JIT op (indexbinary.ss.register_new) is registered for its x, y and theta
  types. op(theta) took the one registered at theta's type T for x, y and
  theta, and cast the inputs to T -> it takes the one for the inputs' types
  and theta's, without casting, as a JIT IndexUnaryOp takes its thunk: with
  an op registered at FP64, op(10.0) on FP64 data runs as before, op(10.0)
  on FP32 data ran in FP64 -> KeyError naming (FP32, FP32, FP64), and op(10)
  raised KeyError when bound -> when it runs. op.types is keyed
  (x, y, theta), so registering the same x and theta types for another y no
  longer raises "already defined", and op["FP64", "FP32"] is x and y, no
  longer x and theta: op["FP64", "FP64", "FP32"] is the op with an FP32
  theta.
- A bound op pickles with all its types: op["INT64", "FP64"](1) came back as
  op["INT64"](1), whose y is INT64.
- op(theta).type: the bound type -> AttributeError, as for any untyped op;
  op(theta)[T] is op[T](theta). repr(op(2)): binary.f[INT64] ->
  indexbinary.f(2). op(theta)(A & B) works, as for other BinaryOps.
- A Scalar theta is copied when given, so a later change to it does not
  reach the op, as when the op bound it at once.

find_opclass now reads an opclass attribute on an OpBase subclass, so
op(theta) is still a BinaryOp there. The user guide's examples drop the
op[float](0.5) that hid the old typing. The comments on IndexUnaryOp's and
SelectOp's UDT caches now say they are keyed by (dtype, thunk_type).
@eriknw
Refuse a record literal whose array field part does not broadcast to …
102f3a0
…the field

Behavior fix. Where a literal for a record UDT is typed (a lifted op such as
+, ==, a user's op, an IndexUnaryOp thunk, an ewise_union default, and ==
on a Scalar), a part for an array field must now broadcast to the field's
shape, or ValueError names both shapes. An array UDT's element takes the
shape it broadcasts to with a literal, as numpy does, so INT8[3] plus
np.array([[1, 300, 3]]) is INT64[1, 3]; that is unchanged. A record's field
keeps its shape, so for the record the typing gave up, and the literal was
converted into the field as numpy assigns it: the leading axis was dropped
and 300 wrapped to 44, silently, even beside a user's op, which refuses a
literal that does not fit. Stores (r[0] = ...) still convert as numpy
assigns, as they do for an array UDT.

With r a Vector of a record with an int8[3] field a and an int64 field b,
holding ([1, 2, 3], 0) (before -> after):

- r + (np.array([[1, 300, 3]]), 0): ([2, 46, 6], 0) -> ValueError
  "array([[  1, 300,   3]]) has shape (1, 3), which does not fit field
  'a' of R, of shape (3,): ...".
- r.apply(user_op, right=(np.array([[1, 300, 3]]), 0)), with user_op
  returning y: ([1, 44, 3], 0) -> the same ValueError.
- r == (np.array([[1, 300, 3]]), 0): False -> the same ValueError; with
  np.array([[1, 2, 3]], np.int8) it was True.
- r + ([[1, 300, 3]], 0), and the other paths with it: numpy's "setting an
  array element with a sequence" ValueError -> this ValueError.
- r + ([1, 2], 0): numpy's "could not broadcast input array from shape (2,)
  into shape (3,)" -> this ValueError.
- r[0] = (np.array([[1, 300, 3]]), 0): ([1, 44, 3], 0), unchanged.
@eriknw
Cover more literals a UDT refuses and the Numba error of a lifted op
530932a
Literal typing had arms no test reached: a comparison with a record literal
of the wrong length (alone and nested beside an int beyond its leaf), a part
holding numpy values that numpy cannot make an array of or that is not
numbers, and text a bytes field cannot encode. An array of records now pins
the UdfParseError a lifted op gives when Numba cannot compile it, and the
NoJITWarning is checked to stay silent for a pair it warned about.

Two arms of the monoid identity are excluded from coverage: the binary op
compiles first, and it refuses what would reach them.
@eriknw
Give each type signature of a JIT IndexBinaryOp its own C name
83643e1
indexbinary.ss.register_new compiled every type signature of one op under
the op's own name. SuiteSparse finds a compiled kernel in memory by its op's
name and the types in the kernel's encoding, and an ewise or mxm kernel's
encoding has x's, y's and z's types but not theta's. So two signatures that
differ only in theta's type ran whichever kernel was compiled first. Now each
signature's C function is compiled as name_N_x_M_y_theta, with N and M the
lengths of the x and y parts (built-in types by name, UDTs by their C name):
the C definition is the user's with the function renamed. A name SuiteSparse
would cut at 127 characters is the op name's first 100 and a digest of the
whole. The op's jit_c_name and jit_c_source show the C name and the source
SuiteSparse compiles; jit_c_definition is still the definition given.
JIT IndexUnaryOps and SelectOps are not affected: the thunk's type is in
their kernels' encodings.

Behavior change (before -> after): with add_theta registered at FP64 for x,
y and theta, and then at an FP32 theta with
(*z) = (*x) + (*y) + 2 * (double)(*theta), the FP32 signature on FP64 data
with theta 10 gave [2, 2, 4, 0] (the FP64 kernel, reading the float as a
double) -> [22, 22, 24, 20]. op.jit_c_name: None ->
add_theta_4_FP64_4_FP64_FP32.
@eriknw
Name each op registered from C by its full name and definition
cd6e0a8
The ss.register_new functions of unary, binary, indexunary (and select) gave
SuiteSparse the op's last name as its C name, and indexbinary's the last name
and types. SuiteSparse finds a compiled kernel in memory by its op's name,
and checks an op's definition only when it loads a kernel from disk. So, in
one process, an op registered again under its name with another definition
(after delattr, as when a notebook cell is run again), or an op of the same
last name in another namespace (ss.f and ss.nested.f), ran the kernel
compiled first.

Now every op registered from C source is compiled under a C name made of its
last name, the types for an IndexBinaryOp (whose theta type is not in its
kernels' encoding), and a 12-hex digest of these with the full dotted name
and the definition: f_3a1c90e2d4b7, or add_theta_4_FP64_4_FP64_4_FP32_...
for an IndexBinaryOp. A name SuiteSparse would cut at 127 characters is the
last name's first 100 characters and a 16-hex digest. The function is renamed
in the definition wherever it is named, except as a struct member (after "."
or "->"). The same name and definition give the same C name, so a kernel is
compiled once for them. jit_c_name and jit_c_source, None for these ops
before (except IndexBinaryOp, from the previous commit), show the C name and
the renamed source; jit_c_definition is still the definition given. The
helper is ss._utils._jit_c_source.

Behavior changes (before -> after), in one process:

- binary.ss.register_new("f", x + y) and ("nested.f", x - y), both FP64:
  nested.f computed x + y -> x - y.
- unary.ss.register_new("u", x * x), delattr(unary.ss, "u"), then
  register_new("u", x + 10): u computed x * x -> x + 10.
@eriknw
Refuse an empty Scalar theta for an IndexBinaryOp
ea63892
SuiteSparse binds theta without checking that it holds a value: it reads
the Scalar's value array. Binding an empty Scalar crashed the interpreter,
and op(s) with no type, which binds a copy of s, ran with theta 0. Both now
raise EmptyObject, as apply does for an empty IndexUnaryOp thunk, in every
bind: typed or not, is_udt or not, a JIT op's too.

Behavior change (before -> after), with s = Scalar(int):

- op[int](s) and op(s, dtype=int): segmentation fault -> EmptyObject.
- op(s): an op that ran with theta 0 -> EmptyObject, at the bind. With s a
  C scalar (is_cscalar=True): segmentation fault -> EmptyObject.
@eriknw eriknw changed the title Reject record UDT pairs whose leaf counts differ Type UDT ops like built-in ops; fix UDT literals, casts and comparisons Nov 3, 2026
eriknw added 8 commits November 3, 2026 07:32
@eriknw
Leave a dtypes.ss.register_new name free when the registration fails
8cb81c3
register_new recorded the name as a JIT C type name as soon as
SuiteSparse made the type, before the numpy and Numba types were made.
When one of those raised (an np_type with an array of bool, which Numba
cannot type), the name stayed reserved, and a correct second call
raised "name for dtype is unavailable". Main allowed the second call; the
check against C type names came with "Give each UDT layout its own JIT C
name". The name is now recorded after the type is registered. The type
made by the failed call is never used, so no JIT kernel knows its name.

Behavior change (before -> after), with d the typedef of pt_t:

- register_new("pt_t", d, np_type=np.dtype((np.bool_, (2,)))) raises
  ValueError (unchanged), then register_new("pt_t", d,
  np_type=np.dtype([("x", "f8"), ("y", "f8")])): ValueError "'pt_t' name
  for dtype is unavailable" -> registers dtypes.ss.pt_t.
@eriknw
Type a Python date or duration in a record literal as numpy does
b331963
A record literal's date or duration field takes the field's type only
when its value converts into the field and back unchanged; otherwise it
keeps its own type, so == compares it whole. A numpy date did this
already, but a Python datetime.date, datetime.datetime or
datetime.timedelta, or a date given as text, always took the field's type.
So it was cut to the field's unit before == compared it: noon compared
equal to its day in a field of days, and 5 s and 1 us equal to 5 s in a
field of seconds. Main gave the same wrong answers (it gave them for numpy
dates too). Now such a value is first made the numpy value it stands for
(a datetime in microseconds, a timedelta in microseconds, a date as text
in the unit it is written to), and is typed as that numpy value is.

Behavior change (before -> after), with v a Vector of a record of an
M8[D] and an m8[s] field holding (2020-01-01, 5 s), and s = v[0].new():

- v == (datetime(2020, 1, 1, 12), np.timedelta64(5, "s")), and with
  "2020-01-01T12:00" for the date: True -> False; the same for s ==.
- v == (np.datetime64("2020-01-01"), timedelta(seconds=5, microseconds=1)):
  True -> False.
- v.apply(user_op, right=(datetime(2020, 1, 1, 12), np.timedelta64(5,
  "s"))): stored the day, silently -> ValueError "does not fit", as for
  np.datetime64("2020-01-01T12:00").
- v + (datetime(2020, 1, 1, 12), ...): UdfParseError from Numba ->
  KeyError naming datetime64[D] and datetime64[us].
- A datetime at midnight, a date, and text such as "2020-01-01" still
  compare equal and fit the field.
@eriknw
Compare a UDT of dates or durations with one in another unit as numpy…
cf64e89
… does

A UDT that is a bare datetime64 or timedelta64 (not a record or an array)
converted a date or duration literal into its own unit before == compared
it, so noon compared equal to its day in a UDT of days. Main raised
TypeError here for a Vector, and gave True for a Scalar; "Compare UDTs that
are neither records nor arrays in eq and ne" made the Vector give True too.
A numpy duration in another unit was a KeyError instead, even when it was
the same duration.

Now such a literal is typed as in a record's date field: the UDT itself
when it converts into the UDT and back unchanged, else numpy's type for it
(a datetime is in microseconds, a timedelta too, a date as text in the unit
it is written to). Under eq and ne, a UDT of dates or durations pairs with
one in another unit, as text of one kind pairs at another width, and they
compare as numpy compares them.

Behavior change (before -> after), with day a Vector of an M8[D] UDT
holding 2020-01-01 and five one of an m8[s] UDT holding 5 s:

- day == np.datetime64("2020-01-01T12:00"), datetime(2020, 1, 1, 12) or
  "2020-01-01T12:00": True -> False; != the opposite; the same for
  day[0].new() ==.
- five == np.timedelta64(5000, "ms"): KeyError -> True; 5001 ms: KeyError
  -> False. five == timedelta(seconds=5, microseconds=1): True -> False.
- binary.eq(five & a Vector of m8[ms] holding 5000 ms), and five.isequal
  of it: KeyError -> True.
- day.apply(user_op, right=datetime(2020, 1, 1, 12)): stored the day,
  silently -> ValueError "does not fit".
- five.apply(user_op, right=np.timedelta64(5000, "ms")): converted with a
  DeprecationWarning -> converts with none, as in a record's field.
- day == timedelta(5): numpy's ValueError -> binary.eq's KeyError.
@eriknw
Type a literal beside a typed op in the op's own input type
2385b58
A literal operand beside a UDT became an element of that UDT even when
the op was typed and takes that operand in another type: op[A, INT64]
takes its thunk, or its second input, as an INT64. Apply and select pass
a UDT thunk or operand to SuiteSparse as a pointer to its bytes, so the op
read the bytes of the A element as an int64: silently wrong answers.
Main did the same. A Scalar of the op's type gave the right answer.

Now a typed op's own input type on the literal's side decides: its first
input type for a left operand, else its second input or thunk type.
GraphBLAS converts a value to a built-in input type, so the literal is
typed as beside a built-in typed op (its own type: 2.5 for an INT64 thunk
is converted to 2, as for an op typed on built-in types); a UDT input type
takes only a literal that fits it. _literal_operand takes left=True from
apply's left operand and from ewise_union's left default.

Behavior change (before -> after), with v a Vector of a record A of two
float64 fields holding (1.0, 2.0) and (3.0, 4.0), and user ops typed
[A, INT64] (or [INT64, A]) that compare or add field a with the other:

- v.apply(iu[A, INT64], 2): [False, False] -> [False, True], as with
  Scalar.from_value(2); the same for thunks 2.5 and np.int8(2).
- v.select(se[A, INT64], 2), on a Vector or Matrix: empty -> index 1.
- v.apply(f[A, INT64], right=2) and v.apply(g[INT64, A], left=2): about
  4.6e18 for each element -> [3.0, 5.0].
- With B a record of two int64 fields: v.apply(h[A, B], right=(2, 3)):
  about 4.6e18 -> [3.0, 5.0]; v.apply(iu_rec[A, B], (2, 0)): [False,
  False] -> [False, True], and with (2.5, 0): [False, False] -> ValueError
  "does not fit".
- Scalar(A).ewise_mult(2, f[A, INT64]): GraphBLAS's DomainMismatch -> 3.0.
@eriknw
Accumulate into a UDT Scalar as into a built-in Scalar
7afa685
The accumulator is typed on the object's dtype, and for a built-in Scalar
GraphBLAS converts the value to that type before it accumulates: an INT8
Scalar of -1 with s += 0.5 (Scalar.__iadd__ is s(binary.plus) << 0.5)
adds 0 and stays -1, as v(accum=binary.plus) << 0.5 does for a Vector of
either kind. A UDT Scalar used the untyped op instead (_scalar_accum_op),
so it computed -1 + 0.5 in FP64 and truncated the sum to 0. That was meant
to match a Vector's v += 0.5, but that is v << v + 0.5, not an accumulate.
Main gave the built-in answer.

Now a UDT Scalar accumulates with the typed op, and the value is converted
to the Scalar's type first, as storing it would convert it: a literal by
Scalar.from_value, a Scalar or an element of another UDT by the store cast.
A built-in Scalar into a UDT Scalar still raises DomainMismatch, as a store
of it does, into a Vector too.

Behavior change (before -> after), with s an INT8[3] UDT Scalar of
[-1, -1, -1]:

- s += 0.5 and s(accum=binary.plus) << 0.5: [0, 0, 0] -> [-1, -1, -1]
  (an INT8 Scalar gives -1; a Vector of the UDT gives [-1, -1, -1]).
- s(accum=binary.plus) << (0.5, 1.5, 2.5): [0, 0, 1] -> [-1, 0, 1], as for
  a Vector of the UDT.
- s(accum=binary.plus) << v[1], with v a Vector of an FP64[3] UDT holding
  [0.5, 1.5, 2.5]: [0, 0, 1] -> [-1, 0, 1], as with the same value in a
  Scalar of that UDT, which was cast first already.
@eriknw
Pair an array literal with an array UDT when their shapes broadcast
658b522
udt.rst says a numpy array beside a UDT pairs as an array UDT of its own
shape, and array UDTs pair when their shapes broadcast. A numpy array
paired only when it broadcast to the UDT's shape, or had that shape apart
from leading 1s. Any other array, even one whose shape broadcasts with
the UDT's, was converted into the UDT, where numpy raised "could not
broadcast", while the same values as a UDT operand worked. Main raised
too (a KeyError for the UDT pair). Now such an array is typed in its own
shape and pairs as an array UDT operand does; the result has the shape
numpy's would. An array that broadcasts to the UDT's shape is typed in the
UDT's shape, as before, which a literal that must become an element needs.

Behavior change (before -> after), with v a Vector of an INT8[3] UDT
holding [1, 2, 3] and u one of an INT8[1] UDT holding [1]:

- v + np.array([[1, 2, 3], [4, 5, 6]]), and apply(binary.plus) with it on
  either side: ValueError "could not broadcast input array from shape
  (2,3) into shape (3,)" -> INT64[2, 3] [[2, 4, 6], [5, 7, 9]], as with
  the same values in an INT64[2, 3] UDT.
- v == np.array([[1, 2, 3], [1, 2, 3]]), on a Vector or a Scalar: the same
  ValueError -> True.
- u + np.array([1, 2, 3]), and u + (1, 2, 3): ValueError -> INT64[3]
  [2, 3, 4].
- v + [[1, 2, 3], [4, 5, 6]]: numpy's "setting an array element with a
  sequence" ValueError -> INT64[2, 3], as for the array.
- v.ewise_union(v, binary.plus, a 2-by-3 array, 0): numpy's ValueError ->
  ValueError "does not fit".
@eriknw
Give IndexBinaryOp is_positional, so dtype in op and op[dtype] work
339fd93
OpBase.__getitem__ reads is_positional for a type the op does not have,
and IndexBinaryOp had none, so asking an IndexBinaryOp (registered without
is_udt) for a type it does not take raised AttributeError. Main did the
same; IndexBinaryOp has not been released. It is now False, as on a bound
IndexBinaryOp and every other op class.

Behavior change (before -> after), with op an IndexBinaryOp of x + y +
theta and R a record UDT:

- R in op: AttributeError -> False, as for an IndexUnaryOp.
- op[R] and op(Scalar.from_value((1.0, 2), R), dtype=R): AttributeError ->
  KeyError "... does not work with R".
@eriknw
Refuse a bound IndexBinaryOp in reduce with the usual TypeError
82f3312
reduce turns a BinaryOp that is part of a Monoid into that Monoid, and
refuses any other BinaryOp with a TypeError naming it. It reads the op's
monoid for that, and the error message its parent's; neither a bound
IndexBinaryOp nor IndexBinaryOp had the attribute, so reduce raised
AttributeError, for ibo[T](theta) on main and for the new untyped
ibo(theta) too. Both now have monoid = None, as a BinaryOp of no Monoid
has. IndexBinaryOp has not been released.

Behavior change (before -> after), with v an INT64 Vector and ibo an
IndexBinaryOp of x + y + theta:

- v.reduce(ibo(1)) and v.reduce(ibo[int](1)): AttributeError
  "'_BoundIndexBinaryOp' object has no attribute 'monoid'" -> TypeError
  "Bad type for argument `op` in Vector.reduce(...) ... is not known to be
  part of a Monoid", as for binary.minus.
- A.reduce_rowwise(ibo(1)): AttributeError -> NotImplementedException, as
  for binary.minus.
@eriknw
eriknw removed this pull request from stack #657 November 3, 2026 16:44
@eriknw
eriknw added this pull request to stack #660 November 3, 2026 16:45
This was referenced Nov 3, 2026
Follow-ups to #591: Monoids on UDT pairs, out-of-range ints, select.row bounds, wording #661
Merged
Fix UDT, dtype and JIT edge cases found in #591's review #663
Open
Add indexunary helpers, break per-call agg import, document from_string #604
Open
Drop unify's ignored scalar flags inside the library, and share the UDT leaf walkers #665
Open
@eriknw

eriknw commented Nov 9, 2026

Copy link
Copy Markdown
Member Author

This grew larger than expected, but has a lot of things that ought to go in. I expect to do some followups to clean some things up. Merging.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@eriknw eriknw mentioned this pull request Nov 12, 2026
Give UDTs C names that no JIT kernel identifier collides with #669
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

1 participant

@eriknw
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.