GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — Let a configured JIT "on" compile through cmake on Windows by eriknw · Pull Request #659 · python-graphblas/python-graphblas

Conversation

@eriknw

@eriknw eriknw commented Nov 3, 2026

Copy link
Copy Markdown
Member

Stacked on #656. One commit: "Let a configured "on" compile through cmake where
SuiteSparse uses cmake". #652 adds a sentence to the user guide saying that on Windows
a configured "on" becomes "run" ("Say that the JIT stays at "run" on Windows, where
SuiteSparse compiles with cmake"); this PR rewrites that sentence.

The problem

SuiteSparse built with MSVC, as conda-forge builds it for Windows, compiles JIT kernels
only through cmake: gb.ss.config["jit_use_cmake"] is True there and cannot be set
False. SuiteSparse passes the configured compiler to cmake as CMAKE_C_COMPILER, which
cmake's Visual Studio generator ignores; it finds Visual Studio's compiler itself.
conda-forge's builds configure the build machine's cl.exe, a path under
C:/Program Files/Microsoft Visual Studio/ that users rarely have. #652 sets "on" at
init only where the configured compiler exists, so on Windows a configured "on"
(GRAPHBLAS_JIT_C_CONTROL=on, gb.config.set(jit_c_control="on") or a YAML file)
quietly became "run", and nothing compiled. main behaves the same.

The cmake path itself works on Windows CI. #652's test_set_jit_c_control_at_init
pointed the compiler setting at a directory and expected the probe to fail; on every
Windows job with a JIT build in one CI run of this stack (81 jobs, SuiteSparse 8.2.0
through 10.5.2 and upstream), the probe compiled through cmake and the control stayed
"on" (fixed in #652 by "Skip the unusable-compiler probe check where SuiteSparse
compiles with cmake").

What changes

  • A configured "on" looks for cmake where SuiteSparse uses it. At init, where
    jit_use_cmake is True, python-graphblas checks for cmake on the PATH
    (shutil.which) instead of the configured compiler path, then probes as before by
    compiling a small type. A failing probe still means "run", with no error. Nothing
    changes where jit_use_cmake is False, which is everywhere but MSVC builds unless a
    user sets it.
  • UDT work never raises the default "run" where jit_use_cmake is True. Elsewhere,
    UDT operators and registration from C source raise the default "run" to "on" when
    they need a kernel. Through cmake each kernel costs about 10 seconds (below), so
    there only an "on" the user set compiles, even where the configured compiler happens
    to exist (the build machine's cl.exe path, or conda's clang.exe, which the
    compiler repair finds). UDT operators then call their Numba functions. Registering C
    source under that "run" raises an error that says SuiteSparse compiles through cmake
    here and that gb.ss.config["jit_c_control"] = "on" opts in, instead of suggesting
    gb.ss.fix_jit_config(), which does not help there.
  • The probe's name includes the cmake settings. The probe type is named after a
    hash of the toolchain settings, so a toolchain the cache has not seen compiles a
    probe of its own rather than loading one that another toolchain left there.
    jit_use_cmake and jit_c_cmake_libs are now part of that hash; without them, a
    probe through cmake on Linux or macOS loaded the probe compiled directly and tested
    nothing. Each existing cache compiles its probe once more (about 0.3 s directly). A
    test fails if either setting is taken out of the hash, and checks that init, which
    reads the settings through the C API, names the probe as later calls do.
  • Diagnostics. scripts/jit_diagnostics.py prints cmake's path and
    jit_use_cmake, times the diagnostic type's registration, and ends with a new
    section that sets "on" as init does, with burble on, then prints the resulting
    control and how long the probe took, through cmake or the compiler. When
    fix_jit_config() returns None it now says "no compiler found to repair from"; it
    said "no conda", also with CONDA_PREFIX set.
  • Docs. The user guide's JIT section says that on Windows "on" needs cmake
    (conda install cmake) and Visual Studio or its Build Tools rather than the
    configured compiler, that UDT work does not raise the default "run" there, and that a
    new kernel takes about 10 seconds through cmake on Windows. The UDT page's list of
    what raises "run" gains the exception, and so do the jit_c_control and
    jit_use_cmake entries of gb.ss.config.

Cost of a kernel through cmake

In that CI run the test made exactly one cold compile through cmake per Windows job, so
the time it took measures one kernel. Over the 81 jobs: median 10.4 s, 90th percentile
16.2 s, fastest 2.6 s, slowest 56 s. Each kernel gets a fresh build directory and a
fresh cmake configure, so a real kernel costs at least this much. For comparison, on an
M-series Mac the probe compiles in 0.2 to 0.3 s directly and in about 0.6 s through
cmake with its Makefile generator.

What Windows CI should show

  • JIT diagnostics: a cmake: which=... line and jit_use_cmake: True. The "JIT
    compilation test" still fails under the default "run", now with the cmake message.
    The new last section shows the cmake and MSBuild output, then Result: on and
    Probe through cmake: N s, one more sample of the cost above in every job. A wheel
    draw (SuiteSparse without the JIT) shows Result: run and No probe: ....
  • Tests: on JIT draws, test_ss_import_leaves_jit_c_control_alone[on] (a fresh
    process with GRAPHBLAS_JIT_C_CONTROL=on) and
    test_set_jit_c_control_at_init_compiles_through_cmake assert "on" after a real
    compile through cmake; if it fails, the assertion asks whether Visual Studio's C
    compiler is installed. The rest of test_ssjit.py still skips its JIT tests on
    Windows (see the second question below), and the tests that need UDT work to raise
    "run" skip wherever SuiteSparse compiles only through cmake.

Tested locally on macOS (arm64, SuiteSparse 10.5.2, cmake 4.3): test_ssjit.py and
test_core.py with --runslow (90 passed, 1 skipped), and test_op.py and
test_dtype.py with --runslow. The cmake branches ran there with jit_use_cmake set
to True, including a real probe through cmake on a fresh cache; MSVC and the Visual
Studio generator run only on CI.

Open questions

  1. Should UDT work ever use cmake by default? In this PR it never does. At about
    10 s per kernel, a session that touches a handful of UDT operations would wait tens
    of seconds to minutes the first time (the cache keeps the kernels for later runs).
    Without it, Windows users get the Numba fallback, typically 2 to 3 times slower for
    elementwise operations, unless they set "on".
  2. fix_jit_config() in the tests' _setup_jit. It repairs only the compiler path,
    so on Windows it returns None and test_ssjit.py compiles no JIT kernels there.
    Making it accept cmake would run them all through cmake: test_ssjit.py and
    test_core.py compile about 261 kernels into a fresh cache, roughly 45 minutes per
    Windows job at 10 s each.

🤖 Generated with Claude Code

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@eriknw
eriknw added this pull request to stack #660 November 3, 2026 16:45
@codecov

codecov Bot commented Nov 3, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 91.00529% with 17 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (51-coverage-main-gaps@d78f352). Learn more about missing BASE report.

Additional details and impacted files
@@                   Coverage Diff                    @@
##             51-coverage-main-gaps     #659   +/-   ##
========================================================
  Coverage                         ?   99.42%           
========================================================
  Files                            ?      109           
  Lines                            ?    35247           
  Branches                         ?     4014           
========================================================
  Hits                             ?    35044           
  Misses                           ?      117           
  Partials                         ?       86           
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@eriknw
eriknw force-pushed the 54-jit-on-windows-cmake branch from 06b89f2 to 325fef0 Compare November 3, 2026 20:07
@eriknw
eriknw force-pushed the 54-jit-on-windows-cmake branch from 325fef0 to 798f817 Compare November 3, 2026 23:09
@eriknw
eriknw removed this pull request from stack #660 November 3, 2026 23:09
@eriknw
eriknw added this pull request to stack #662 November 3, 2026 23:09
@eriknw
eriknw removed this pull request from stack #662 November 3, 2026 23:11
@eriknw eriknw mentioned this pull request Nov 3, 2026
Fix UDT, dtype and JIT edge cases found in #591's review #663
Open
@eriknw
eriknw added this pull request to stack #664 November 3, 2026 23:12
@eriknw
eriknw force-pushed the 54-jit-on-windows-cmake branch 2 times, most recently from 6131394 to bfb380e Compare November 4, 2026 20:50
@eriknw
eriknw removed this pull request from stack #664 November 4, 2026 20:50
@eriknw
eriknw added this pull request to stack #666 November 4, 2026 20:51
@eriknw
eriknw force-pushed the 54-jit-on-windows-cmake branch from bfb380e to d08c138 Compare November 8, 2026 04:51
@eriknw
eriknw force-pushed the 54-jit-on-windows-cmake branch 3 times, most recently from 06cad53 to 4550cc1 Compare November 9, 2026 15:51
@eriknw
eriknw force-pushed the 54-jit-on-windows-cmake branch from 4550cc1 to 059c0a5 Compare November 10, 2026 06:25
@eriknw
eriknw force-pushed the 54-jit-on-windows-cmake branch 6 times, most recently from 4fca0a4 to d0ea5a0 Compare November 11, 2026 03:26
@eriknw
eriknw force-pushed the 54-jit-on-windows-cmake branch 9 times, most recently from e547574 to ca85506 Compare November 11, 2026 05:42
@eriknw
eriknw removed this pull request from stack #666 November 12, 2026 06:13
@eriknw
eriknw changed the base branch from 51-coverage-main-gaps to main November 12, 2026 06:14
@eriknw
eriknw force-pushed the 54-jit-on-windows-cmake branch from ca85506 to e709fb7 Compare November 12, 2026 06:15
@eriknw eriknw mentioned this pull request Nov 12, 2026
Give UDTs C names that no JIT kernel identifier collides with #669
Open
@eriknw
eriknw changed the base branch from main to 51-coverage-main-gaps November 12, 2026 06:16
@eriknw
eriknw added this pull request to stack #670 November 12, 2026 06:17
@eriknw
eriknw force-pushed the 54-jit-on-windows-cmake branch from e709fb7 to 7cf6aae Compare November 12, 2026 17:10
eriknw added 2 commits November 12, 2026 11:25
@eriknw
Let a configured "on" compile through cmake where SuiteSparse uses cmake
a113b45
SuiteSparse built with MSVC always compiles JIT kernels through cmake, and
passes the configured compiler to it as CMAKE_C_COMPILER, which cmake's
Visual Studio generator ignores. conda-forge's Windows builds name the build
machine's cl.exe, which users rarely have, so the check at init that the
configured compiler exists turned a configured "on" into "run" there.
Where jit_use_cmake is True, init now looks for cmake on the PATH instead,
and probes as before: a failing probe still means "run".

UDT work no longer raises the default "run" where jit_use_cmake is True,
even where the configured compiler exists (the build machine's cl.exe on a
host with that Visual Studio, or conda's clang.exe, which the repair finds):
a cold probe through cmake took a median of 10.4 s (2.6 to 56 s) over 81
wave-5 Windows CI jobs, so there only an "on" the user set compiles.
Registering C source under that "run" says so, and how to opt in. Tests that
need UDT work to raise "run" skip where SuiteSparse compiles only through
cmake.

The probe's type is named after jit_use_cmake and jit_c_cmake_libs too, so
switching to cmake compiles a probe of its own rather than loading the one
compiled directly; a test checks that, and that init names the probe as
later calls do. scripts/jit_diagnostics.py reports where cmake is and
jit_use_cmake, times the diagnostic type's registration, and sets "on" as
init does, timing the probe, with burble on: on Windows CI that times a
compile through cmake, about the least each new kernel costs there.

The user guide and the jit_use_cmake and jit_c_control entries of
gb.ss.config say what "on" needs on Windows (cmake and Visual Studio), that
UDT work does not raise "run" there, and that a new kernel takes about 10
seconds there.
@eriknw
Expect "on" through cmake only where a probe compiles, or should
55125f8
test_ssjit's _jit_through_cmake_here decided whether an "on" set at init
compiles through cmake by setting "on" and reading it back, which runs
no compile. On Windows with psg's 8.0.2.1 wheel, built with MinGW,
SuiteSparse compiles through cmake but passes gcc-style flags, which
cmake's Visual Studio generator hands to cl.exe, which refuses them
(D8021: invalid numeric argument '/Wundef'). Init rightly keeps "run"
there, but the helper said yes, so
test_ss_import_leaves_jit_c_control_alone[on] and
test_set_jit_c_control_at_init_compiles_through_cmake asserted "on" and
failed, with a message asking whether Visual Studio was installed, which
it was.

The helper now asks a real probe, as init does. A failed probe with
gcc-style flags (-W..., -std=, -f...) means cmake cannot work here, so
the tests check the "run" fallback instead; a failed probe with flags
for cl.exe still counts, so the tests report a broken Visual Studio
setup. The message no longer blames a missing Visual Studio. A test
drives the helper with a stand-in probe and flags on any host.
@eriknw
eriknw force-pushed the 54-jit-on-windows-cmake branch from 7cf6aae to 55125f8 Compare November 12, 2026 17:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

1 participant

@eriknw
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.