GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |
GeezerVison is showing github.com — Fix chop_threshold crashes in the pandas-free complex repr by eriknw · Pull Request #620 · python-graphblas/python-graphblas Conversation
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@ ## 25-pandas-free-repr #620 +/- ## ====================================================== Coverage ? 87.00% ====================================================== Files ? 108 Lines ? 34583 Branches ? 3981 ====================================================== Hits ? 30088 Misses ? 4196 Partials ? 299 🚀 New features to boost your workflow:
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
a4bdb8d to
53ad8e3
Compare
The COO form formats a complex column through _format_float_column, where display.chop_threshold compares each value's magnitude. Two crashes hid there, found by an overnight differential fuzz against the pandas renderer: - abs() of a python complex raises OverflowError for finite components near the float max, where numpy's abs returns inf. pandas therefore never chops such a value and renders the object fine; the pandas-free path crashed. Treat an overflowing magnitude as never chopped. Byte-identity for this case is restored, verified against the pandas renderer at 6f1eb02 (repr and HTML hashes equal). - A value that actually chops was replaced with float 0.0, which puts a j-less string into the complex column, and _trim_zeros_complex cannot parse that (IndexError). pandas 3.0.3 has the same bug in its own chop path (pandas.io.formats.format._trim_zeros_complex raises the same IndexError on the same input), so byte-identity here means identity with a crash. Chop to the value's own type of zero instead: the chopped cell renders 0.000000e+00+0.000000e+00j and the object reprs where pandas cannot. The second point is a deliberate, documented divergence from pandas, reachable only with display.chop_threshold set on a complex dtype.
On Windows with python-suitesparse-graphblas older than 7.4.3.1 the CFFI
bindings are built without complex support (MSVC lacks C99 _Complex), so
dtypes._supports_complex is False, FC64 is never registered, and
Vector.from_coo(..., dtype="FC64") raises ValueError before the test
gets to the formatting code under test. Guard with the same
skipif("not dtypes._supports_complex") pattern test_scalar.py uses;
the module already imports dtypes, which the string condition needs.
Verified the test still runs and passes on a complex-capable host, and
that the skipif string evaluates to True in the module namespace when
_supports_complex is flipped to False.
test_chop_threshold_complex failed on the macos-latest job that drew numpy
1.24.4:
RuntimeWarning: overflow encountered in absolute
The pandas-free float formatter takes np.abs of the column to choose between
fixed and scientific notation. For a complex value with components near the
float max the magnitude is inf, which those checks already handle, but numpy
1.24 on arm64 also raises the FP overflow flag computing it (1.26 and 2.4 do
not), so the warning escaped from repr and the test suite's filterwarnings
setting turned it into an error. Reproduced locally with numpy 1.24.4,
pandas 2.2.3 and psg 8.2.0.1, then green with the magnitudes computed under
np.errstate(over="ignore").
pandas' FloatArrayFormatter makes the same np.abs call unguarded (2.2 and
3.0), so this departs from it only in not warning.
test_chop_threshold_complex covered FC64 only. Run it on FC32 as well: its largest value renders, and a value below the threshold chops to complex zero, as for FC64. Replacing the chop's own type of zero with a float zero makes the FC32 case fail with the same IndexError. FC32 reaches the threshold by a different route. The COO frame hands the formatter python complexes, so abs() is computed in double and is finite where pandas computes an inf magnitude in float32. The two differ only for a threshold above the float32 max, so a comment notes it rather than code.
Computing the magnitudes under np.errstate(over="ignore") fixed a RuntimeWarning that only numpy 1.24 on arm64 raises, so on any other numpy no test failed with that fix removed. Stand in for that numpy with an abs whose own arithmetic overflows (the squares of components near the float max), and check that repr and the HTML repr raise no RuntimeWarning for an FC64 value there. Without the errstate the test fails with "overflow encountered in multiply". FC32 values reach the formatter widened to double, so their magnitudes never overflow and the test covers FC64 only.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
There was an error while loading. Please reload this page.