GeezerVison is showing github.com — BUG: Reset an unrepresentable fill_value on a dtype change by danggeun · Pull Request #32508 · numpy/numpy
Conversation
danggeun
commented
Sep 5, 2026
•
edited
Loading
Uh oh!
There was an error while loading. Please reload this page.
PR summary
As discussed in gh-28255. Since #32423, MaskedArray._update_from validates an inherited fill_value against the new dtype and falls back to the default when the cast fails. It decides "fails" by catching an exception, which misses casts that fail through the floating point error state instead.
>>> a = np.ma.arange(9.0)
>>> np.ones_like(a, dtype='int64').fill_value
np.int64(999999)
>>> a.fill_value # only reads it
np.float64(1e+20)
>>> np.ones_like(a, dtype='int64').fill_value
RuntimeWarning: invalid value encountered in cast
np.int64(-9223372036854775808)
1e20 is not a value the user picked, it is the default fill_value for float64, and reading the attribute stores it as a 0-d array. Casting that stored np.float64(1e20) to int64 does not raise: it sets the invalid flag, warns, and returns an out-of-range integer (INT64_MIN on x86; INT64_MAX on the macOS machine in the issue thread). A Python float 1e20 passed directly goes through the scalar path and raises, which is why fill_value=1e20 at construction already fails cleanly.
This wraps that call in np.errstate(invalid='raise') and adds FloatingPointError to the except clause, so the fallback added in #32423 also runs for this case.
Scope:
Tests: the new test compares against the result when fill_value was never read rather than against a specific number, so it does not depend on the platform or on the default integer width. In gh-28255 I said this would not touch existing tests; running the full numpy/ma suite showed that TestMaskedArrayFunctions.test_where in numpy/ma/tests/test_core.py asserted the RuntimeWarning from exactly this cast (set_fill_value(1e20) followed by astype(int)). It now asserts that the cast emits no warning and that ixm.fill_value is the default for the new dtype. The rest of numpy/ma passes unchanged (run against 2.5.2 with the change applied).
Fixes #28255
First time committer introduction
Not my first PR (#32405 was a whitespace fix), but my first change to numpy.ma. I came to numpy.ma through gh-15601, which I am also preparing a fix for, and found this one while going through open ma issues around the time #32423 was being reviewed.
AI Disclosure
I used Claude throughout: to help investigate the cause, to draft the two-line change, the tests, and the release note from my notes, and to put this description and the code comments into English. I reviewed and applied the changes, and ran the reproducer, the new tests, and the numpy/ma suite myself.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given how insidious the original bug was, this is a needed fix. And the fix is at the right place.
The other option would be to fix the _check_fill_value() function itself, to raise a FloatingPointError instead of only a warning. But the impact radius of this might be higher, so this is probably a sufficient fix here, as-is.
I think even if we somehow in the future can separate out user fill value and default fill value, i.e. separate out intended fill value vs inadvertent fill value (as the bug this fixes was), then this fix is still necessary to get meaningful fill values where there's a floating point overflow. So valuable regardless.
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
danggeun reacted with thumbs up emoji
|
Thanks, I agree this is correct and in the vein of what we ended up with for #32423.
If you want to follow up, there is still a remaining code path that needs a similar fix:
>>> a = np.ma.arange(9.0)
>>> a.fill_value
np.float64(1e+20)
>>> np.ma.array(a, dtype='int64').fill_value
RuntimeWarning: invalid value encountered in cast
np.int64(9223372036854775807)
This happens because MaskedArray.__new__ re-runs _check_fill_value on the inherited fill_value without the fallback patched here. We should probably make MaskedArray.__new__ and _update_from call a shared helper that contains the fill_value fallback logic.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
|
Also argh unfortunately I missed it before merging but if you have time or interest it would be nice to make this and #32423 share a release note. You can make the content of the release note identical for both PRs and then towncrier will generate a changelog entry that links to both PRs.
|
seberg and danggeun reacted with thumbs up emoji
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
|
Thanks for pointing out the remaining path. I'll open a follow-up that moves the existing fallback into a shared helper used by both MaskedArray.__new__ and _update_from. In __new__ it only applies to the fill_value inherited from data; a fill_value passed by the user still raises as before. I'll also make the release notes for this PR, #32423 and the follow-up identical so towncrier groups them into one entry.
|
Sorry, something went wrong.
Uh oh!
There was an error while loading. Please reload this page.
charris
added a commit
that referenced
this pull request
Sep 10, 2026
The backport of #32508 brought with it a number of new tests, this
fixes a test failure by adding a needed import.
charris
added a commit
that referenced
this pull request
Sep 10, 2026
BUG: Reset an unrepresentable fill_value on a dtype change (#32508)
Reviewers
1 more reviewer
Reviewers whose approvals may not affect merge requirements
Uh oh!
There was an error while loading. Please reload this page.
Assignees
No one assigned
Milestone
No milestone
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
|