GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — BUG: Reset an unrepresentable fill_value on a dtype change by danggeun · Pull Request #32508 · numpy/numpy

Conversation

@danggeun

@danggeun danggeun commented Sep 5, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

PR summary

As discussed in gh-28255. Since #32423, MaskedArray._update_from validates an inherited fill_value against the new dtype and falls back to the default when the cast fails. It decides "fails" by catching an exception, which misses casts that fail through the floating point error state instead.

>>> a = np.ma.arange(9.0)
>>> np.ones_like(a, dtype='int64').fill_value
np.int64(999999)
>>> a.fill_value          # only reads it
np.float64(1e+20)
>>> np.ones_like(a, dtype='int64').fill_value
RuntimeWarning: invalid value encountered in cast
np.int64(-9223372036854775808)

1e20 is not a value the user picked, it is the default fill_value for float64, and reading the attribute stores it as a 0-d array. Casting that stored np.float64(1e20) to int64 does not raise: it sets the invalid flag, warns, and returns an out-of-range integer (INT64_MIN on x86; INT64_MAX on the macOS machine in the issue thread). A Python float 1e20 passed directly goes through the scalar path and raises, which is why fill_value=1e20 at construction already fails cleanly.

This wraps that call in np.errstate(invalid='raise') and adds FloatingPointError to the except clause, so the fallback added in #32423 also runs for this case.

Scope:

  • Only invalid is raised. over is left alone on purpose: a float fill_value that saturates to inf in a narrower float dtype is still a usable fill_value, and replacing it with the default would not be an improvement.
  • Whether a lossy float-to-integer cast sets the invalid flag is platform dependent. -1.0 to uint8 sets no flag on x86 and wraps to 255, so this change does not touch it there; on platforms where it does set the flag (BUG: MaskedArray.astype('uint8') with certain fill_value raises warning on ARM (Mac M3) inside Docker (Ubuntu 24.04/25.04) and leads inconsistent output #28403), the fill_value now falls back to the default instead of a platform-specific value.
  • arr.fill_value = ... still raises, since the setter calls _check_fill_value directly.
  • The wider questions raised in BUG: np.ones_like behaves differently depending on whether the fill_value of the input array has been accessed #28255 (reading fill_value storing the default, separating default and user fill values) are not addressed here.

Tests: the new test compares against the result when fill_value was never read rather than against a specific number, so it does not depend on the platform or on the default integer width. In gh-28255 I said this would not touch existing tests; running the full numpy/ma suite showed that TestMaskedArrayFunctions.test_where in numpy/ma/tests/test_core.py asserted the RuntimeWarning from exactly this cast (set_fill_value(1e20) followed by astype(int)). It now asserts that the cast emits no warning and that ixm.fill_value is the default for the new dtype. The rest of numpy/ma passes unchanged (run against 2.5.2 with the change applied).

Fixes #28255

First time committer introduction

Not my first PR (#32405 was a whitespace fix), but my first change to numpy.ma. I came to numpy.ma through gh-15601, which I am also preparing a fix for, and found this one while going through open ma issues around the time #32423 was being reviewed.

AI Disclosure

I used Claude throughout: to help investigate the cause, to draft the two-line change, the tests, and the release note from my notes, and to put this description and the code comments into English. I reviewed and applied the changes, and ran the reproducer, the new tests, and the numpy/ma suite myself.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@github-actions github-actions Bot added the 00 - Bug label Sep 5, 2026
jdavies-st
jdavies-st reviewed Sep 7, 2026
View reviewed changes

@jdavies-st jdavies-st left a comment

Copy link
Copy Markdown
Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given how insidious the original bug was, this is a needed fix. And the fix is at the right place.

The other option would be to fix the _check_fill_value() function itself, to raise a FloatingPointError instead of only a warning. But the impact radius of this might be higher, so this is probably a sufficient fix here, as-is.

I think even if we somehow in the future can separate out user fill value and default fill value, i.e. separate out intended fill value vs inadvertent fill value (as the bug this fixes was), then this fix is still necessary to get meaningful fill values where there's a floating point overflow. So valuable regardless.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

danggeun reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@charris charris added the 09 - Backport-Candidate PRs tagged should be backported label Sep 7, 2026
danggeun added 2 commits September 8, 2026 03:31
@danggeun
BUG: Reset an unrepresentable fill_value on a dtype change
1b90f78
@danggeun
DOC: Rename release note to PR number
4e5dfa2
@danggeun
danggeun force-pushed the bugfix/ma-fill-value-fp-cast branch from 697f5ac to 4e5dfa2 Compare September 7, 2026 18:34
@ngoldbaum

ngoldbaum commented Sep 8, 2026

Copy link
Copy Markdown
Member

Thanks, I agree this is correct and in the vein of what we ended up with for #32423.

If you want to follow up, there is still a remaining code path that needs a similar fix:

>>> a = np.ma.arange(9.0)
>>> a.fill_value
np.float64(1e+20)
>>> np.ma.array(a, dtype='int64').fill_value
RuntimeWarning: invalid value encountered in cast
np.int64(9223372036854775807)

This happens because MaskedArray.__new__ re-runs _check_fill_value on the inherited fill_value without the fallback patched here. We should probably make MaskedArray.__new__ and _update_from call a shared helper that contains the fill_value fallback logic.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ngoldbaum
ngoldbaum merged commit 3274e70 into numpy:main Sep 8, 2026
91 checks passed

Uh oh!

There was an error while loading. Please reload this page.

@ngoldbaum

ngoldbaum commented Sep 8, 2026

Copy link
Copy Markdown
Member

Also argh unfortunately I missed it before merging but if you have time or interest it would be nice to make this and #32423 share a release note. You can make the content of the release note identical for both PRs and then towncrier will generate a changelog entry that links to both PRs.

seberg and danggeun reacted with thumbs up emoji
All reactions
  • 👍 2 reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@danggeun

danggeun commented Sep 8, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for pointing out the remaining path. I'll open a follow-up that moves the existing fallback into a shared helper used by both MaskedArray.__new__ and _update_from. In __new__ it only applies to the fill_value inherited from data; a fill_value passed by the user still raises as before. I'll also make the release notes for this PR, #32423 and the follow-up identical so towncrier groups them into one entry.

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@charris charris mentioned this pull request Sep 9, 2026
BUG: Reset an unrepresentable fill_value on a dtype change (#32508) #32557
Merged
@charris charris removed the 09 - Backport-Candidate PRs tagged should be backported label Sep 9, 2026
danggeun added a commit to danggeun/numpy that referenced this pull request Sep 9, 2026
@danggeun
BUG: Reset an inherited fill_value in MaskedArray.__new__ as well
36fc28a
Follow-up to numpy#32508.
danggeun added a commit to danggeun/numpy that referenced this pull request Sep 9, 2026
@danggeun
BUG: Reset an inherited fill_value in MaskedArray.__new__ as well
c537180
Follow-up to numpy#32508.
danggeun added a commit to danggeun/numpy that referenced this pull request Sep 9, 2026
@danggeun
BUG: Reset an inherited fill_value in MaskedArray.__new__ as well
a79fbe1
Follow-up to numpy#32508.
@danggeun danggeun mentioned this pull request Sep 9, 2026
BUG: Reset an inherited fill_value in MaskedArray.__new__ as well #32562
Merged
@danggeun
danggeun deleted the bugfix/ma-fill-value-fp-cast branch September 9, 2026 21:24
charris pushed a commit that referenced this pull request Sep 10, 2026
@danggeun @charris
BUG: Reset an unrepresentable fill_value on a dtype change (#32508)
bdf28f6
Co-authored-by: Yeonho Kim <[email protected]>
charris added a commit that referenced this pull request Sep 10, 2026
@charris
MAINT: Import common_fill_values.
974fd1a
The backport of #32508 brought with it a number of new tests, this
fixes a test failure by adding a needed import.
charris added a commit that referenced this pull request Sep 10, 2026
@charris
Merge pull request #32557 from charris/backport-32508-32358
421a366
BUG: Reset an unrepresentable fill_value on a dtype change  (#32508)
@charris charris mentioned this pull request Sep 20, 2026
BUG: Reset an inherited fill_value in MaskedArray.__new__ as well (#32562) #32710
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer
@jdavies-st jdavies-st jdavies-st left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

00 - Bug

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

BUG: np.ones_like behaves differently depending on whether the fill_value of the input array has been accessed

Uh oh!

There was an error while loading. Please reload this page.

4 participants

@danggeun @ngoldbaum @jdavies-st @charris
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.