GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — fix: Normalize Dask timestamps to UTC without a row-wise apply by Daksha1611 · Pull Request #6818 · feast-dev/feast

Conversation

@Daksha1611

@Daksha1611 Daksha1611 commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

_normalize_timestamp made timestamp columns tz-aware with a row-wise apply that
declared meta=(timestamp_field, "datetime64[ns, UTC]"). meta only declares the
dtype — the partition gets whatever the lambda returns. With zero rows the lambda never
runs, so the computed column stayed tz-naive while meta claimed UTC, and the tz-aware
comparison in _filter_ttl then raised:

TypeError: Invalid comparison between dtype=datetime64[ns] and DatetimeArray

A zero-row entity_df is a normal case in batch scoring — the upstream query matched
nothing for that run — so get_historical_features crashed instead of returning an
empty result.

While fixing this I found the same mismatch on a second path: a column carrying a
non-UTC timezone also diverged from the declared meta, because the lambda returned any
value that already had tzinfo untouched. That is covered by a test here too.

Both applies are replaced with dd.to_datetime(..., utc=True), which localizes tz-naive
values, converts tz-aware ones, and yields the correct dtype for an empty frame. It is
also vectorized, so it drops a Python-level call per row on non-empty frames.

Which issue(s) this PR fixes:

Fixes #6817

Checks

  • I've made sure the tests are passing.
  • My commits are signed off (git commit -s)
  • My PR title follows conventional commits format

Testing Strategy

  • Unit tests
  • Integration tests
  • Manual tests
  • Testing is not required for this change

New sdk/python/tests/unit/infra/offline_stores/test_dask_empty_entity_df.py covers
normalization and TTL filtering at 0 and 1 rows, plus the non-UTC case. Three of its
five assertions fail on master and pass here; the two 1-row cases pass both ways,
pinning the non-empty behaviour as unchanged.

Verified end to end as well: get_historical_features(...).to_df() against a file
offline store with an otherwise identical entity frame raised TypeError at 0 rows
before this change, and now returns an empty frame with the expected columns while the
1-row case is unchanged.

sdk/python/tests/unit/infra/offline_stores plus test_unit_feature_store.py: 235
passed. The 20 errors in that run are MongoDB testcontainers failing to start locally
and are present on master too.

ruff check, ruff format --check and mypy are clean on both changed files.

Misc

Point-in-time correctness is unaffected — I re-ran the filter_by_created_timestamp
scenario (a row backfilled after the entity timestamp) and it still excludes the late
row with the flag and includes it without.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@Daksha1611
fix: Normalize Dask timestamps to UTC without a row-wise apply
042397e
_normalize_timestamp made timestamp columns tz-aware with a row-wise apply
declaring meta="datetime64[ns, UTC]". meta only declares the dtype; the
partition gets whatever the lambda returns. With zero rows the lambda never
runs, so the computed column stayed tz-naive while meta claimed UTC, and the
tz-aware comparison in _filter_ttl then raised:

    TypeError: Invalid comparison between dtype=datetime64[ns] and DatetimeArray

A zero-row entity_df is a normal case in batch scoring, when the upstream
query matched nothing for that run, so get_historical_features crashed rather
than returning an empty result.

A column carrying a non-UTC timezone diverged from the declared meta the same
way, because the lambda returned any value that already had tzinfo untouched.

Replace both applies with dd.to_datetime(..., utc=True), which localizes
tz-naive values, converts tz-aware ones, and yields the correct dtype for an
empty frame. It is also vectorized, so it avoids a Python-level call per row
on non-empty frames.

Signed-off-by: Daksha1611 <[email protected]>
@Daksha1611
Daksha1611 requested a review from a team as a code owner September 6, 2026 18:33
@codecov-commenter

codecov-commenter commented Sep 9, 2026

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 47.08%. Comparing base (5ad5592) to head (042397e).
⚠️ Report is 1 commits behind head on master.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #6818      +/-   ##
==========================================
- Coverage   47.08%   47.08%   -0.01%     
==========================================
  Files         419      419              
  Lines       51878    51876       -2     
  Branches     7525     7525              
==========================================
- Hits        24429    24427       -2     
  Misses      25700    25700              
  Partials     1749     1749              
Flag Coverage Δ *Carryforward flag
go-feature-server 30.58% <ø> (ø)
python-unit 48.39% <ø> (-0.01%) ⬇️ Carriedforward from 5ad5592

*This pull request uses carry forward flags. Click here to find out more.

Files with missing lines Coverage Δ
sdk/python/feast/infra/offline_stores/dask.py 52.19% <ø> (-0.19%) ⬇️

Continue to review full report in Codecov by Harness.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 5ad5592...042397e. Read the comment docs.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Daksha1611
Merge branch 'master' into fix/dask-empty-entity-df
0604a7a
@Daksha1611 Daksha1611 mentioned this pull request Oct 6, 2026
fix: Fall back to the serialized body when UDF source misses globals #6816
Merged
5 of 7 tasks
@haoxu0 haoxu0 added the ok-to-test label Oct 6, 2026
@Daksha1611
Merge branch 'master' into fix/dask-empty-entity-df
15f72fe

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

ok-to-test

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

get_historical_features raises TypeError on a zero-row entity_df (Dask/file offline store)

Uh oh!

There was an error while loading. Please reload this page.

3 participants

@Daksha1611 @codecov-commenter @haoxu0
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.