GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — perf(postgres): Optimizing feast offline Store for date-range multi-FV retrieval by Vperiodt · Pull Request #6057 · feast-dev/feast

Conversation

@Vperiodt

@Vperiodt Vperiodt commented Mar 4, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor

What this PR does / why we need it:

  1. Replaces the date-range multi-FV path in the Postgres offline store (previously base_entities + one LEFT JOIN LATERAL per feature view) with a set-based LOCF (Last Observation Carried Forward) implementation

  2. Uses a single timeline: stack (UNION ALL of spine + feature rows), COUNT for group boundaries, FIRST_VALUE for forward-fill, then filter to spine and apply per-FV TTL.

Which issue(s) this PR fixes:

Fixes slow get_historical_features on the Postgres offline store for date-range retrieval with multiple feature views. The LATERAL/inequality joins had O(N×M) cost. This PR switches to a LOCF path: one stacked timeline of size L, sort O(L log L), and O(L) window passes, which makes total cost as O(L log L).

Misc

Scenario: get_historical_features(features=[...], start_date=2023-01-01, end_date=2023-01-07) with no entity DataFrame and two feature views:

  • driver_fv: entity driver_id, TTL 1 day, feature score
  • customer_fv: entity customer_id, TTL 0, feature amount

Inputs

Parameter Value
start_date 2023-01-01
end_date 2023-01-07
entity_df None (non-entity mode)
FVs driver_fv (driver_id, score, TTL 1d), customer_fv (customer_id, amount, TTL 0)

1. Feature data window (__data_raw)

Feature data is explicitly pulled from lookback to end_date so LOCF can carry the last observation before start_date.

Time window used for feature data

FV TTL Time window
driver_fv 1 day [lookback_start_date, end_date]
customer_fv 0 [start_date, end_date]

With lookback_start_date = start_date - max_ttl (e.g. 2022-12-31 for 1-day TTL), driver_fv rows before 2023-01-01 are now included so LOCF can fill correctly.

-- Per-FV __data_raw: feature rows from lookback_start_date..end_date
"driver_fv__data_raw" AS (
  SELECT "ts" AS event_timestamp, "driver_id", "score" AS "score"
  FROM "driver_table" AS sub
  WHERE "ts" BETWEEN '2023-01-01 00:00:00+00' - interval '1 day' AND '2023-01-07 00:00:00+00'
),
"customer_fv__data_raw" AS (
  SELECT "ts" AS event_timestamp, "customer_id", "amount" AS "amount"
  FROM "customer_table" AS sub
  WHERE "ts" BETWEEN '2023-01-01 00:00:00+00' AND '2023-01-07 00:00:00+00'
),

2. Spine: unified (entity, timestamp) grid

Spine is built from each FV’s __data / __data_raw over [start_date, end_date]. When FVs have different entity sets, the template emits NULL AS "entity" for the FV that doesn’t have that entity:

spine AS (
  SELECT DISTINCT d.event_timestamp, d."driver_id", NULL AS "customer_id"
  FROM "driver_fv__data_raw" d
  WHERE d.event_timestamp BETWEEN '2023-01-01 00:00:00+00' AND '2023-01-07 00:00:00+00'
  UNION
  SELECT DISTINCT d.event_timestamp, NULL AS "driver_id", d."customer_id"
  FROM "customer_fv__data_raw" d
  WHERE d.event_timestamp BETWEEN '2023-01-01 00:00:00+00' AND '2023-01-07 00:00:00+00'
),

Example spine shape (

Spine = distinct (event_timestamp, driver_id, customer_id) from both FVs; missing entity is NULL.

event_timestamp driver_id customer_id
2023-01-01 00:00:00+00 101 NULL
2023-01-01 00:00:00+00 NULL 201
2023-01-02 00:00:00+00 101 NULL
… … …

3. TTL in the final SELECT

-- driver_fv has TTL 86400 (1 day)
CASE WHEN (spine.event_timestamp - "driver_fv__f"."driver_fv__filled_ts") <= make_interval(secs => 86400)
     THEN "driver_fv__f"."score" ELSE NULL END AS "score",
-- customer_fv has TTL 0: no CASE
"customer_fv__f"."amount" AS "amount"

Example final output shape

After LOCF and TTL: one row per (event_timestamp, driver_id, customer_id); score is NULL when outside TTL.

event_timestamp driver_id customer_id score amount
2023-01-01 00:00:00+00 101 NULL 0.8 NULL
2023-01-01 00:00:00+00 NULL 201 NULL 99.5
2023-01-02 00:00:00+00 101 NULL 0.9 NULL
… … … … …
Open with Devin

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@Vperiodt
Vperiodt marked this pull request as ready for review March 4, 2026 07:06
@Vperiodt
Vperiodt requested a review from a team as a code owner March 4, 2026 07:06
devin-ai-integration[bot]

This comment was marked as resolved.

Sign in to view

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@tokoko

tokoko commented Mar 4, 2026

Copy link
Copy Markdown
Collaborator

@Vperiodt thanks for the PR. if it's not too much trouble, can you give a small example of result queries for say.. 2 fv scenario and highlight differences this PR introduces? there jinja templates are hard enough to read through even when one knows what the intent is 😄

Vperiodt reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Vperiodt

Vperiodt commented Mar 4, 2026 •
edited
Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link
Copy Markdown
Contributor Author

@Vperiodt thanks for the PR. if it's not too much trouble, can you give a small example of result queries for say.. 2 fv scenario and highlight differences this PR introduces? there jinja templates are hard enough to read through even when one knows what the intent is 😄

sure ! will add it in the description itself

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

devin-ai-integration[bot]

This comment was marked as resolved.

Sign in to view

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@Vperiodt
Vperiodt marked this pull request as draft March 5, 2026 17:18
@Vperiodt
Vperiodt marked this pull request as ready for review March 9, 2026 21:20
devin-ai-integration[bot]

This comment was marked as resolved.

Sign in to view

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@ntkathole ntkathole mentioned this pull request Mar 10, 2026
feat: Add non-entity retrieval support for ClickHouse offline store #6066
Merged
5 tasks done
YassinNouh21
YassinNouh21 reviewed Mar 15, 2026
View reviewed changes

@YassinNouh21 YassinNouh21 left a comment

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The LOCF approach is a solid improvement over the LATERAL join path — O(L log L) window passes vs O(N×M) inequality joins is a real win for large date-range queries. The per-FV independent pipeline is clean and avoids cross-FV interference.

A few issues to address before merge — one pre-existing bug that this PR should fix since it's modifying the non-entity path, dead code in the template, and some test coverage gaps.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread sdk/python/feast/infra/offline_stores/contrib/postgres_offline_store/postgres.py Outdated
else start_date
)

entity_df = pd.DataFrame(

@YassinNouh21 YassinNouh21 Mar 15, 2026

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug (pre-existing, should fix in this PR): This still uses pd.date_range(start=start_date, ...)[:1] which takes start_date as the entity timestamp, not end_date. This was identified in the PR #6066 review — the ClickHouse implementation correctly uses [end_date].

Since this PR is already modifying the non-entity retrieval path, it's a good opportunity to fix this:

entity_df = pd.DataFrame({"event_timestamp": [end_date]})

In the current code, the entity_df isn't used by the LOCF query path (the spine replaces it), but it's still uploaded to a temp table via _upload_entity_df — so at minimum it wastes a round-trip to Postgres. Even better: skip creating/uploading the entity_df entirely when start_date and end_date are set, since the LOCF path never references left_table_query_string.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Vperiodt reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
Comment thread sdk/python/feast/infra/offline_stores/contrib/postgres_offline_store/postgres.py Outdated
# Compute lookback_start_date for LOCF: pull feature data
# from (start_date - max_ttl) so window functions can
# forward-fill the last observation before start_date.
max_ttl_seconds = 0

@YassinNouh21 YassinNouh21 Mar 15, 2026

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicate computation: When start_date is None, max_ttl_seconds is already computed in the block above (lines 147-151) to derive start_date. This block recomputes the exact same value. Consider hoisting max_ttl_seconds to a single computation before both uses:

max_ttl_seconds = max(
    (int(fv.ttl.total_seconds()) for fv in feature_views
     if fv.ttl and isinstance(fv.ttl, timedelta)),
    default=0,
)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Vperiodt reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
Comment thread sdk/python/feast/infra/offline_stores/contrib/postgres_offline_store/postgres.py Outdated
{% endfor %}

{# Build list of output feature names per FV for consistent column ordering #}
{% set all_feature_cols = [] %}

@YassinNouh21 YassinNouh21 Mar 15, 2026

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dead code: all_feature_cols is computed here but never referenced in the rest of the template. The final SELECT iterates over featureview.features directly and computes col_name inline. This entire block (lines 569-579) can be removed.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Vperiodt reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
Comment thread sdk/python/feast/infra/offline_stores/contrib/postgres_offline_store/postgres.py Outdated
"{{ featureview.name }}__grouped" AS (
SELECT *,
COUNT(feature_anchor) OVER (
PARTITION BY {% if featureview.entities %}{% for entity in featureview.entities %}"{{ entity }}"{% if not loop.last %}, {% endif %}{% endfor %}{% else %}1{% endif %}

@YassinNouh21 YassinNouh21 Mar 15, 2026

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Entityless FV edge case: PARTITION BY 1 works in Postgres but is non-standard and some query planners may not optimize it well. An alternative is PARTITION BY (SELECT NULL) or simply omitting the PARTITION clause (which gives one partition over the whole result set).

Also worth adding a test case for entityless feature views to verify the PARTITION BY 1 and {% else %}1{% endif %} paths generate valid SQL.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Vperiodt reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
Comment thread sdk/python/feast/infra/offline_stores/contrib/postgres_offline_store/postgres.py
LEFT JOIN "{{ featureview.name }}__filled" AS "{{ featureview.name }}__f"
ON spine.event_timestamp = "{{ featureview.name }}__f".event_timestamp
{% for entity in all_entities %}
AND spine."{{ entity }}" IS NOT DISTINCT FROM "{{ featureview.name }}__f"."{{ entity }}"

@YassinNouh21 YassinNouh21 Mar 15, 2026

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good use of IS NOT DISTINCT FROM here. This fixes a bug in the old LATERAL join path where NULL = NULL would fail the join for FVs with different entity sets. Worth calling this out in the PR description as a bug fix.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Vperiodt reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
Comment thread sdk/python/feast/infra/offline_stores/contrib/postgres_offline_store/postgres.py Outdated
{% set col_name = featureview.field_mapping.get(feature, feature) %}
{% endif %}
{% if featureview.ttl != 0 %}
CASE WHEN (spine.event_timestamp - "{{ featureview.name }}__f"."{{ featureview.name }}__filled_ts") <= {{ featureview.ttl }} * interval '1' second

@YassinNouh21 YassinNouh21 Mar 15, 2026

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The TTL enforcement via CASE is correct — when filled_ts is NULL (no prior observation), the subtraction yields NULL, the comparison is UNKNOWN, and CASE falls through to ELSE NULL.

One edge case: if featureview.ttl is a very large number, {{ featureview.ttl }} * interval '1' second could overflow on some Postgres versions. Consider using make_interval(secs => {{ featureview.ttl }}) for safety, though in practice TTLs are unlikely to overflow.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Vperiodt reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
Comment thread sdk/python/feast/infra/offline_stores/contrib/postgres_offline_store/postgres.py
WHERE "{{ featureview.timestamp_field }}" BETWEEN '{{ lookback_start_date or start_date }}' AND '{{ end_date }}'
),
{% if featureview.created_timestamp_column %}
"{{ featureview.name }}__data" AS (

@YassinNouh21 YassinNouh21 Mar 15, 2026

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: When there's no created_timestamp_column, this CTE is a no-op passthrough (SELECT * FROM __data_raw). Postgres will likely optimize it away, but you could simplify by using __data_raw as the alias directly:

{% if featureview.created_timestamp_column %}
"{{ featureview.name }}__data" AS (
    SELECT * FROM (
        ... dedup logic ...
    ) __dedup WHERE __rn = 1
),
{% endif %}

Then reference {{ featureview.name }}__data{% if not featureview.created_timestamp_column %}_raw{% endif %} downstream. Though I understand the current approach is simpler to maintain.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Vperiodt reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
Comment thread sdk/python/tests/unit/infra/offline_stores/contrib/postgres_offline_store/test_postgres.py
def test_lateral_join_ttl_constraints(self):
"""Test that LATERAL JOINs include proper TTL constraints"""
from jinja2 import BaseLoader, Environment
def test_locf_template_multi_fv_date_range(self):

@YassinNouh21 YassinNouh21 Mar 15, 2026

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good tests for the happy path and TTL=0 case. Consider adding coverage for:

  1. Different entity sets across FVs — e.g., fv1 has driver_id, fv2 has customer_id. This exercises the NULL AS "{{ entity }}" and IS NOT DISTINCT FROM paths which are the trickiest part of the LOCF approach.
  2. Entityless feature view — verifies the PARTITION BY 1 fallback generates valid SQL.
  3. created_timestamp_column set — verifies the dedup CTE (__data_raw → __data via ROW_NUMBER) is generated correctly.
  4. full_feature_names=True — verifies the featureview.name ~ '__' ~ feature column naming.

These are the main code paths that differ from the happy path and are most likely to produce malformed SQL.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Vperiodt reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@YassinNouh21 YassinNouh21 mentioned this pull request Mar 15, 2026
fix(postgres): Use end_date in synthetic entity_df for non-entity retrieval #6110
Merged
3 tasks done
@Vperiodt

Vperiodt commented Mar 24, 2026

Copy link
Copy Markdown
Contributor Author

hey ! @YassinNouh21 Thanks for your review earlier, it would be great if you can review it again

All reactions

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

@jyejare jyejare added the ok-to-test label Apr 7, 2026
@jyejare

jyejare commented Apr 7, 2026

Copy link
Copy Markdown
Collaborator

@Vperiodt Could you please rebase the PR for integration tests run ?

Vperiodt reacted with thumbs up emoji
All reactions
  • 👍 1 reaction

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Vperiodt added 9 commits April 10, 2026 15:42
@Vperiodt
Optimizing Date-Range Queries
309cdac
Signed-off-by: Vanshika Vanshika <[email protected]>

rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED
@Vperiodt
fix issues
3da5af3
Signed-off-by: Vanshika Vanshika <[email protected]>

rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED
@Vperiodt
fix lint
34eed9a
Signed-off-by: Vanshika Vanshika <[email protected]>

rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED
@Vperiodt
featureview entities
a4139b7
Signed-off-by: Vanshika Vanshika <[email protected]>

rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED
@Vperiodt
fix LOCF spine UNION and empty-entity SQL
74a474c
Signed-off-by: Vanshika Vanshika <[email protected]>

rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED
@Vperiodt
added fixes
6f754b8
Signed-off-by: Vanshika Vanshika <[email protected]>

rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED
@Vperiodt
updates test
7594205
Signed-off-by: Vanshika Vanshika <[email protected]>

rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED
@Vperiodt
fix entity_df range
f770c41
Signed-off-by: Vanshika Vanshika <[email protected]>

rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED
@Vperiodt
updated postgres.py
f235eb8
Signed-off-by: Vanshika Vanshika <[email protected]>

rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED
@Vperiodt
Vperiodt force-pushed the patch-query branch from d96cee1 to f235eb8 Compare April 10, 2026 10:15
@Vperiodt
fix-lint
8c66e6f
Signed-off-by: Vanshika Vanshika <[email protected]>

rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED
ntkathole
ntkathole reviewed Apr 13, 2026
View reviewed changes
Comment thread sdk/python/feast/infra/offline_stores/contrib/postgres_offline_store/postgres.py Outdated
Comment on lines 137 to 170
# Default to current time if end_date not provided
if end_date is None:
end_date = datetime.now(tz=timezone.utc)
else:
end_date = make_tzaware(end_date)
# Find the maximum TTL across all feature views to ensure we capture enough data
max_ttl_seconds = max(
(
int(fv.ttl.total_seconds())
for fv in feature_views
if fv.ttl and isinstance(fv.ttl, timedelta)
),
default=0,
)

# Calculate start_date from TTL if not provided
if start_date is None:
if max_ttl_seconds > 0:
# Start from (end_date - max_ttl) to ensure we capture all relevant features
start_date = end_date - timedelta(seconds=max_ttl_seconds)
else:
# If no TTL is set, default to 30 days before end_date
start_date = end_date - timedelta(days=30)
else:
start_date = make_tzaware(start_date)

# Compute lookback_start_date for LOCF: pull feature data
# from (start_date - max_ttl) so window functions can
# forward-fill the last observation before start_date.
lookback_start_date: Optional[datetime] = (
start_date - timedelta(seconds=max_ttl_seconds)
if max_ttl_seconds > 0
else start_date
)

@ntkathole ntkathole Apr 13, 2026

Copy link
Copy Markdown
Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it;s still betetr to use compute_non_entity_date_range helper, you can do:

if entity_df is None:
    start_date, end_date = compute_non_entity_date_range(
        feature_views, start_date=start_date, end_date=end_date,
    )
    # Only extra piece needed for LOCF: lookback window
    max_ttl_seconds = max(
        (int(fv.ttl.total_seconds()) for fv in feature_views
         if fv.ttl and isinstance(fv.ttl, timedelta)),
        default=0,
    )
    lookback_start_date = (
        start_date - timedelta(seconds=max_ttl_seconds)
        if max_ttl_seconds > 0 else start_date
    )
    entity_df = pd.DataFrame({"event_timestamp": [end_date]})
    skip_entity_upload = True

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Vperiodt reacted with thumbs up emoji
All reactions
  • 👍 1 reaction
@jyejare jyejare mentioned this pull request Apr 16, 2026
feat: DuckDB historical retrieval without entity dataframe #6108
Open
jyejare
jyejare reviewed Apr 16, 2026
View reviewed changes

@jyejare jyejare left a comment

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Summary

The LOCF (Last Observation Carried Forward) approach is a solid replacement for the LATERAL JOIN path — the per-FV independent forward-fill pipelines (__stacked → __grouped → __filled) correctly avoid cross-FV interference, and the edge cases are well handled (entityless FVs with PARTITION BY (SELECT NULL), different entity sets with NULL padding + IS NOT DISTINCT FROM, created_timestamp_column dedup via ROW_NUMBER). The make_interval(secs => ...) improvement applies to the existing entity_df path too, which is a nice bonus.

Test coverage is strong with 7 new test methods, all using sqlglot.parse validation.

However, CI is currently failing (lint-python and unit-test-python 3.10), so this is not merge-ready yet.

Blockers

  1. CI failures — lint-python and unit-test-python (3.10, ubuntu-latest) are red on the latest run (Apr 13). Please investigate and fix.
  2. Reuse compute_non_entity_date_range — as @ntkathole flagged, the inlined date-range logic should use the existing utility. This also eliminates the duplicate max_ttl_seconds computation. See inline comment for the suggested pattern.

Minor Items

  1. Skip entity_df creation in LOCF path — the synthetic entity_df and dummy left_table_query_string are never used by the LOCF template. Consider skipping that overhead entirely.
  2. The IS NOT DISTINCT FROM fix for NULL entity comparisons is an improvement over the old LATERAL path — worth calling out in the PR description as a correctness fix.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread sdk/python/feast/infra/offline_stores/contrib/postgres_offline_store/postgres.py Outdated
Comment on lines 137 to 170
# Default to current time if end_date not provided
if end_date is None:
end_date = datetime.now(tz=timezone.utc)
else:
end_date = make_tzaware(end_date)
# Find the maximum TTL across all feature views to ensure we capture enough data
max_ttl_seconds = max(
(
int(fv.ttl.total_seconds())
for fv in feature_views
if fv.ttl and isinstance(fv.ttl, timedelta)
),
default=0,
)

# Calculate start_date from TTL if not provided
if start_date is None:
if max_ttl_seconds > 0:
# Start from (end_date - max_ttl) to ensure we capture all relevant features
start_date = end_date - timedelta(seconds=max_ttl_seconds)
else:
# If no TTL is set, default to 30 days before end_date
start_date = end_date - timedelta(days=30)
else:
start_date = make_tzaware(start_date)

# Compute lookback_start_date for LOCF: pull feature data
# from (start_date - max_ttl) so window functions can
# forward-fill the last observation before start_date.
lookback_start_date: Optional[datetime] = (
start_date - timedelta(seconds=max_ttl_seconds)
if max_ttl_seconds > 0
else start_date
)

@jyejare jyejare Apr 16, 2026

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 to @ntkathole's comment here. The compute_non_entity_date_range utility already exists and handles the same defaulting logic (end_date → now, start_date from max TTL, 30-day fallback). Inlining it here diverges from the other offline stores and duplicates code.

Also, max_ttl_seconds is computed twice when start_date is None — once to derive start_date and again for lookback_start_date. Hoisting it to a single computation before both uses would clean this up:

if entity_df is None:
    start_date, end_date = compute_non_entity_date_range(
        feature_views, start_date=start_date, end_date=end_date,
    )
    max_ttl_seconds = max(
        (int(fv.ttl.total_seconds()) for fv in feature_views
         if fv.ttl and isinstance(fv.ttl, timedelta)),
        default=0,
    )
    lookback_start_date = (
        start_date - timedelta(seconds=max_ttl_seconds)
        if max_ttl_seconds > 0 else start_date
    )
    entity_df = pd.DataFrame({"event_timestamp": [end_date]})
    skip_entity_upload = True

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
Comment thread sdk/python/feast/infra/offline_stores/contrib/postgres_offline_store/postgres.py
Comment on lines +204 to 210
if skip_entity_upload:
# LOCF path never uses left_table
left_table_query_string = (
"(SELECT NULL::timestamptz AS event_timestamp LIMIT 0)"
)
elif use_cte:
left_table_query_string = entity_df

@jyejare jyejare Apr 16, 2026

Copy link
Copy Markdown
Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the LOCF path, the synthetic entity_df (single-row DataFrame) is still being created and passed through _get_entity_schema / _get_entity_df_event_timestamp_range, and the left_table_query_string is set to a dummy (SELECT NULL::timestamptz ... LIMIT 0). Since the LOCF template never references left_table_query_string, consider skipping entity_df creation and schema inference entirely when skip_entity_upload=True. This avoids the unnecessary overhead and makes the code intent clearer.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions
@Vperiodt
Merge branch 'feast-dev:master' into patch-query
25bc86d
Vperiodt and others added 5 commits April 16, 2026 19:35
@Vperiodt
address reviews
b6a4703
Signed-off-by: Vanshika Vanshika <[email protected]>

rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED
@Vperiodt
Merge branch 'feast-dev:master' into patch-query
d014b09
@Vperiodt
lint-fix
69565bb
Signed-off-by: Vanshika Vanshika <[email protected]>

rh-pre-commit.version: 2.3.2
rh-pre-commit.check-secrets: ENABLED
@Vperiodt
Merge branch 'feast-dev:master' into patch-query
b5d7c8f
@Vperiodt
Merge branch 'master' into patch-query
4db3b40

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@jyejare jyejare jyejare left review comments
@ntkathole ntkathole ntkathole left review comments
@devin-ai-integration devin-ai-integration[bot] devin-ai-integration[bot] left review comments
+1 more reviewer
@YassinNouh21 YassinNouh21 YassinNouh21 left review comments
Reviewers whose approvals may not affect merge requirements

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

ok-to-test

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

5 participants

@Vperiodt @tokoko @jyejare @ntkathole @YassinNouh21
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.