GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — feat: Implemented Tiling Support for Time-Windowed Aggregations by ntkathole · Pull Request #5724 · feast-dev/feast

Conversation

ntkathole commented Nov 16, 2025

Copy link

Copy Markdown

Member

What this PR does / why we need it:

This PR implements tiling for computing time-windowed aggregations efficiently by pre-aggregating data into small, time-bucketed units (tiles) that can be reused across multiple queries. Instead of scanning all raw events for each window, it:

Pre-aggregates data into small time buckets (tiles)
Computes windowed aggregations by subtracting tiles: window_at_T = tile_at_T - tile_at_(T-window_size)
Stores intermediate representations (IRs) for holistic aggregations (avg, std, var) to enable correct merging
Integrates tiling into Spark and Ray compute engines for StreamFeatureView

Sawtooth Window Tiling

Tile Generation: Events bucketed into hop-sized intervals (e.g., 1-minute tiles)
Cumulative Aggregation: Each tile contains cumulative sum/count from start
Window Computation: Windowed aggregation = tile_at_T - tile_at_(T-window_size)
IR Handling: For holistic aggs, IRs subtracted separately, then final value recomputed

Enable Tiling in StreamFeatureView

sales_features = StreamFeatureView(
    name="sales_features",
    entities=[customer],
    source=kafka_source,
    aggregations=[
        Aggregation(
            column="amount",
            function="avg",
            time_window=timedelta(minutes=5),
        ),
        Aggregation(
            column="amount",
            function="sum",
            time_window=timedelta(minutes=5),
        ),
    ],
    timestamp_field="event_timestamp",
    enable_tiling=True,                      # Enable tiling optimization
    tiling_hop_size=timedelta(minutes=1),    # Generate tiles every 1 minute
)

Architecture

┌─────────────────────────────────────────────────────────────┐
│  StreamFeatureView (enable_tiling=True)                     │
└────────────────────────┬────────────────────────────────────┘
                         │
         ┌───────────────┴───────────────┐
         │                               │
    ┌────▼────┐                    ┌────▼────┐
    │  Spark  │                    │   Ray   │
    │  Node   │                    │  Node   │
    └────┬────┘                    └────┬────┘
         │                               │
         │  Convert to pandas            │
         ▼                               ▼
┌─────────────────────────────────────────────────────────────┐
│  Tiling Logic.                                                        │
│  ┌──────────────────────────────────────────────────────┐   │
│  │  orchestrator.apply_sawtooth_window_tiling()         │   │
│  │  - Group by entity + hop interval                    │   │
│  │  - Compute cumulative aggregations (tiles)           │   │
│  │  - Output: Cumulative tiles at each hop              │   │
│  └──────────────────────┬───────────────────────────────┘   │
│                         │                                    │
│  ┌──────────────────────▼───────────────────────────────┐   │
│  │  tile_subtraction.convert_cumulative_to_windowed()   │   │
│  │  - Subtract tiles: window = tile_T - tile_(T-W)      │   │
│  │  - Recompute holistic aggs from IRs                  │   │
│  │  - Output: Windowed aggregations                     │   │
│  └──────────────────────┬───────────────────────────────┘   │
│                         │                                    │
│  ┌──────────────────────▼───────────────────────────────┐   │
│  │  tile_subtraction.deduplicate_keep_latest()          │   │
│  │  - Keep latest window per entity                     │   │
│  └──────────────────────────────────────────────────────┘   │
└─────────────────────────┬───────────────────────────────────┘
                          │  Convert back to engine format
                          ▼
                  ┌───────────────┐
                  │ Online Store  │
                  │               │
                  └───────────────┘

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

franciscojavierarceo reacted with hooray emoji

franciscojavierarceo reacted with rocket emoji

All reactions

🎉 1 reaction
🚀 1 reaction

ntkathole requested a review from a team as a code owner

November 16, 2025 11:17

ntkathole force-pushed the aggregation_tiling branch from 6b05deb to 570debe Compare

November 16, 2025 11:33

franciscojavierarceo requested a review from Copilot

November 17, 2025 15:34

Copilot started reviewing on behalf of franciscojavierarceo

November 17, 2025 15:34

View session

Copilot finished reviewing on behalf of franciscojavierarceo

November 17, 2025 15:37

ntkathole force-pushed the aggregation_tiling branch from 570debe to dd83c13 Compare

November 17, 2025 15:41

Copilot AI reviewed

Nov 17, 2025

View reviewed changes

Copilot AI left a comment

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements tiling support for efficient time-windowed aggregations in Feast's streaming feature views. The implementation uses a sawtooth window tiling algorithm with intermediate representations (IRs) to enable correct merging of holistic aggregations (avg, std, var) while providing performance benefits for streaming scenarios.

Key Changes:

Adds core tiling logic with IR-based aggregation in new infra/tiling/ module
Extends StreamFeatureView with enable_tiling and tiling_hop_size configuration options
Integrates tiling into Spark and Ray compute engines with pandas-based processing
Updates protobuf definitions and documentation

Reviewed Changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 23 comments.

Show a summary per file

File	Description
`sdk/python/feast/infra/tiling/base.py`	Defines IR metadata structures for algebraic and holistic aggregations
`sdk/python/feast/infra/tiling/orchestrator.py`	Implements cumulative tile generation using sawtooth window algorithm
`sdk/python/feast/infra/tiling/tile_subtraction.py`	Converts cumulative tiles to windowed aggregations via tile subtraction
`sdk/python/feast/infra/tiling/__init__.py`	Exports tiling module public API
`sdk/python/feast/stream_feature_view.py`	Adds tiling configuration to StreamFeatureView class
`protos/feast/core/StreamFeatureView.proto`	Adds protobuf fields for tiling configuration
`sdk/python/feast/protos/feast/core/StreamFeatureView_pb2.py`	Generated protobuf Python code for tiling fields
`sdk/python/feast/protos/feast/core/StreamFeatureView_pb2.pyi`	Generated protobuf type stubs for tiling fields
`sdk/python/feast/infra/compute_engines/spark/nodes.py`	Implements tiling execution path for Spark engine
`sdk/python/feast/infra/compute_engines/spark/feature_builder.py`	Passes tiling config to Spark aggregation nodes
`sdk/python/feast/infra/compute_engines/ray/nodes.py`	Implements tiling execution path for Ray engine
`sdk/python/feast/infra/compute_engines/ray/feature_builder.py`	Passes tiling config to Ray aggregation nodes
`sdk/python/feast/utils.py`	Extracts input columns from aggregations for feature views
`sdk/python/tests/unit/infra/compute_engines/spark/test_nodes.py`	Updates test to pass required spark_session parameter
`docs/getting-started/concepts/tiling.md`	Comprehensive documentation on tiling concepts and usage
`docs/getting-started/concepts/stream-feature-view.md`	Adds reference to tiling documentation
`docs/getting-started/concepts/README.md`	Adds tiling.md to concepts index

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

There was an error while loading. Please reload this page.

haoxu0 reviewed

Nov 20, 2025

View reviewed changes

Comment thread

sdk/python/feast/aggregation/tiling/__init__.py

		@@ -0,0 +1,27 @@
		"""
		Tiling for efficient time-windowed aggregations.

haoxu0 Nov 20, 2025

Copy link

Copy Markdown

Collaborator

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Really cool. Is there a way to merge with the Aggregation interface? Or something like
aggregation/

Aggregation -- Base aggregation interface
tiling/

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

franciscojavierarceo reacted with heart emoji

ntkathole reacted with eyes emoji

All reactions

❤️ 1 reaction
👀 1 reaction

ntkathole Nov 21, 2025

Copy link

Copy Markdown

Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea, this make sense. Changed now to:

feast/
  - aggregation/
    - __init__.py (Aggregation class)
    - tiling/

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

franciscojavierarceo reacted with hooray emoji

All reactions

🎉 1 reaction

ntkathole force-pushed the aggregation_tiling branch 2 times, most recently from 7189d22 to 839c7a4 Compare

November 21, 2025 15:51

ntkathole added 2 commits

November 21, 2025 21:24


          feat: Implemented tiling support

0df22f3

Signed-off-by: ntkathole <[email protected]>


          chore: Created Aggregation interface

712d5f9

Signed-off-by: ntkathole <[email protected]>

ntkathole force-pushed the aggregation_tiling branch from 839c7a4 to 712d5f9 Compare

November 21, 2025 15:55

ntkathole added the ok-to-test label

Nov 23, 2025

ntkathole requested review from franciscojavierarceo and haoxu0

November 24, 2025 12:53

franciscojavierarceo approved these changes

Nov 28, 2025

View reviewed changes

franciscojavierarceo left a comment

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Happy Thanksgiving! 🚀🚀🚀

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions


          Merge branch 'master' into aggregation_tiling

49c2792

franciscojavierarceo merged commit 7a99166 into feast-dev:master

Nov 28, 2025

19 checks passed

Uh oh!

There was an error while loading. Please reload this page.

jfw-ppi pushed a commit to jfw-ppi/feast that referenced this pull request

Nov 30, 2025


          feat: Implemented Tiling Support for Time-Windowed Aggregations (feas…

c980db9

…t-dev#5724)

jfw-ppi pushed a commit to jfw-ppi/feast that referenced this pull request

Nov 30, 2025


          feat: Implemented Tiling Support for Time-Windowed Aggregations (feas…

04dc34f

…t-dev#5724)

Signed-off-by: Jacob Weinhold <[email protected]>

franciscojavierarceo pushed a commit that referenced this pull request

Dec 16, 2025


          chore(release): release 0.58.0

b601ef2

# [0.58.0](v0.57.0...v0.58.0) (2025-12-16)

### Bug Fixes

* Add java proto ([#5719](#5719)) ([fc3ea20](fc3ea20))
* Add possibility to force full features names for materialize ops ([#5728](#5728)) ([55c9c36](55c9c36))
* Fixed file registry cache sync ([09505d4](09505d4))
* Handle hyphon in sqlite project name ([#5575](#5575)) ([#5749](#5749)) ([b8346ff](b8346ff))
* Pinned substrait to fix protobuf issue ([d0ef4da](d0ef4da))
* Set TLS certificate annotation only on gRPC service ([#5715](#5715)) ([75d13db](75d13db))
* SQLite online store deletes tables from other projects in shared registry scenarios ([#5766](#5766)) ([fabce76](fabce76))
* Validate not existing entity join keys for preventing panic ([0b93559](0b93559))

### Features

* Add annotations for pod templates ([534e647](534e647))
* Add Pytorch template ([#5780](#5780)) ([6afd353](6afd353))
* Add support for extra options for stream source ([#5618](#5618)) ([18956c2](18956c2))
* Added matched_tag field search api results with fuzzy search capabilities ([#5769](#5769)) ([4a9ffae](4a9ffae))
* Added support for enabling metrics in Feast Operator ([#5317](#5317)) ([#5748](#5748)) ([a8498c2](a8498c2))
* Configure CacheTTLSecondscache,CacheMode for file-based registry in Feast Operator([#5708](#5708)) ([#5744](#5744)) ([f25f83b](f25f83b))
* Implemented Tiling Support for Time-Windowed Aggregations ([#5724](#5724)) ([7a99166](7a99166))
* Offline Store historical features retrieval based on datetime range for spark ([#5720](#5720)) ([27ec8ec](27ec8ec))
* Offline Store historical features retrieval based on datetime range in dask ([#5717](#5717)) ([a16582a](a16582a))
* Production ready feast operator with v1 apiversion ([#5771](#5771)) ([49359c6](49359c6))
* Support for Map value data type ([#5768](#5768)) ([#5772](#5772)) ([b99a8a9](b99a8a9))

antznette1 pushed a commit to antznette1/feast that referenced this pull request

Jan 3, 2026


          chore(release): release 0.58.0

a78a5de

# [0.58.0](feast-dev/feast@v0.57.0...v0.58.0) (2025-12-16)

### Bug Fixes

* Add java proto ([feast-dev#5719](feast-dev#5719)) ([fc3ea20](feast-dev@fc3ea20))
* Add possibility to force full features names for materialize ops ([feast-dev#5728](feast-dev#5728)) ([55c9c36](feast-dev@55c9c36))
* Fixed file registry cache sync ([09505d4](feast-dev@09505d4))
* Handle hyphon in sqlite project name ([feast-dev#5575](feast-dev#5575)) ([feast-dev#5749](feast-dev#5749)) ([b8346ff](feast-dev@b8346ff))
* Pinned substrait to fix protobuf issue ([d0ef4da](feast-dev@d0ef4da))
* Set TLS certificate annotation only on gRPC service ([feast-dev#5715](feast-dev#5715)) ([75d13db](feast-dev@75d13db))
* SQLite online store deletes tables from other projects in shared registry scenarios ([feast-dev#5766](feast-dev#5766)) ([fabce76](feast-dev@fabce76))
* Validate not existing entity join keys for preventing panic ([0b93559](feast-dev@0b93559))

### Features

* Add annotations for pod templates ([534e647](feast-dev@534e647))
* Add Pytorch template ([feast-dev#5780](feast-dev#5780)) ([6afd353](feast-dev@6afd353))
* Add support for extra options for stream source ([feast-dev#5618](feast-dev#5618)) ([18956c2](feast-dev@18956c2))
* Added matched_tag field search api results with fuzzy search capabilities ([feast-dev#5769](feast-dev#5769)) ([4a9ffae](feast-dev@4a9ffae))
* Added support for enabling metrics in Feast Operator ([feast-dev#5317](feast-dev#5317)) ([feast-dev#5748](feast-dev#5748)) ([a8498c2](feast-dev@a8498c2))
* Configure CacheTTLSecondscache,CacheMode for file-based registry in Feast Operator([feast-dev#5708](feast-dev#5708)) ([feast-dev#5744](feast-dev#5744)) ([f25f83b](feast-dev@f25f83b))
* Implemented Tiling Support for Time-Windowed Aggregations ([feast-dev#5724](feast-dev#5724)) ([7a99166](feast-dev@7a99166))
* Offline Store historical features retrieval based on datetime range for spark ([feast-dev#5720](feast-dev#5720)) ([27ec8ec](feast-dev@27ec8ec))
* Offline Store historical features retrieval based on datetime range in dask ([feast-dev#5717](feast-dev#5717)) ([a16582a](feast-dev@a16582a))
* Production ready feast operator with v1 apiversion ([feast-dev#5771](feast-dev#5771)) ([49359c6](feast-dev@49359c6))
* Support for Map value data type ([feast-dev#5768](feast-dev#5768)) ([feast-dev#5772](feast-dev#5772)) ([b99a8a9](feast-dev@b99a8a9))

Signed-off-by: Anthonette Adanyin <[email protected]>

franciscojavierarceo pushed a commit that referenced this pull request

Jan 5, 2026


          chore(release): release 0.58.0

807e80e

# [0.58.0](v0.57.0...v0.58.0) (2025-12-16)

### Bug Fixes

* Add java proto ([#5719](#5719)) ([fc3ea20](fc3ea20))
* Add possibility to force full features names for materialize ops ([#5728](#5728)) ([55c9c36](55c9c36))
* Fixed file registry cache sync ([09505d4](09505d4))
* Handle hyphon in sqlite project name ([#5575](#5575)) ([#5749](#5749)) ([b8346ff](b8346ff))
* Pinned substrait to fix protobuf issue ([d0ef4da](d0ef4da))
* Set TLS certificate annotation only on gRPC service ([#5715](#5715)) ([75d13db](75d13db))
* SQLite online store deletes tables from other projects in shared registry scenarios ([#5766](#5766)) ([fabce76](fabce76))
* Validate not existing entity join keys for preventing panic ([0b93559](0b93559))

### Features

* Add annotations for pod templates ([534e647](534e647))
* Add Pytorch template ([#5780](#5780)) ([6afd353](6afd353))
* Add support for extra options for stream source ([#5618](#5618)) ([18956c2](18956c2))
* Added matched_tag field search api results with fuzzy search capabilities ([#5769](#5769)) ([4a9ffae](4a9ffae))
* Added support for enabling metrics in Feast Operator ([#5317](#5317)) ([#5748](#5748)) ([a8498c2](a8498c2))
* Configure CacheTTLSecondscache,CacheMode for file-based registry in Feast Operator([#5708](#5708)) ([#5744](#5744)) ([f25f83b](f25f83b))
* Implemented Tiling Support for Time-Windowed Aggregations ([#5724](#5724)) ([7a99166](7a99166))
* Offline Store historical features retrieval based on datetime range for spark ([#5720](#5720)) ([27ec8ec](27ec8ec))
* Offline Store historical features retrieval based on datetime range in dask ([#5717](#5717)) ([a16582a](a16582a))
* Production ready feast operator with v1 apiversion ([#5771](#5771)) ([49359c6](49359c6))
* Support for Map value data type ([#5768](#5768)) ([#5772](#5772)) ([b99a8a9](b99a8a9))

Signed-off-by: Francisco Javier Arceo <[email protected]>

Reviewers

Copilot code review Copilot

Copilot review effort

Applies to this pull request for everyone.Learn more about Copilot code review.

Copilot left review comments

franciscojavierarceo

franciscojavierarceo approved these changes

haoxu0

Awaiting requested review from haoxu0

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

ok-to-test

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Uh oh!

There was an error while loading. Please reload this page.

4 participants

Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.