GeezerVisonThe World Wide Web, restored to 800 pixels of glory. | |||||||||||||||||||||||||||||||||||||||||||
| Home|About|Gaming cabinet|Archives|How it works|Starter sites | |||||||||||||||||||||||||||||||||||||||||||
GeezerVison is showing github.com — fix: Preserve event-time ordering within Redis online_write_batch by adarshsm · Pull Request #6656 · feast-dev/feast Conversation
|
| Test | Before fix | After fix |
|---|---|---|
keeps_latest_event_time_within_batch[descending] |
fails | passes |
keeps_latest_event_time_within_batch[unordered] |
fails | passes |
keeps_latest_event_time_within_batch[ascending] |
passes | passes |
async_keeps_latest_event_time_within_batch |
fails | passes |
still_skips_rows_older_than_stored_value |
passes | passes |
The two that pass before the fix are the load-bearing ones — they're there to show the
change repairs the broken orderings without disturbing the ascending case or the existing
staleness guard.
The pre-fix failure is assert b'\x18\n' == b'\x18\x1e', i.e. int32_val=10 where 30 was
expected — the reporter's "returns 10 instead of 30", reproduced deterministically.
test_redis.py goes 18 → 23 passing. ruff check, ruff format --check and
mypy feast/infra/online_stores/redis.py are all clean.
I ran unit tests only, not the integration suite — it needs cloud credentials I don't have.
Happy to add an integration test if you'd like one for this path.
One small correction to the issue for the record: it describes the code as skipping
"records with timestamps older than what's already been processed in the current batch."
Nothing intra-batch is tracked today, which is exactly why the bug exists — but the symptom
as reported is accurate.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
The reason will be displayed to describe this comment to others. Learn more.
The per-key running maximum is applied consistently in both sync and async write paths, and the added cases cover descending, ascending, unordered, and pre-existing stored timestamps. I found no blocking issue in the remote diff.
Sorry, something went wrong.
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@ ## master #6656 +/- ## ========================================== + Coverage 45.98% 46.06% +0.07% ========================================== Files 414 414 Lines 50037 50045 +8 Branches 7147 7147 ========================================== + Hits 23012 23051 +39 + Misses 25413 25380 -33 - Partials 1612 1614 +2
... and 1 file with indirect coverage changes Continue to review full report in Codecov by Harness.
🚀 New features to boost your workflow:
|
Sorry, something went wrong.
There was an error while loading. Please reload this page.
online_write_batch reads every existing event timestamp in a single pipeline before it queues any write, so rows in the same batch that share an entity key all compared against the same pre-batch snapshot. None of them could see the others, so every row passed the staleness check and Redis applied the queued writes in list order — the last row won regardless of its event time. A batch of [t3, t2, t1] for one entity key left the value belonging to t1 in the store. Track the latest event timestamp queued so far for each entity key and compare each row against whichever is newer: the stored timestamp, or an earlier row of the same batch. Batches that already arrived in ascending order keep their existing behaviour, and rows older than the value already stored are still skipped. online_write_batch_async had the identical defect and gets the same fix. The skip_dedup fast path is deliberately left alone, since it documents that out-of-order writes are not a concern when it is enabled. Fixes feast-dev#5163 Signed-off-by: adarshsm <[email protected]>
a1ebc75 to
edc9781
Compare
There was an error while loading. Please reload this page.
There was an error while loading. Please reload this page.
Successfully merging this pull request may close these issues.
Redis online store fails to maintain event-time ordering in batch writes
There was an error while loading. Please reload this page.