GV

GeezerVison

The World Wide Web, restored to 800 pixels of glory.

Home|About|Gaming cabinet|Archives|How it works|Starter sites

On this site

  • Sign in
  • ActionsCI/CD and automation
  • CodespacesCloud development environments
  • All Features
  • Documentation
  • Training
  • Certifications
  • Build with Copilot
  • Customize Copilot
  • Integrations
  • Trending repos
  • GitHub Sponsors
  • Developer articles
  • Enterprises

This view

  • Original page
  • New address

GeezerVison is showing github.com — fix(bigquery): Enable list inference for parquet loads in offline_write_batch by Jwrede · Pull Request #6365 · feast-dev/feast

Conversation

@Jwrede

@Jwrede Jwrede commented May 3, 2026

Copy link
Copy Markdown
Contributor

Summary

  • BigQuery's parquet loader drops array/list column data (writes empty []) when enable_list_inference is not set on ParquetOptions
  • Added parquet_options.enable_list_inference = True to the LoadJobConfig in offline_write_batch so list-typed features (e.g. STRING_LIST, INT32_LIST) are preserved when pushed via PushMode.OFFLINE or PushMode.ONLINE_AND_OFFLINE
  • Added a unit test verifying the LoadJobConfig has the correct parquet_options

Fixes

Fixes #5845

Test plan

  • New unit test test_offline_write_batch_enables_list_inference passes
  • All existing BigQuery unit tests pass (12/12)
  • Manual verification with a BigQuery table containing list-typed columns (requires GCP access)

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.
All reactions
@Jwrede
Jwrede requested review from a team and sudohainguyen as code owners May 3, 2026 17:23
@Jwrede Jwrede mentioned this pull request May 13, 2026
feat(bigquery): Support DATE-type event timestamp columns #6362
Merged
@ntkathole
ntkathole force-pushed the fix/bq-list-inference branch from 84cde71 to 899ca66 Compare May 14, 2026 06:55
ntkathole
ntkathole approved these changes May 14, 2026
View reviewed changes
@ntkathole ntkathole added the ok-to-test label May 14, 2026
@Jwrede @ntkathole
fix(bigquery): Enable list inference for parquet loads in offline_wri…
fe19c06 
…te_batch

When pushing features with array/list types (e.g. STRING_LIST) to
BigQuery via offline_write_batch, the data arrives as empty arrays
because BigQuery's parquet loader does not infer list structure by
default. Set parquet_options.enable_list_inference = True on the
LoadJobConfig so array columns are written correctly.

Fixes feast-dev#5845

Signed-off-by: Jonathan Wrede <[email protected]>
@ntkathole
ntkathole force-pushed the fix/bq-list-inference branch from 899ca66 to fe19c06 Compare May 14, 2026 07:35
@ntkathole
ntkathole merged commit 9243497 into feast-dev:master May 14, 2026
17 of 25 checks passed

Uh oh!

There was an error while loading. Please reload this page.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ntkathole ntkathole ntkathole approved these changes
@sudohainguyen sudohainguyen Awaiting requested review from sudohainguyen sudohainguyen is a code owner

Uh oh!

There was an error while loading. Please reload this page.

Assignees

No one assigned

Labels

ok-to-test

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

BigQuery offline store loses array data when pushing features with list types

Uh oh!

There was an error while loading. Please reload this page.

2 participants

@Jwrede @ntkathole
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.