GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — feat: Adjust ray offline store to support abfs(s) ADLS Azure Storage by jbauer12 · Pull Request #5911 · feast-dev/feast

feat: Adjust ray offline store to support abfs(s) ADLS Azure Storage #5911

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

ntkathole merged 8 commits into feast-dev:master from jbauer12:ray_azure_integration

Jan 30, 2026

Uh oh!

There was an error while loading. Please reload this page.

Merged

feat: Adjust ray offline store to support abfs(s) ADLS Azure Storage #5911

Show file tree

Hide file tree

Changes from all commits

Commits

Show all changes

8 commits

Select commit Hold shift + click to select a range

d07896c

feat: adjust ray offline store to support abfs(s) ADLS Azure Storage

p3s-jbauer Jan 27, 2026

3de9dcd

Delete test_registry file.

p3s-jbauer Jan 27, 2026

a5dc522

Feedback from PR Review.

p3s-jbauer Jan 27, 2026

9c140c7

Added additional comment about env var.

p3s-jbauer Jan 27, 2026

f57b172

Fixed potential issue from PR.

p3s-jbauer Jan 28, 2026

ce98866

Fixed found issue by Devin from PR.

p3s-jbauer Jan 29, 2026

7f1d005

Apply formatting

p3s-jbauer Jan 29, 2026

74967ba

Fixed found issue by Devin from PR.

p3s-jbauer Jan 29, 2026

File filter

Filter by extension .md (1) .py (2) All 2 file types selected

Viewed files

Clear filters

Conversations

Failed to load comments.

Uh oh!

There was an error while loading. Please reload this page.

Jump to

Jump to file

Failed to load files.

Uh oh!

There was an error while loading. Please reload this page.

Diff view

Unified

Split

Hide whitespace

Diff view

Unified

Split

Hide whitespace

There are no files selected for viewing

7 changes: 6 additions & 1 deletion 7 docs/reference/offline-stores/ray.md

Show comments

View file

Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -27,7 +27,7 @@ The template provides a complete working example with sample datasets and demons
		The Ray offline store provides:
		- Ray-based data reading from file sources (Parquet, CSV, etc.)
		- Support for local, remote, and KubeRay (Kubernetes-managed) clusters
		- Integration with various storage backends (local files, S3, GCS, HDFS)
		- Integration with various storage backends (local files, S3, GCS, HDFS, Azure Blob)
		- Efficient data filtering and column selection
		- Timestamp-based data processing with timezone awareness
		- Enterprise-ready KubeRay cluster support via CodeFlare SDK
Expand Down Expand Up		@@ -463,6 +463,11 @@ job.persist(gcs_storage, allow_overwrite=True)
		# HDFS
		hdfs_storage = SavedDatasetFileStorage(path="hdfs://namenode:8020/datasets/driver_features.parquet")
		job.persist(hdfs_storage, allow_overwrite=True)

		# Azure Blob Storage / Azure Data Lake Storage Gen2
		# By setting AZURE_STORAGE_ANON=False it uses DefaultAzureCredential
		az_storage = SavedDatasetFileStorage(path="abfss://container@stc_account.dfs.core.windows.net/datasets/driver_features.parquet")
		job.persist(az_storage, allow_overwrite=True)
		```

		### Using Ray Cluster
Expand Down

5 changes: 4 additions & 1 deletion 5 sdk/python/feast/infra/compute_engines/ray/job.py

Show comments

View file

Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -18,6 +18,9 @@
		from feast.infra.compute_engines.dag.model import DAGFormat
		from feast.infra.compute_engines.dag.plan import ExecutionPlan
		from feast.infra.compute_engines.dag.value import DAGValue
		from feast.infra.offline_stores.contrib.ray_offline_store.ray import (
		REMOTE_STORAGE_SCHEMES,
		)
		from feast.infra.offline_stores.file_source import SavedDatasetFileStorage
		from feast.infra.offline_stores.offline_store import RetrievalJob, RetrievalMetadata
		from feast.infra.ray_initializer import get_ray_wrapper
Expand Down Expand Up		@@ -205,7 +208,7 @@ def persist(
		destination_path = storage.file_options.uri

		# Check if destination already exists
		if not destination_path.startswith(("s3://", "gs://", "hdfs://")):
		if not destination_path.startswith(REMOTE_STORAGE_SCHEMES):
		import os

		if not allow_overwrite and os.path.exists(destination_path):
Expand Down

27 changes: 20 additions & 7 deletions 27 sdk/python/feast/infra/offline_stores/contrib/ray_offline_store/ray.py

Show comments

View file

Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -62,6 +62,12 @@
		from feast.utils import _get_column_names, make_df_tzaware, make_tzaware

		logger = logging.getLogger(__name__)
		# Remote storage URI schemes supported by the Ray offline store
		# S3: Amazon S3
		# GCS: Google Cloud Storage
		# HDFS: Hadoop Distributed File System
		# Azure: Azure Storage Gen2
		REMOTE_STORAGE_SCHEMES = ("s3://", "gs://", "hdfs://", "abfs://", "abfss://")


		def _get_data_schema_info(
Expand Down Expand Up		@@ -1160,13 +1166,13 @@ def persist(
		f"Ray offline store only supports SavedDatasetFileStorage, got {type(storage)}"
		)
		destination_path = storage.file_options.uri
		if not destination_path.startswith(("s3://", "gs://", "hdfs://")):
		if not destination_path.startswith(REMOTE_STORAGE_SCHEMES):
		if not allow_overwrite and os.path.exists(destination_path):
		raise SavedDatasetLocationAlreadyExists(location=destination_path)
		try:
		ray_ds = self._get_ray_dataset()

		if not destination_path.startswith(("s3://", "gs://", "hdfs://")):
		if not destination_path.startswith(REMOTE_STORAGE_SCHEMES):
		os.makedirs(os.path.dirname(destination_path), exist_ok=True)

		ray_ds.write_parquet(destination_path)
Expand Down Expand Up		@@ -1956,12 +1962,19 @@ def normalize_timestamps(batch: pd.DataFrame) -> pd.DataFrame:
		filesystem, resolved_path = FileSource.create_filesystem_and_path(
		absolute_path, destination.s3_endpoint_override
		)
		path_obj = Path(resolved_path)
		if path_obj.suffix == ".parquet":
		path_obj = path_obj.with_suffix("")
		if not absolute_path.startswith(("s3://", "gs://")):
		if absolute_path.startswith(REMOTE_STORAGE_SCHEMES):
		write_path = (
		absolute_path[:-8]
		if absolute_path.endswith(".parquet")
		else absolute_path
		)
		else:
		path_obj = Path(resolved_path)
		if path_obj.suffix == ".parquet":
		path_obj = path_obj.with_suffix("")
		path_obj.mkdir(parents=True, exist_ok=True)
		ds.write_parquet(str(path_obj))
		write_path = str(path_obj)
		ds.write_parquet(write_path)
		except Exception as e:
		raise RuntimeError(f"Failed to write logged features: {e}")

Expand Down

Oops, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.

You are visitor 01402503 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.