GeezerVison

The World Wide Web, restored to 800 pixels of glory.

GeezerVison is showing github.com — fix: Issue-3771, fix document error as not shortcut -f option in code. by shuchu · Pull Request #3783 · feast-dev/feast

fix: Issue-3771, fix document error as not shortcut -f option in code. #3783

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed

shuchu wants to merge 69 commits into feast-dev:master from shuchu:issue-3771

Uh oh!

There was an error while loading. Please reload this page.

Closed

fix: Issue-3771, fix document error as not shortcut -f option in code. #3783

Show file tree

Hide file tree

Changes from 1 commit

Commits

Show all changes

69 commits

Select commit Hold shift + click to select a range

49da37c

fix: issue-3771, fix document error as not shortcut -f option in code.

shuchu Oct 9, 2023

018d0ea

fix: Update jackson and remove unnecessary logging (#3809)

achals Oct 20, 2023

ea12133

chore: Add harry to CODEOWNERS (#3808)

achals Oct 20, 2023

f7a98e2

chore: Bump golang.org/x/net from 0.7.0 to 0.17.0 (#3792)

dependabot[bot] Oct 20, 2023

98e0ce8

chore: Bump @babel/traverse from 7.17.10 to 7.23.2 in /sdk/python/fea…

dependabot[bot] Oct 20, 2023

f05a6e7

chore: Bump avro from 1.10.0 to 1.11.3 in /sdk/python/requirements (#…

dependabot[bot] Oct 20, 2023

fb6b807

feat: Support GCS filesystem for bytewax engine (#3774)

sudohainguyen Oct 20, 2023

9b0e5ce

feat: Make bytewax job write as mini-batches (#3777)

sudohainguyen Oct 20, 2023

7376db8

fix: Resolve hbase hotspot issue when materializing (#3790)

sudohainguyen Oct 20, 2023

175d796

fix: Make generated temp table name escaped (#3797)

BearDimonR Oct 20, 2023

b3852bf

fix: Adopt connection pooling for HBase (#3793)

sudohainguyen Oct 21, 2023

417b16b

fix: Redundant feature materialization and premature incremental mate…

james-crabtree-sp Oct 25, 2023

0151961

chore: Bump urllib3 from 1.26.16 to 1.26.18 in /sdk/python/requiremen…

dependabot[bot] Oct 25, 2023

01db8cc

feat: Add get online feature rpc to gprc server (#3815)

sudohainguyen Oct 31, 2023

9583ed6

fix: Add async refresh to prevent synchronous refresh in main thread …

sudohainguyen Oct 31, 2023

25e9775

fix: Bytewax engine create configmap from object (#3821)

sudohainguyen Nov 8, 2023

2151c39

fix: Set upper bound for bigquery client due to its breaking changes

sudohainguyen Nov 4, 2023

5dc0b24

fix: update dependencies versions due to conflicts

sudohainguyen Nov 8, 2023

1f91fc6

chore: generate reqs file from up-to-date deps

sudohainguyen Nov 8, 2023

9cf9d96

feat: Optimize bytewax pod resource with zero-copy

sudohainguyen Nov 4, 2023

5983f40

fix: improve parsing bytewax job status

sudohainguyen Nov 6, 2023

ae1bb8b

fix: make bytewax settings unexposed

sudohainguyen Nov 8, 2023

4337c89

feat: add redis sentinel support

snowron Nov 2, 2023

3387a15

feat: add redis sentinel support

snowron Nov 2, 2023

aad8718

feat: add redis sentinel support format lint

snowron Nov 2, 2023

77d7eb0

chore: Bump pyarrow

dependabot[bot] Nov 9, 2023

052182b

fix: upgrade the pyarrow to latest v14.0.1 for CVE-2023-47248.

shuchu Nov 17, 2023

4f8efa7

chore: Bump follow-redirects from 1.15.0 to 1.15.4 in /sdk/python/fea…

dependabot[bot] Jan 10, 2024

e436f77

chore: Bump follow-redirects from 1.14.7 to 1.15.4 in /ui (#3878)

dependabot[bot] Jan 10, 2024

b26f25a

Update release.yml to latest nodejs

woop Jan 13, 2024

9237361

chore(release): release 0.35.0

feast-ci-bot Jan 13, 2024

8688acd

feat: Add python bytes to array type conversion support proto (#3874)

robhowley Jan 15, 2024

99178f9

chore: Bump jupyter-server from 2.10.1 to 2.11.2 in /sdk/python/requi…

dependabot[bot] Jan 15, 2024

f4a3cb1

chore: Bump jinja2 from 3.1.2 to 3.1.3 in /sdk/python/requirements (#…

dependabot[bot] Jan 16, 2024

ac6529c

Move maintainers to emeritus (#3888)

woop Jan 16, 2024

4c90624

chore: Bump jupyter-lsp from 2.2.0 to 2.2.2 in /sdk/python/requiremen…

dependabot[bot] Jan 19, 2024

8f65fe1

chore: Bump cryptography from 41.0.5 to 41.0.6 in /sdk/python/require…

dependabot[bot] Jan 19, 2024

bdd7dfb

fix: Allow trancated timestamps when converting (#3861)

j1wonpark Jan 19, 2024

0a06a2b

chore: Bump zod from 3.15.1 to 3.22.3 in /sdk/python/feast/ui (#3816)

dependabot[bot] Jan 22, 2024

b80bcd6

chore: Bump google.golang.org/grpc from 1.53.0 to 1.56.3 (#3820)

dependabot[bot] Jan 22, 2024

61fdb00

chore: Bump @babel/traverse from 7.16.10 to 7.23.2 in /ui (#3801)

dependabot[bot] Jan 22, 2024

a75d417

chore: Bump zod from 3.19.1 to 3.22.3 in /ui (#3817)

dependabot[bot] Jan 22, 2024

86d6221

fix: Correct the returning class proto type of StreamFeatureView to S…

shuchu Jan 24, 2024

9a3590e

fix: Verify the existence of Registry tables in snowflake before call…

shuchu Jan 24, 2024

c0d358a

revert: Verify the existence of Registry tables in snowflake… (#3907)

etirelli Jan 25, 2024

8d6bec8

feat: Add support for arrays in snowflake (#3769)

JohnLemmonMedely Jan 25, 2024

ea8ad17

feat: Support s3gov schema by snowflake offline store during material…

alex-vinnik-sp Jan 27, 2024

2f99a61

fix: Create index only if not exists during MySQL online store update…

gee-senbong Jan 30, 2024

987f0fd

fix: Bytewax materialization engine fails when loading feature_store.…

marti-jorda-roca Jan 30, 2024

fa8cfd4

Update maintainers.md (#3918)

woop Jan 30, 2024

936ecfd

docs: Updating maintainers.md (#3919)

franciscojavierarceo Jan 30, 2024

8bce6dc

docs: Updated development guide to include compiling the protos (#3896)

franciscojavierarceo Jan 31, 2024

7153cad

fix: Prevent spamming pull busybox from dockerhub (#3923)

sudohainguyen Feb 2, 2024

bf026a0

chore: Bump jupyterlab from 4.0.8 to 4.0.11 in /sdk/python/requiremen…

dependabot[bot] Feb 4, 2024

f494f02

fix: Remove unnecessary dependency on mysqlclient (#3925)

tokoko Feb 4, 2024

1f3cab8

ci: Extend python base version for test cases (#3929)

sudohainguyen Feb 6, 2024

373e624

feat: Add gRPC Registry Server (#3924)

tokoko Feb 6, 2024

b4aed65

chore: Set upper bound for moto package (#3937)

sudohainguyen Feb 7, 2024

49d2988

fix: Trino as an OfflineStore Access Denied when BasicAuthenticaion (…

Woo-Dong Feb 8, 2024

c16e5af

fix: Typo Correction in Feast UI Readme (#3939)

tsisodia10 Feb 8, 2024

bdce99d

docs: Add ScyllaDB as online store alternative (fixed DCO) (#3944)

zseta Feb 9, 2024

7d75fc5

fix: Add conn.commit() to Postgresonline_write_batch.online_write_bat…

job-almekinders Feb 9, 2024

d3a2a45

fix: Transformation server doesn't generate files from proto (#3902)

tokoko Feb 9, 2024

4e450ad

chore: Bumping fastapi + starlette (#3938)

bushwhackr Feb 10, 2024

dbb59ba

fix: Rewrite Spark materialization engine to use mapInPandas (#3936)

tokoko Feb 13, 2024

5c9f592

chore: Loosen fsspec requirements to allow recent releases (#3922)

cburroughs Feb 14, 2024

ec11a7c

feat: Update the Pydantic from v1 to v2 (#3948)

shuchu Feb 15, 2024

8d45c93

fix: issue-3771, fix document error as not shortcut -f option in code.

shuchu Oct 9, 2023

4016aa4

Merge branch 'issue-3771' of github.com:shuchu/feast into issue-3771

shuchu Feb 16, 2024

File filter

Filter by extension .py (35) .txt (6) No extension (1) All 3 file types selected

Only manifest files

Viewed files

Clear filters

Conversations

Failed to load comments.

Uh oh!

There was an error while loading. Please reload this page.

Jump to

Jump to file

Failed to load files.

Uh oh!

There was an error while loading. Please reload this page.

Diff view

Unified

Split

Hide whitespace

Diff view

Unified

Split

Hide whitespace

Previous commit

Next commit

chore: Bumping fastapi + starlette (#3938)

Loading branch information

Uh oh!

There was an error while loading. Please reload this page.

bushwhackr authored Feb 10, 2024

commit 4e450ad3b1b6d2f66fd87e07805bb57772390142

There are no files selected for viewing

2 changes: 1 addition & 1 deletion 2 Makefile

Show comments

View file

Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -310,7 +310,7 @@ format-python:
		cd ${ROOT_DIR}/sdk/python; python -m black --target-version py38 feast tests

		lint-python:
		cd ${ROOT_DIR}/sdk/python; python -m mypy
		cd ${ROOT_DIR}/sdk/python; python -m mypy --exclude=/tests/ --follow-imports=skip feast
		cd ${ROOT_DIR}/sdk/python; python -m isort feast/ tests/ --check-only
		cd ${ROOT_DIR}/sdk/python; python -m flake8 feast/ tests/
		cd ${ROOT_DIR}/sdk/python; python -m black --check feast tests
Expand Down

19 changes: 9 additions & 10 deletions 19 sdk/python/feast/data_source.py

Show comments

View file

Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -11,7 +11,6 @@
		# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
		# See the License for the specific language governing permissions and
		# limitations under the License.

		import enum
		import warnings
		from abc import ABC, abstractmethod
Expand Down Expand Up		@@ -485,12 +484,12 @@ def to_proto(self) -> DataSourceProto:
		return data_source_proto

		def validate(self, config: RepoConfig):
		pass
		raise NotImplementedError

		def get_table_column_names_and_types(
		self, config: RepoConfig
		) -> Iterable[Tuple[str, str]]:
		pass
		raise NotImplementedError

		@staticmethod
		def source_datatype_to_feast_value_type() -> Callable[[str], ValueType]:
Expand Down Expand Up		@@ -534,12 +533,12 @@ def __init__(
		self.schema = schema

		def validate(self, config: RepoConfig):
		pass
		raise NotImplementedError

		def get_table_column_names_and_types(
		self, config: RepoConfig
		) -> Iterable[Tuple[str, str]]:
		pass
		raise NotImplementedError

		def __eq__(self, other):
		if not isinstance(other, RequestSource):
Expand Down Expand Up		@@ -610,12 +609,12 @@ def source_datatype_to_feast_value_type() -> Callable[[str], ValueType]:
		@typechecked
		class KinesisSource(DataSource):
		def validate(self, config: RepoConfig):
		pass
		raise NotImplementedError

		def get_table_column_names_and_types(
		self, config: RepoConfig
		) -> Iterable[Tuple[str, str]]:
		pass
		raise NotImplementedError

		@staticmethod
		def from_proto(data_source: DataSourceProto):
Expand All		@@ -639,7 +638,7 @@ def from_proto(data_source: DataSourceProto):

		@staticmethod
		def source_datatype_to_feast_value_type() -> Callable[[str], ValueType]:
		pass
		raise NotImplementedError

		def get_table_query_string(self) -> str:
		raise NotImplementedError
Expand Down Expand Up		@@ -772,12 +771,12 @@ def __hash__(self):
		return super().__hash__()

		def validate(self, config: RepoConfig):
		pass
		raise NotImplementedError

		def get_table_column_names_and_types(
		self, config: RepoConfig
		) -> Iterable[Tuple[str, str]]:
		pass
		raise NotImplementedError

		@staticmethod
		def from_proto(data_source: DataSourceProto):
Expand Down

2 changes: 1 addition & 1 deletion 2 sdk/python/feast/feature_service.py

Show comments

View file

Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -56,7 +56,7 @@ def __init__(
		*,
		name: str,
		features: List[Union[FeatureView, OnDemandFeatureView]],
		tags: Dict[str, str] = None,
		tags: Optional[Dict[str, str]] = None,
		description: str = "",
		owner: str = "",
		logging_config: Optional[LoggingConfig] = None,
Expand Down

2 changes: 1 addition & 1 deletion 2 sdk/python/feast/feature_view.py

Show comments

View file

Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -101,7 +101,7 @@ def __init__(
		name: str,
		source: DataSource,
		schema: Optional[List[Field]] = None,
		entities: List[Entity] = None,
		entities: Optional[List[Entity]] = None,
		ttl: Optional[timedelta] = timedelta(days=0),
		online: bool = True,
		description: str = "",
Expand Down

3 changes: 2 additions & 1 deletion 3 sdk/python/feast/importer.py

Show comments

View file

Open in desktop

Original file line number	Diff line number	Diff line change
		@@ -1,4 +1,5 @@
		import importlib
		from typing import Optional

		from feast.errors import (
		FeastClassImportError,
Expand All		@@ -7,7 +8,7 @@
		)


		def import_class(module_name: str, class_name: str, class_type: str = None):
		def import_class(module_name: str, class_name: str, class_type: Optional[str] = None):
		"""
		Dynamically loads and returns a class from a module.

Expand Down

11 changes: 9 additions & 2 deletions 11 sdk/python/feast/infra/contrib/spark_kafka_processor.py

Show comments

View file

Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -5,6 +5,7 @@
		from pyspark.sql import DataFrame, SparkSession
		from pyspark.sql.avro.functions import from_avro
		from pyspark.sql.functions import col, from_json
		from pyspark.sql.streaming import StreamingQuery

		from feast.data_format import AvroFormat, JsonFormat
		from feast.data_source import KafkaSource, PushMode
Expand Down Expand Up		@@ -63,7 +64,13 @@ def __init__(
		self.join_keys = [fs.get_entity(entity).join_key for entity in sfv.entities]
		super().__init__(fs=fs, sfv=sfv, data_source=sfv.stream_source)

		def ingest_stream_feature_view(self, to: PushMode = PushMode.ONLINE) -> None:
		# Type hinting for data_source type.
		# data_source type has been checked to be an instance of KafkaSource.
		self.data_source: KafkaSource = self.data_source # type: ignore

		def ingest_stream_feature_view(
		self, to: PushMode = PushMode.ONLINE
		) -> StreamingQuery:
		ingested_stream_df = self._ingest_stream_data()
		transformed_df = self._construct_transformation_plan(ingested_stream_df)
		online_store_query = self._write_stream_data(transformed_df, to)
Expand Down Expand Up		@@ -122,7 +129,7 @@ def _ingest_stream_data(self) -> StreamTable:
		def _construct_transformation_plan(self, df: StreamTable) -> StreamTable:
		return self.sfv.udf.__call__(df) if self.sfv.udf else df

		def _write_stream_data(self, df: StreamTable, to: PushMode):
		def _write_stream_data(self, df: StreamTable, to: PushMode) -> StreamingQuery:
		# Validation occurs at the fs.write_to_online_store() phase against the stream feature view schema.
		def batch_write(row: DataFrame, batch_id: int):
		rows: pd.DataFrame = row.toPandas()
Expand Down

11 changes: 6 additions & 5 deletions 11 sdk/python/feast/infra/contrib/stream_processor.py

Show comments

View file

Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -3,6 +3,7 @@
		from typing import TYPE_CHECKING, Optional

		from pyspark.sql import DataFrame
		from typing_extensions import TypeAlias

		from feast.data_source import DataSource, PushMode
		from feast.importer import import_class
Expand All		@@ -17,7 +18,7 @@
		}

		# TODO: support more types other than just Spark.
		StreamTable = DataFrame
		StreamTable: TypeAlias = DataFrame


		class ProcessorConfig(FeastConfigBaseModel):
Expand Down Expand Up		@@ -54,28 +55,28 @@ def ingest_stream_feature_view(self, to: PushMode = PushMode.ONLINE) -> None:
		Ingests data from the stream source attached to the stream feature view; transforms the data
		and then persists it to the online store and/or offline store, depending on the 'to' parameter.
		"""
		pass
		raise NotImplementedError

		def _ingest_stream_data(self) -> StreamTable:
		"""
		Ingests data into a StreamTable.
		"""
		pass
		raise NotImplementedError

		def _construct_transformation_plan(self, table: StreamTable) -> StreamTable:
		"""
		Applies transformations on top of StreamTable object. Since stream engines use lazy
		evaluation, the StreamTable will not be materialized until it is actually evaluated.
		For example: df.collect() in spark or tbl.execute() in Flink.
		"""
		pass
		raise NotImplementedError

		def _write_stream_data(self, table: StreamTable, to: PushMode) -> None:
		"""
		Launches a job to persist stream data to the online store and/or offline store, depending
		on the 'to' parameter, and returns a handle for the job.
		"""
		pass
		raise NotImplementedError


		def get_stream_processor_object(
Expand Down

6 changes: 3 additions & 3 deletions 6 sdk/python/feast/infra/offline_stores/contrib/athena_offline_store/athena_source.py

Show comments

View file

Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -297,9 +297,9 @@ class SavedDatasetAthenaStorage(SavedDatasetStorage):
		def __init__(
		self,
		table_ref: str,
		query: str = None,
		database: str = None,
		data_source: str = None,
		query: Optional[str] = None,
		database: Optional[str] = None,
		data_source: Optional[str] = None,
		):
		self.athena_options = AthenaOptions(
		table=table_ref, query=query, database=database, data_source=data_source
Expand Down

2 changes: 1 addition & 1 deletion 2 sdk/python/feast/infra/offline_stores/contrib/athena_offline_store/tests/data_source.py

Show comments

View file

Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -51,7 +51,7 @@ def create_data_source(
		suffix: Optional[str] = None,
		timestamp_field="ts",
		created_timestamp_column="created_ts",
		field_mapping: Dict[str, str] = None,
		field_mapping: Optional[Dict[str, str]] = None,
		) -> DataSource:

		table_name = destination_name
Expand Down

8 changes: 4 additions & 4 deletions 8 sdk/python/feast/infra/offline_stores/contrib/mssql_offline_store/tests/data_source.py

Show comments

View file

Open in desktop

Original file line number	Diff line number	Diff line change
		@@ -1,4 +1,4 @@
		from typing import Dict, List
		from typing import Dict, List, Optional

		import pandas as pd
		import pytest
Expand Down Expand Up		@@ -66,7 +66,7 @@ def create_data_source(
		destination_name: str,
		timestamp_field="ts",
		created_timestamp_column="created_ts",
		field_mapping: Dict[str, str] = None,
		field_mapping: Optional[Dict[str, str]] = None,
		**kwargs,
		) -> DataSource:
		# Make sure the field mapping is correct and convert the datetime datasources.
Expand Down Expand Up		@@ -99,10 +99,10 @@ def create_data_source(
		)

		def create_saved_dataset_destination(self) -> SavedDatasetStorage:
		pass
		raise NotImplementedError

		def get_prefixed_table_name(self, destination_name: str) -> str:
		return f"{self.project_name}_{destination_name}"

		def teardown(self):
		pass
		raise NotImplementedError

2 changes: 1 addition & 1 deletion 2 sdk/python/feast/infra/offline_stores/contrib/postgres_offline_store/tests/data_source.py

Show comments

View file

Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -85,7 +85,7 @@ def create_data_source(
		suffix: Optional[str] = None,
		timestamp_field="ts",
		created_timestamp_column="created_ts",
		field_mapping: Dict[str, str] = None,
		field_mapping: Optional[Dict[str, str]] = None,
		) -> DataSource:
		destination_name = self.get_prefixed_table_name(destination_name)

Expand Down

4 changes: 2 additions & 2 deletions 4 sdk/python/feast/infra/offline_stores/contrib/spark_offline_store/tests/data_source.py

Show comments

View file

Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -2,7 +2,7 @@
		import shutil
		import tempfile
		import uuid
		from typing import Dict, List
		from typing import Dict, List, Optional

		import pandas as pd
		from pyspark import SparkConf
Expand Down Expand Up		@@ -70,7 +70,7 @@ def create_data_source(
		destination_name: str,
		timestamp_field="ts",
		created_timestamp_column="created_ts",
		field_mapping: Dict[str, str] = None,
		field_mapping: Optional[Dict[str, str]] = None,
		**kwargs,
		) -> DataSource:
		if timestamp_field in df:
Expand Down

2 changes: 1 addition & 1 deletion 2 sdk/python/feast/infra/offline_stores/file_source.py

Show comments

View file

Open in desktop

Original file line number	Diff line number	Diff line change
Expand Up		@@ -183,7 +183,7 @@ def create_filesystem_and_path(
		return None, path

		def get_table_query_string(self) -> str:
		pass
		raise NotImplementedError


		class FileOptions:
Expand Down

Oops, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

Add this suggestion to a batch that can be applied as a single commit. This suggestion is invalid because no changes were made to the code. Suggestions cannot be applied while the pull request is closed. Suggestions cannot be applied while viewing a subset of changes. Only one suggestion per line can be applied in a batch. Add this suggestion to a batch that can be applied as a single commit. Applying suggestions on deleted lines is not supported. You must change the existing code in this line in order to create a valid suggestion. Outdated suggestions cannot be applied. This suggestion has been applied or marked resolved. Suggestions cannot be applied from pending reviews. Suggestions cannot be applied on multi-line comments. Suggestions cannot be applied while the pull request is queued to merge. Suggestion cannot be applied right now. Please check back later.

You are visitor 01402520 — GeezerVison does not store the pages you look up.
Best viewed with Netscape Navigator 4.7 or Internet Explorer 6 at 800×600. No cookies required.