GeezerVison

ntkathole requested a review from a team as a code owner

January 21, 2026 15:19

ntkathole marked this pull request as draft

January 21, 2026 15:19

franciscojavierarceo reviewed

View reviewed changes

Comment thread

docs/reference/openlineage.md


		## Lineage Visualization

		Use [Marquez](https://marquezproject.ai/) to visualize your Feast lineage:

franciscojavierarceo Jan 21, 2026

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh this is a weird name for openlineage's visualization but this is the name of it. leaving a comment for future

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

franciscojavierarceo reviewed

View reviewed changes

Comment thread

sdk/python/feast/openlineage/emitter.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

franciscojavierarceo reviewed

View reviewed changes

Comment thread

sdk/python/feast/openlineage/emitter.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

franciscojavierarceo reviewed

View reviewed changes

Comment thread

sdk/python/feast/openlineage/emitter.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

franciscojavierarceo reviewed

View reviewed changes

Comment thread

sdk/python/feast/openlineage/emitter.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

franciscojavierarceo reviewed

View reviewed changes

Comment thread

sdk/python/feast/feature_store.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

ntkathole force-pushed the openlineage branch 3 times, most recently from 4eef1b9 to 73dafb6 Compare

January 24, 2026 10:07

ntkathole marked this pull request as ready for review

January 24, 2026 10:09

ntkathole added the ok-to-test label

Jan 24, 2026

ntkathole force-pushed the openlineage branch 3 times, most recently from d87c190 to b743c1e Compare

January 26, 2026 15:42

devin-ai-integration Bot reviewed

View reviewed changes

devin-ai-integration Bot left a comment

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

View issue and 4 additional flags in Devin Review.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

Comment thread

sdk/python/feast/openlineage/mappers.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

ntkathole force-pushed the openlineage branch from b743c1e to e1ddd6b Compare

January 26, 2026 15:54

devin-ai-integration Bot reviewed

View reviewed changes

devin-ai-integration Bot left a comment

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 3 new potential issues.

View issues and 10 additional flags in Devin Review.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

Comment thread

sdk/python/feast/openlineage/emitter.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

sdk/python/feast/feature_store.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

sdk/python/feast/openlineage/emitter.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

devin-ai-integration Bot reviewed

View reviewed changes

devin-ai-integration Bot left a comment

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

View issue and 16 additional flags in Devin Review.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

Comment thread

sdk/python/feast/openlineage/emitter.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

ntkathole force-pushed the openlineage branch from f5640a8 to ce3ec64 Compare

January 26, 2026 16:28

devin-ai-integration Bot reviewed

View reviewed changes

devin-ai-integration Bot left a comment

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

View issue and 15 additional flags in Devin Review.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

Comment thread

sdk/python/feast/feature_store.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

ntkathole force-pushed the openlineage branch 2 times, most recently from b51e87a to 40a8559 Compare

January 26, 2026 16:43

devin-ai-integration Bot reviewed

View reviewed changes

devin-ai-integration Bot left a comment

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 3 new potential issues.

View issues and 15 additional flags in Devin Review.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

Comment thread

sdk/python/feast/feature_store.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

sdk/python/feast/feature_store.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

Comment thread

sdk/python/feast/feature_store.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

ntkathole force-pushed the openlineage branch 2 times, most recently from e350360 to 29194d7 Compare

January 27, 2026 13:03

devin-ai-integration Bot reviewed

Jan 27, 2026

View reviewed changes

devin-ai-integration Bot left a comment

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

View issue and 15 additional flags in Devin Review.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

Comment thread

sdk/python/feast/feature_store.py

Comment on lines 886 to +905

		if progress_ctx:
		progress_ctx.cleanup()

		# Emit OpenLineage events for applied objects
		self._emit_openlineage_apply_diffs(registry_diff)

		def _emit_openlineage_apply_diffs(self, registry_diff: RegistryDiff):
		"""Emit OpenLineage events for objects applied via diffs."""
		if self._openlineage_emitter is None:
		return

		# Collect all objects that were added or updated
		objects: List[Any] = []
		for feast_object_diff in registry_diff.feast_object_diffs:
		if feast_object_diff.new_feast_object is not None:
		objects.append(feast_object_diff.new_feast_object)

		if objects:
		self._emit_openlineage_apply(objects)

devin-ai-integration Bot Jan 27, 2026

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 OpenLineage apply events emitted even when _apply_diffs fails

FeatureStore._apply_diffs() emits OpenLineage apply events unconditionally after the try/finally block.

Actual: If infra_diff.update(...), apply_diff_to_registry(...), or update_infra(...) raises, the exception propagates but _emit_openlineage_apply_diffs(registry_diff) still runs, emitting lineage for objects that were not successfully applied.
Expected: Apply lineage should only be emitted after a successful apply/commit, or at least be gated on success.

Click to expand

In sdk/python/feast/feature_store.py:859-905, the OpenLineage emission is outside the try and will run even on failure:

try:
    infra_diff.update(...)
    apply_diff_to_registry(...)
    self._registry.update_infra(..., commit=True)
finally:
    ...

# always runs, even if the try-block raised
self._emit_openlineage_apply_diffs(registry_diff)

Impact: Produces incorrect lineage (false positives) in OpenLineage/Marquez, which can mislead debugging, auditing, and governance workflows.

(Refers to lines 859-905)

Recommendation: Move _emit_openlineage_apply_diffs(...) into the success path (after commit) or guard it with a success flag set only when the apply/commit completes without exception.

Was this helpful? React with 👍 or 👎 to provide feedback.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

ntkathole reacted with thumbs down emoji

All reactions

👎 1 reaction

devin-ai-integration Bot reviewed

Jan 27, 2026

View reviewed changes

Comment thread

sdk/python/feast/repo_config.py

Comment on lines +236 to +250

		def to_openlineage_config(self):
		"""Convert to feast.openlineage.OpenLineageConfig."""
		from feast.openlineage.config import OpenLineageConfig as OLConfig

		return OLConfig(
		enabled=self.enabled,
		transport_type=self.transport_type,
		transport_url=self.transport_url,
		transport_endpoint=self.transport_endpoint,
		api_key=self.api_key,
		namespace=self.namespace,
		producer=self.producer,
		emit_on_apply=self.emit_on_apply,
		emit_on_materialize=self.emit_on_materialize,
		)

devin-ai-integration Bot Jan 27, 2026

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚩 additional_config field lost during OpenLineageConfig conversion

The OpenLineageConfig dataclass (in openlineage/config.py) defines an additional_config: Dict[str, Any] field for transport-specific settings like Kafka bootstrap servers. However, the OpenLineageConfig pydantic model (in repo_config.py) doesn't define this field, and to_openlineage_config() doesn't pass it during conversion.

This means users cannot specify additional_config in feature_store.yaml, and any programmatically-set additional_config would be lost during conversion. If additional_config is only meant for programmatic use (not YAML config), this is fine. Otherwise, the pydantic model should include this field.

See sdk/python/feast/repo_config.py:236-250 (conversion) and sdk/python/feast/openlineage/config.py:51 (field definition).

Was this helpful? React with 👍 or 👎 to provide feedback.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

franciscojavierarceo reviewed

Jan 27, 2026

View reviewed changes

Comment thread

sdk/python/feast/feature_store.py

		progress_ctx.cleanup()

		# Emit OpenLineage events for applied objects
		self._emit_openlineage_apply_diffs(registry_diff)

franciscojavierarceo Jan 27, 2026 •

edited

Loading

Uh oh!

There was an error while loading. Please reload this page.

Copy link

Copy Markdown

Member

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should only emit if it's successful, right? here we could have an exception and still emit a lineage event

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

ntkathole Jan 28, 2026

Copy link

Copy Markdown

Member Author

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, and it's not emitting if apply failed. _emit_openlineage_apply_diffs() only executes if the entire try block completes successfully without raising an exception. I think devin is confused with try/finally code block here.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

franciscojavierarceo reacted with thumbs up emoji

franciscojavierarceo reacted with heart emoji

All reactions

👍 1 reaction
❤️ 1 reaction

devin-ai-integration Bot reviewed

Jan 28, 2026

View reviewed changes

devin-ai-integration Bot left a comment

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

View issue and 18 additional flags in Devin Review.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

Comment thread

sdk/python/feast/openlineage/mappers.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

ntkathole force-pushed the openlineage branch from 348187d to e8b524b Compare

January 29, 2026 05:51

devin-ai-integration Bot reviewed

View reviewed changes

devin-ai-integration Bot left a comment

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

View issue and 21 additional flags in Devin Review.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

Comment thread

sdk/python/feast/openlineage/emitter.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

ntkathole force-pushed the openlineage branch from e8b524b to b7fd6b5 Compare

January 29, 2026 05:59

devin-ai-integration Bot reviewed

View reviewed changes

devin-ai-integration Bot left a comment

Copy link

Copy Markdown

Contributor

There was a problem hiding this comment.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

View issue and 21 additional flags in Devin Review.

Sorry, something went wrong.

Uh oh!

There was an error while loading. Please reload this page.

All reactions

Comment thread

sdk/python/feast/feature_store.py Resolved

Uh oh!

There was an error while loading. Please reload this page.

franciscojavierarceo approved these changes

View reviewed changes

Comment thread

docs/reference/openlineage.md Resolved

Uh oh!

There was an error while loading. Please reload this page.

ntkathole and others added 5 commits

January 29, 2026 20:15


          feat: Added support for OpenLineage integration

9e0fc93

Signed-off-by: ntkathole <[email protected]>


          feat: Added openlineage in requirements

54f03f2

Signed-off-by: ntkathole <[email protected]>


          feat: Keep event type as complete instead of other

0a7799b

Signed-off-by: ntkathole <[email protected]>


          feat: Added blog post for OpenLineage integration

5812bce

Signed-off-by: ntkathole <[email protected]>


          Update docs/reference/openlineage.md

01d69a1

Co-authored-by: Francisco Javier Arceo <[email protected]>

ntkathole force-pushed the openlineage branch from 026da68 to 01d69a1 Compare

January 29, 2026 14:45

ntkathole merged commit df70d8d into master