GeezerVison

/static/python

/static/openapi

/openapi/

/static/openapi.json

label: 'Use Cases',

collapsed: false,

items: [

'use_cases/how-to-build',

{

type: 'category',

label: 'Agents',

Guides and runnable demos, sorted by what you want to build.

Guides and runnable demos, sorted by what you want to build. Every use case has the same shape: see [How to build a use case](/use_cases/how-to-build) for the steps and a starting point for each.

<HubGrid />

- **Agents write SQL.** An agent can define the context it needs as a SQL view and deploy it. The views, no matter how complex, instantly reflect changes to inputs (live and historical data).

- **Current context for every action.** When input data changes, agents instantly know what changed in the views they listen to or query.

**Build it:** connect your sources, let the agent create views with [`fda create`](/interface/cli), and have it read them with [`fda exec`](/sql/ad-hoc) or subscribe to their changes over [HTTP](/connectors/sinks/http). See [How to build a use case](/use_cases/how-to-build).

<UseCaseGrid items={[

{

kind: 'Demo',

Agents act for users, so each action needs a permission check. Fine-grained authorization models express permissions as a graph of users, agents, groups and resources. Feldera evaluates these graphs with [recursive SQL](/sql/recursion), and updates the results incrementally when a permission or a resource changes. [Auth0 FGA](https://www.feldera.com/blog/auth0-and-feldera-incrementally-computing-7-billion-permission-checks), part of Okta, uses Feldera to precompute billions of permission checks for AI agents and search.

Agents act for users, so each action needs a permission check. Fine-grained authorization is a graph problem: permissions are a graph of users, agents, groups and resources, and a check asks whether a path connects a user to a resource. Feldera evaluates these graphs with [recursive SQL](/sql/recursion), and updates the results incrementally when a permission or a resource changes. The same pattern works for any graph: reachability, hierarchies and transitive closure. [Auth0 FGA](https://www.feldera.com/blog/auth0-and-feldera-incrementally-computing-7-billion-permission-checks), part of Okta, uses Feldera to precompute billions of permission checks for AI agents and search.

- **Permission checks for every user and agent.** Feldera computes the access rights across deeply nested access graphs, so a check reads a precomputed answer.

- **Current after each change.** When a permission changes, Feldera updates only the access rights that the change affects.

**Build it:** load users, groups, objects and relationship tuples from [PostgreSQL CDC](/connectors/sources/postgresql-cdc) or [Kafka](/connectors/sources/kafka). Resolve permissions with [recursive views](/sql/recursion), and write the results to [Redis](/connectors/sinks/redis) or [PostgreSQL](/connectors/sinks/postgresql) for fast lookups. Start from the [static FGA](https://github.com/feldera/feldera/blob/main/crates/pipeline-manager/demos/sql/09-fine-grained-authorization.sql) or [dynamic FGA](https://github.com/feldera/feldera/blob/main/crates/pipeline-manager/demos/sql/10-dynamic-fga.sql) example SQL.

<UseCaseGrid items={[

{

kind: 'Customer story',

- **Exact features for each transaction.** Joins and [rolling aggregates](/tutorials/time-series) are updated incrementally, so each feature vector is correct when the transaction arrives.

- **Models in the loop.** A model can [read its inputs from a view and write its scores to a table](/use_cases/model_scoring/), so the pipeline measures its accuracy while it runs.

Load transactions from [Kafka](/connectors/sources/kafka) and history from [Delta Lake](/connectors/sources/delta). Write the features as views with joins and rolling aggregates. Read feature vectors with an [HTTP change stream](/connectors/sinks/http) or write them to a feature store. Start from the [fraud detection](https://github.com/feldera/feldera/blob/main/crates/pipeline-manager/demos/sql/02-fraud-detection.sql) or [feature engineering](https://github.com/feldera/feldera/blob/main/crates/pipeline-manager/demos/sql/05-feature-engineering.sql) example SQL. The steps are in [How to build a use case](/use_cases/how-to-build).

<UseCaseGrid items={[

title: How to build a use case

description: Every Feldera use case has the same shape. Connect sources to SQL tables, write the logic as SQL views, and send the results out. Steps, example SQL and starting points for each use case.

import {FlowDiagram} from '@site/src/components/UseCases';

Every Feldera use case has the same shape. Input connectors feed changes into SQL tables. SQL views compute the answers, and Feldera keeps them current on every change. Output connectors, queries or a change stream deliver the results. To build your use case, start from the [example](#pick-a-starting-point) closest to it, then change the sources, the views and the outputs.

<FlowDiagram

nodes={[

{label: 'Sources', sub: 'Lakehouse, Kafka, CDC'},

{label: 'SQL tables and views', sub: 'one Feldera pipeline'},

{label: 'Results', sub: 'sinks, queries, change stream'},

]}

edges={['changes', 'changes']}

/>

Run Feldera locally with [Docker](/get-started/docker), or use the [online sandbox](/get-started/sandbox). Install the [`fda` CLI](/interface/cli) to manage pipelines from a terminal, a script or an agent.

A table declares the shape of an input. Its `connectors` property says where the data comes from. This table loads a Delta Lake table, then follows new versions of it:

CREATE TABLE orders (

order_id BIGINT NOT NULL PRIMARY KEY,

customer_id BIGINT,

status VARCHAR,

total DECIMAL(12, 2)

) WITH ('connectors' = '[{

}]');

See all [input connectors](/connectors/sources/): Delta Lake, Iceberg, Kafka, PostgreSQL CDC, Debezium, S3, HTTP and more.

Write the answers you need as SQL views. Views can use [full SQL](/sql), build on each other, and be [recursive](/sql/recursion). Declare a view `MATERIALIZED` to [query it](/sql/ad-hoc) at any time:

CREATE MATERIALIZED VIEW revenue_by_status AS

SELECT status, COUNT(*) AS orders, SUM(total) AS revenue

FROM orders

GROUP BY status;

Attach [output connectors](/connectors/sinks/) to a view to write its changes to Kafka, Delta Lake, PostgreSQL, Redis and other destinations:

CREATE VIEW large_orders

WITH ('connectors' = '[{

}]')

AS SELECT * FROM orders WHERE total > 10000;

fda create my-pipeline program.sql

fda start my-pipeline

You can also use the [web console](/interface/web-console), the [Python SDK](pathname:///python/) or the [REST API](/api) ([OpenAPI spec](pathname:///openapi.json)).

Query a materialized view:

fda exec my-pipeline "SELECT * FROM revenue_by_status"

Or subscribe to a view's changes over [HTTP](/connectors/sinks/http). With `send_snapshot=true`, you get the current contents first, then every change:

curl -X POST 'http://localhost:8080/v0/pipelines/my-pipeline/egress/revenue_by_status?format=json&send_snapshot=true'

Each example below is a complete pipeline. Copy its SQL, then change the connectors and the views.

| Use case | Sources | Views | Results | Start from |

|---|---|---|---|---|

| [Agent context](/use_cases/agents#instantly-fresh-context) | Any mix of historical and live data | Views that the agent defines and deploys with `fda` | Ad-hoc queries, HTTP change stream | [Agentic fraud detection](https://github.com/feldera/feldera-demos/tree/main/agentic-fraud-detection) |

| [Fine-grained authorization](/use_cases/agents#agent-security) (a graph use case) | Users, groups, objects and relationship tuples from PostgreSQL CDC or Kafka | Recursive views that resolve nested permissions | Redis, PostgreSQL or Kafka for fast lookups | [Static FGA](https://github.com/feldera/feldera/blob/main/crates/pipeline-manager/demos/sql/09-fine-grained-authorization.sql), [dynamic FGA](https://github.com/feldera/feldera/blob/main/crates/pipeline-manager/demos/sql/10-dynamic-fga.sql), [FGA guide](/use_cases/fine_grained_authorization/intro) |

| Other graphs: reachability, hierarchies, transitive closure | Node and edge tables from any source | [Recursive views](/sql/recursion) | Any output | The FGA examples, [recursive queries](/sql/recursion) |

| [Real-time medallion](/use_cases/real-time-pipelines) | Bronze Delta Lake or Iceberg tables | Silver and gold views | Delta Lake | [Medallion SQL](https://github.com/feldera/feldera/blob/main/crates/pipeline-manager/demos/sql/01-medallion-architecture.sql), [guide](/use_cases/medallion_architecture/part1) |

| [Lakehouse batch jobs](/use_cases/real-time-pipelines) | The batch job's Delta Lake, Iceberg or S3 tables | The job's SQL as views | Back to the lakehouse | [Batch analytics SQL](https://github.com/feldera/feldera/blob/main/crates/pipeline-manager/demos/sql/00-accelerating-batch-analytics.sql), [guide](/use_cases/batch/intro) |

| [Streaming and CDC](/use_cases/real-time-pipelines) | Kafka, Debezium, PostgreSQL CDC | Views of any complexity | Kafka, PostgreSQL, Redis, HTTP | [Debezium PostgreSQL](https://github.com/feldera/feldera-demos/tree/main/debezium-postgres), [Debezium MySQL](https://github.com/feldera/feldera-demos/tree/main/debezium-mysql) |

| [Fraud detection](/use_cases/fraud-detection) | Transactions and reference data from Kafka and Delta Lake | Rolling-window features | A model, or a feature store | [Fraud SQL](https://github.com/feldera/feldera/blob/main/crates/pipeline-manager/demos/sql/02-fraud-detection.sql), [feature engineering SQL](https://github.com/feldera/feldera/blob/main/crates/pipeline-manager/demos/sql/05-feature-engineering.sql), [guide](/use_cases/fraud_detection/) |

More complete pipelines: [example SQL programs](https://github.com/feldera/feldera/tree/main/crates/pipeline-manager/demos/sql) and [runnable demos](https://github.com/feldera/feldera-demos).

The `felderize` skill in [Feldera Agent Skills](https://github.com/feldera/feldera-skills) translates Spark SQL to Feldera SQL. The [batch jobs guide](/use_cases/batch/intro) shows a full conversion.

- **History and live data in one pipeline.** Load a snapshot from [Delta Lake](/connectors/sources/delta) or [Iceberg](/connectors/sources/iceberg), then continue on changes from [Kafka](/connectors/sources/kafka), [Debezium](/connectors/sources/debezium) or [PostgreSQL CDC](/connectors/sources/postgresql-cdc).

- **Correct results.** Each view always equals the result of its query on all of the input. With [fault tolerance](/pipelines/fault-tolerance-overview), a pipeline continues from where it stopped after a crash.

The steps are in [How to build a use case](/use_cases/how-to-build). The usual shapes:

| Shape | Sources | Results | Start from |

|---|---|---|---|

| Real-time medallion | Bronze [Delta Lake](/connectors/sources/delta) or [Iceberg](/connectors/sources/iceberg) tables in `snapshot_and_follow` mode | Silver and gold views, written to [Delta Lake](/connectors/sinks/delta) | [Medallion SQL](https://github.com/feldera/feldera/blob/main/crates/pipeline-manager/demos/sql/01-medallion-architecture.sql) |

| Lakehouse batch job | The job's lakehouse or [S3](/connectors/sources/s3) tables | The job's SQL as views, written back to the lakehouse | [Batch analytics SQL](https://github.com/feldera/feldera/blob/main/crates/pipeline-manager/demos/sql/00-accelerating-batch-analytics.sql) |

| Streaming and CDC | [Kafka](/connectors/sources/kafka), [Debezium](/connectors/sources/debezium), [PostgreSQL CDC](/connectors/sources/postgresql-cdc) | [Kafka](/connectors/sinks/kafka), [PostgreSQL](/connectors/sinks/postgresql), [Redis](/connectors/sinks/redis) | The Debezium demos below |

To port Spark SQL, use the `felderize` skill in [Feldera Agent Skills](https://github.com/feldera/feldera-skills).

<UseCaseGrid items={[

Guides and runnable demos for what you want to build. See all [guides](/tutorials/), or learn the product step by step in [Feldera Basics](/tutorials/basics/).

Guides and runnable demos for what you want to build. [How to build a use case](/use_cases/how-to-build) gives the steps and a starting point for each. See all [guides](/tutorials/), or learn the product step by step in [Feldera Basics](/tutorials/basics/).

<HubGrid />

const config: Config = {

title: "Feldera Documentation",

tagline: "Powerup your Data Lake with Incremental View Maintenance",

tagline: "Keep arbitrarily complex SQL queries instantly fresh over changing data.",

// The favicon links are in headTags below, so they are in the HTML on every page, like feldera.com.

// Set the production url of your site here

clientModules: [

require.resolve("./src/clientModules/detect-os.ts"),

require.resolve("./src/clientModules/api-child-attributes.ts"),

// Brand fonts, self-hosted: DM Sans for text, Geist Mono for labels and code.

require.resolve("@fontsource-variable/dm-sans/wght.css"),

require.resolve("@fontsource-variable/geist-mono/wght.css"),

"scripts": {

"docusaurus": "docusaurus",

"start": "docusaurus start --host 0.0.0.0 --no-open",

"build": "mkdir -p openapi && cp ../openapi.json ./openapi/openapi.json && docusaurus build",

"build": "mkdir -p openapi && cp ../openapi.json ./openapi/openapi.json && cp ../openapi.json ./static/openapi.json && docusaurus build",

"swizzle": "docusaurus swizzle",

"deploy": "docusaurus deploy",

"clear": "docusaurus clear",