This project is in active development. A stable version will be released soon.
Requires Java 17+ · Built on Apache TinkerPop 3.8.1
Analyze data from multiple sources using the power of graphs.
Unipop is a data Federation and Virtualization engine that models your data as a "virtual" graph, exposing a querying API using the Gremlin GQL (Sql and SPARQL are also available.)
This means you get the benefits of a graph data model without migrating/replicating/restructuring your data, whether its stored in a RDBMS, NoSql Store, or any other data source (see "Customize and Extend" below.)
Graphs provide a very "natural" way to analyze your data. The simple Vertex/Edge structure makes it very easy to model complex and varied data, and then analyze it by exploring the connections/relationships in it.
This is especially relevant for a data Federation / Virtualization platform, which integrates a large variety of different data sources, structures, and schemas.
Our chosen GQL is Gremlin, which comes as part of the Apache Tinkerpop framework. Let's compare Gremlin to SQL, the industry standard:
| Schema | Relationships | Flexibility | Usability | |
|---|---|---|---|---|
| SQL | Structured - Tables and their fields need to be explicitly defined. | Joins require knowledge of all relationships (PK/FK), and can become quite complicated. | Sql's syntax requires very specific, rigid structures. | Queries are loosely-typed "free text", often requiring complicated ORMs. |
| Gremlin | Unstructured - Different structures can be created on the fly. | Connections (i.e edges) are "First-class citizens", enabling easy exploration of your data. | Queries are written in a pipelined ("functional") syntax, providing considerable flexibility. | Host Language embedding. Easier to read, write, find errors, and reuse queries. |
The Tinkerpop framework also provides us with other useful features "out of the box":
- Traversal Strategies - an extensible query optimization mechanism. Unipop utilizes this to implement different performance optimizations.
- Console & Server - production grade tooling.
- Language Drivers - JavaScript, TypeScript, PHP, Python, Java, Scala, .Net, Go.
- Extensible Query Languages - Gremlin, SQL, SPARQL
- DSL support
- Testing Framework
Unipop is a Maven multi-module project (Java 17, Apache TinkerPop 3.8.1):
| Module | Description |
|---|---|
unipop-core |
Federation engine: the UniGraph implementation, query translation, and TinkerPop traversal strategies. |
unipop-jdbc |
JDBC / SQL source provider (verified against PostgreSQL). |
unipop-elastic |
Elasticsearch source provider (Elasticsearch 8.x, via the Elasticsearch Java API Client). |
unipop-rest |
HTTP / templated-REST source provider (built on the Elasticsearch provider). |
unipop-test |
Cross-backend integration tests. |
Requires JDK 17 and Maven 3.9+.
# JDK 17 must be the active JDK (e.g. Homebrew keg-only openjdk@17): export JAVA_HOME=/opt/homebrew/opt/openjdk@17 # Build all modules (skip tests for a fast build): mvn clean install -DskipTests # Run a module's test suite: mvn -pl unipop-jdbc test # JDBC/PostgreSQL mvn -pl unipop-elastic test # Elasticsearch
The test suites run against real backends, provisioned automatically — no manual setup or Docker image pulls required beyond a running Docker daemon for Elasticsearch:
- JDBC uses an embedded PostgreSQL (
io.zonky.test:embedded-postgres, no Docker). - Elasticsearch / REST use Testcontainers (Docker required).
TinkerPop process compliance is exercised via the shared Gherkin/Cucumber .feature
suites (*FeatureTest); structure compliance via the *StructureSuite classes. As a
partial federation provider, Unipop does not implement every TinkerPop capability, so
the suites run with testFailureIgnore=true and report a partial pass rate rather than
failing the build. See MIGRATION_NOTES.md for details.
TBD
- Console - a local instance with an interactive Shell for issuing queries.
- Server - a web server with WebSocket & HTTP APIs.
- Embedded - run Unipop inside any JVM based application.
Add your data sources to Unipop's configuration. Configuring a source entails mapping its schema to a "property graph" model (i.e. vertices & edges). Unipop is built in an extensible way, enabling many different mapping options.
Console, Server, Embedded, or Language drivers
TBD
TBD
Technical details.
TBD