Documentation | Stackable Data Platform | Platform Docs | Stackable Hub | Discussions | Discord
This is a Kubernetes operator to manage Apache Hadoop.
It is part of the Stackable Data Platform (SDP), a set of Kubernetes operators for open source data products such as Apache Kafka®, Apache Spark™, Apache Superset™, Trino and OpenSearch, all listed below. You describe each product in a single custom resource, and the operator takes care of running it, including configuration, security with TLS, Kerberos and Open Policy Agent, upgrades and the connections between products. It runs on Kubernetes and OpenShift, on premises, in the cloud or on a laptop.
You can install the operator using stackablectl or helm.
Read on to get started with it, or see it in action in one of our demos.
You can follow this tutorial.
The stable documentation for this operator can be found in our Stackable Data Platform documentation. If you are interested in the most recent state of this repository, check out the nightly docs instead.
The documentation for all Stackable products can be found at docs.stackable.tech. Release information, the supported product and platform versions, and the reference for every custom resource are on the Stackable Hub.
If you have a question about the Stackable Data Platform, contact us via our homepage or ask a public question in our Discussions forum.
Without SDP, you assemble each product yourself: from the project's own Helm chart where one exists, otherwise from a third-party chart or your own manifests. Each product then runs, and everything between the products is yours to build and keep working, for example TLS certificates and Kerberos across products, authorization, service discovery, exposing services outside the cluster, and testing upgrades together. SDP covers that part. The products in a release are released together, handle cross-cutting concerns such as security, logging, monitoring and lineage the same way, and get upgrades and fixes as one platform on a predictable schedule. Every image is signed and comes with an SBOM, see verifying image signatures and viewing and verifying SBOMs.
If you run a single product, a well-maintained chart exists for it, and you are happy to own its configuration, security and upgrades yourself, that chart can be a reasonable choice.
This operator is written and maintained by Stackable and is part of the Stackable Data Platform.
Underneath the product operators, the commons, secret and listener operators provide what all products share: S3 and authentication settings, TLS certificates and Kerberos keytabs, exposing services outside the cluster and more.
Stackable GmbH is the company behind the Stackable Data Platform and offers professional services, support subscriptions and custom development.
We develop and test our operators on the following platforms:
- AKS on Microsoft Azure
- EKS on Amazon Web Services (AWS)
- GKE on Google Cloud Platform (GCP)
- IONOS Cloud Managed Kubernetes
- K3s
- kind, for example on a laptop, as in the getting started guide
- Kubernetes
- Red Hat OpenShift
The Kubernetes and OpenShift versions each release supports are listed on hub.stackable.tech.
These are the operators that are currently part of the Stackable Data Platform:
- Stackable Operator for Apache Airflow
- Stackable Operator for Apache Druid, deprecated since SDP 26.7
- Stackable Operator for Apache HBase
- Stackable Operator for Apache Hadoop HDFS
- Stackable Operator for Apache Hive
- Stackable Operator for Apache Kafka
- Stackable Operator for Apache NiFi
- Stackable Operator for the Open Policy Agent (OPA)
- Stackable Operator for OpenSearch
- Stackable Operator for Apache Spark
- Stackable Operator for Apache Superset
- Stackable Operator for Trino
- Stackable Operator for Apache ZooKeeper
And the operators they all build on:
Contributions are welcome. Follow our Contributors Guide to learn how you can contribute. All contributors will have to sign a Contributor License Agreement. This is enforced automatically when you submit a Pull Request where a bot will guide you through the process.
Open Software License version 3.0.
Get started with the community edition! If you want professional support, we offer subscription plans and custom licensing.
If you want to support our work but don't need professional support please consider sponsoring our work.