Open-Source Databases · apache

kyuubi

Apache Kyuubi is a distributed SQL gateway that lets end-users query data lakes and warehouses using standard SQL via JDBC/ODBC, without needing Spark expertise. It runs Spark engines in a multi-tenant, serverless model with built-in resource isolation, high availability, and support for Kubernetes and YARN.

Talk to DEV.co Discuss Implementation

Source: GitHub — github.com/apache/kyuubi

2.4k

GitHub stars

Forks

Scala

Primary language

Apache-2.0

License (OSI-approved)

Key facts

Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.

Field	Value
Repository	apache/kyuubi
Owner	apache
Primary language	Scala
License	Apache-2.0 — OSI-approved
Stars	2.4k
Forks	1k
Open issues	507
Latest release	v1.11.1 (2026-04-13)
Last updated	2026-07-08
Source	https://github.com/apache/kyuubi

What kyuubi is

Kyuubi is a HiveServer2-compatible Thrift gateway written in Scala that decouples SQL query routing from Spark engine execution, enabling multi-tenant workload isolation, dynamic engine scaling, and integration with cluster managers (YARN, Kubernetes). It supports Spark SQL dialects and extends traditional Spark Thrift Server with horizontal scalability and fault tolerance.

Quickstart

Get the kyuubi source

Clone the repository and explore it locally.

terminalbash

1git clone https://github.com/apache/kyuubi.git2cd kyuubi3# follow the project's README for install & configuration

Need it deployed, integrated, or customized instead? DEV.co ships production installs.

Best use cases

Migrating HiveServer2 workloads to Spark

Organizations running HiveServer2 can port queries to Kyuubi while gaining Spark performance, multi-tenancy, resource queues (YARN/Kubernetes), and improved client concurrency without rewriting SQL.

Multi-user lakehouse/data lake query service

Provides a unified SQL portal for ETL and BI analytics on data lakes with automatic resource isolation, access control, and support for multiple catalogs—one data copy, one SQL interface.

Kubernetes-native Spark SQL platform

Deploy serverless Spark engines on Kubernetes with Kyuubi as the stateless gateway, enabling auto-scaling, high availability, and cost-efficient per-tenant engine lifecycle management.

Implementation considerations

Scala codebase requires JVM environment; build, test, and runtime depend on Java/Scala toolchain and Spark version compatibility.
Multi-tenant resource isolation relies on correct configuration of YARN queues, Kubernetes namespaces, or equivalent cluster manager features; misconfiguration can leak resources between tenants.
Engine lifecycle management (startup, shutdown, session pooling) must be tuned for your workload patterns; under-provisioning causes query queuing, over-provisioning wastes resources.
Thrift JDBC/ODBC client libraries must be available and compatible with your BI/ETL tools; some tools may have integration gaps or require custom connectors.
SQL dialect compatibility depends on underlying Spark SQL version; non-standard or legacy SQL may need rewriting before migration from HiveServer2.

When to avoid it — and what to weigh

Requires tight Spark API coupling — If your use case requires deep Spark DataFrame or RDD APIs, Kyuubi's SQL-only gateway model does not expose programmatic Spark interfaces. Direct Spark clusters are a better fit.
Need strong commercial support guarantees — Kyuubi is Apache-licensed open source. While an active community exists, there is no built-in commercial SLA or dedicated vendor support without engaging external consultants.
Single-tenant, low-concurrency workloads — Kyuubi's multi-tenancy and resource isolation overhead adds deployment complexity. Simple single-user Spark Thrift Server may be sufficient and simpler for small teams.
Minimal operational expertise available — Kyuubi deployment spans Scala code, Thrift/JDBC configuration, cluster manager integration (YARN/K8s), and Spark tuning. Requires experienced platform or data engineering team.

License & commercial use

Apache License 2.0 (Apache-2.0). Permissive, OSI-approved license. Allows commercial use, modification, and distribution with attribution and liability disclaimer. No copyleft restrictions.

Apache-2.0 permits commercial deployment without license fees or vendor approval. However, commercial use does not include vendor support, SLA, security patching guarantees, or indemnification. Organizations must assess internal capacity for maintenance, security updates, and operational support; consider engaging Apache-certified consultants if expertise is unavailable.

DEV.co evaluation signals

Editorial assessment — not user reviews. Directional, with an explicit confidence level.

Signal	Assessment
Maintenance	Active
Documentation	Adequate
License clarity	Clear
Deployment complexity	High
DEV.co fit	Good
Assessment confidence	High

Security considerations

Authentication/authorization delegated to Thrift SASL, Kerberos, or LDAP; end-to-end security depends on correct configuration of these mechanisms and cluster manager access controls. Multi-tenancy isolation relies on cluster manager enforcement (YARN queues, Kubernetes RBAC); misconfiguration risks tenant data leakage. Kyuubi server itself is stateless, reducing attack surface, but Spark engines and metadata stores remain sensitive. No vulnerability disclosure policy or security advisory process detailed in provided data; consult Apache Security page for protocol.

Alternatives to consider

Spark Thrift Server (STS)

Built-in, single-process, HiveServer2-compatible SQL gateway. Simpler to deploy for single-tenant use but lacks multi-tenancy, resource isolation, and horizontal scaling that Kyuubi provides.

Presto/Trino

Distributed SQL query engine with multi-cluster federation, lower latency, and native support for diverse data sources. Better for interactive BI but requires different SQL dialect (Trino-flavored), less Spark-native integration, and different operational model.

Apache Hive + HiveServer2

Mature, battle-tested SQL-on-Hadoop platform with strong HDFS integration and Kerberos/LDAP auth. Slower query performance than Spark; if you need Spark performance and multi-tenancy, Kyuubi is the intended upgrade path.

Software development agency

Build on kyuubi with DEV.co software developers

Apache Kyuubi is a mature, Apache-licensed SQL gateway designed for organizations migrating from HiveServer2 or building multi-tenant lakehouse platforms on Spark. Assess your team's Spark and Kubernetes expertise, cluster manager readiness, and multi-tenancy requirements before deployment. Reach out to explore proof-of-concept options or consult Apache-certified partners for implementation support.

Talk to DEV.co

Related open-source tools

Surfaced by semantic similarity across the DEV.co open-source index.

spark hive linkis

Related on DEV.co

Explore the category and the services that help you build with it.

Open-Source Databases Data Platform & DevOps Cloud Deployment Custom Enterprise Software Talk to DEV.co

kyuubi FAQ

Can I migrate from HiveServer2 to Kyuubi without rewriting SQL?

Mostly yes. Kyuubi is HiveServer2-compatible at the Thrift JDBC/ODBC protocol level and supports Spark SQL, which is a superset of Hive SQL. However, dialect differences (UDFs, functions, syntax) may require minor rewrites. Test migration on a staging cluster first.

What cluster managers does Kyuubi support?

YARN (Hadoop) and Kubernetes are primary targets. YARN and K8s integration enables multi-tenant engine provisioning and resource isolation. Support for other cluster managers (Standalone, Mesos) is not clearly stated in provided data; requires review of latest docs.

Do I need to run separate Spark engines per tenant?

Kyuubi supports both dedicated and shared engine models. Multi-tenant isolation can be achieved via YARN queues or Kubernetes namespaces without strictly 1:1 tenant-to-engine mapping. Engine pooling and lifecycle policies are tunable.

Is Kyuubi suitable for real-time analytics?

Unknown. Kyuubi is positioned as a SQL gateway for data warehouses and lakehouses, typically batch/analytical workloads. Latency characteristics and support for streaming SQL are not detailed in provided data; review documentation for performance benchmarks.

Software developers & web developers for hire

DEV.co is a software development agency delivering custom software development services to companies building on open source. Our software developers and web developers design, integrate, and ship production systems — spanning web development, APIs, AI, data, and cloud. If kyuubi is part of your open-source databases roadmap, our team can implement, customize, migrate, and maintain it.

Evaluate Kyuubi for Your Data Platform

Talk to DEV.co Schedule a Consultation