Open-Source Databases · apache

linkis

Apache Linkis is a Java-based middleware layer that sits between applications and data engines (Spark, Hive, Presto, Flink, etc.), enabling unified connectivity and governance. It abstracts away engine complexity, standardizes resource management, and allows cross-engine job orchestration through REST/WS/JDBC interfaces.

Talk to DEV.co Discuss Implementation

Source: GitHub — github.com/apache/linkis

3.4k

GitHub stars

1.2k

Forks

Java

Primary language

Apache-2.0

License (OSI-approved)

Key facts

Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.

Field	Value
Repository	apache/linkis
Owner	apache
Primary language	Java
License	Apache-2.0 — OSI-approved
Stars	3.4k
Forks	1.2k
Open issues	169
Latest release	release-1.8.0 (2025-10-17)
Last updated	2026-07-06
Source	https://github.com/apache/linkis

What linkis is

Linkis provides a computation middleware stack with engine connectors, context services (variables, UDFs, result sets), multi-level label-based routing/load-balancing, resource governance, and unified data source management. Supports diverse engines and languages (SparkSQL, HiveSQL, Python, Shell, Pyspark, Scala) through pluggable EngineConn architecture.

Quickstart

Get the linkis source

Clone the repository and explore it locally.

terminalbash

1git clone https://github.com/apache/linkis.git2cd linkis3# follow the project's README for install & configuration

Need it deployed, integrated, or customized instead? DEV.co ships production installs.

Best use cases

Unified big-data platform gateway

Centralize access to multiple compute engines (Spark, Hive, Presto, Flink) from one REST/JDBC endpoint, reducing application-side complexity and enabling cross-engine job routing and load balancing.

Multi-tenant data analytics orchestration

Enforce resource quotas, traffic control, and task routing policies across users and teams sharing a common compute infrastructure, with unified variable and UDF management.

Context and resource reuse across engines

Share scripts, UDFs, functions, JAR files, and parameter variables across users, systems, and different compute engines without manual duplication or synchronization.

Implementation considerations

Requires JDK 8+ and Java operational expertise for deployment, configuration (multi-level labels, routing rules, resource policies), and troubleshooting.
EngineConn versions and compatibility vary; verify engine dependency versions (Spark 3.2.1, Hive 3.1.3, Flink 1.12.2, etc.) match your infrastructure before deployment.
Multi-tenant setup demands careful label hierarchy design, resource quotas, and traffic control rules to prevent one tenant or workload from starving others.
Unified context service requires coordination: ensure shared script, UDF, and variable lifecycle is well-governed to avoid stale or conflicting definitions.
Error code and troubleshooting documentation must be evaluated for applicability to your engine versions and data stack specifics.

When to avoid it — and what to weigh

Single-engine or simple JDBC use-case — If you only need to query one database or single Spark cluster, Linkis' multi-engine orchestration overhead may not justify the added complexity. Standard JDBC drivers or native SDKs may be simpler.
Lightweight, serverless architectures — Linkis is a stateful, Java-based middleware requiring dedicated deployment and operational overhead. Not suitable for ephemeral, fully managed cloud services or edge environments with minimal infrastructure.
Real-time, sub-second latency requirements — Linkis introduces additional middleware hops and governance overheads. For ultra-low-latency scenarios, direct connections to engines or streaming platforms may be more appropriate.
Minimal Java footprint or non-Java tech stack — Linkis is Java-only. If your platform is Python-first, Go, or Node.js-centric, integration and operational burden increase significantly.

License & commercial use

Apache License 2.0 (Apache-2.0). A permissive OSI license allowing commercial use, modification, and distribution with attribution and liability disclaimer.

Apache-2.0 permits commercial use without per-seat licensing. Derivative works must retain the Apache license. For commercial support, liability indemnification, or SLA guarantees, engage with the Apache Linkis community or commercial vendors offering Linkis-based services. No commercial use restrictions detected in the license itself.

DEV.co evaluation signals

Editorial assessment — not user reviews. Directional, with an explicit confidence level.

Signal	Assessment
Maintenance	Active
Documentation	Adequate
License clarity	Clear
Deployment complexity	High
DEV.co fit	Good
Assessment confidence	High

Security considerations

Standard security practices apply: validate authentication/authorization integration with identity systems; isolate Linkis and engine clusters within secured network boundaries; apply principle of least privilege to data source credentials and UDF/script access; audit multi-tenant label boundaries to prevent cross-tenant data leakage; keep Java/engine dependencies patched. No formal security audit or penetration test results mentioned in the data provided.

Alternatives to consider

Apache Livy

Provides REST API to Spark clusters with session management; lighter-weight but Spark-only, lacks broader engine orchestration, governance, and context services Linkis offers.

Presto / Trino coordinator

Federated query engine supporting multiple data sources; simpler for ad-hoc SQL queries but not a general-purpose orchestration middleware for batch, streaming, and complex workflows.

Apache Airflow + native engine connectors

Orchestration-focused workflow scheduler; offers better DAG visualization and scheduling but requires custom engine integration and lacks unified context/resource governance Linkis provides.

Software development agency

Build on linkis with DEV.co software developers

Simplify multi-engine orchestration, governance, and resource management. Start with the community, or engage a Devco partner for enterprise deployment and support.

Talk to DEV.co

Related open-source tools

Surfaced by semantic similarity across the DEV.co open-source index.

spark hive beam

Related on DEV.co

Explore the category and the services that help you build with it.

Open-Source Databases AI & ML Development Enterprise Data & Analytics Cloud Deployment & DevOps Talk to DEV.co

linkis FAQ

Does Linkis require modification to my existing Spark/Hive clusters?

Linkis interacts with engines via their standard interfaces (JDBC, Thrift, REST). No cluster code changes required, but EngineConn deployment and integration must be configured on Linkis side.

Can I use Linkis in a single-engine environment?

Yes, technically possible, but not the intended use-case. Linkis' value is in multi-engine orchestration and unified governance. For single engines, standard SDKs or direct connectivity may be simpler.

What is the learning curve for operators and developers?

Developers benefit from unified REST/JDBC interfaces (moderate learning for label-based routing and context service). Operators need Java/middleware expertise and understanding of engine-specific tuning. Estimated ramp-up: weeks to months depending on background.

Is Linkis suitable for a data lake platform (Delta Lake, Iceberg, Hudi)?

Yes, Linkis can route Delta/Iceberg/Hudi queries through Spark or Presto EngineConns. However, ensure version compatibility and test thoroughly; data lake-specific features may require custom handlers.

Software development & web development with DEV.co

DEV.co is a software development agency delivering custom software development services to companies building on open source. Our software developers and web developers design, integrate, and ship production systems — spanning web development, APIs, AI, data, and cloud. If linkis is part of your open-source databases roadmap, our team can implement, customize, migrate, and maintain it.

Build a unified data platform with Apache Linkis

Simplify multi-engine orchestration, governance, and resource management. Start with the community, or engage a Devco partner for enterprise deployment and support.

Talk to DEV.co Schedule a Consultation