Open-Source Databases · ibis-project

ibis

Ibis is a portable Python dataframe library that compiles Python expressions into SQL or native backend code, allowing the same code to work across 20+ data backends from DuckDB and Pandas to BigQuery and Snowflake. It emphasizes local iteration with lazy evaluation and remote deployment by changing a single line of code.

Talk to DEV.co Discuss Implementation

Source: GitHub — github.com/ibis-project/ibis

6.6k

GitHub stars

736

Forks

Python

Primary language

Apache-2.0

License (OSI-approved)

Key facts

Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.

Field	Value
Repository	ibis-project/ibis
Owner	ibis-project
Primary language	Python
License	Apache-2.0 — OSI-approved
Stars	6.6k
Forks	736
Open issues	506
Latest release	12.0.0 (2026-02-07)
Last updated	2026-07-08
Source	https://github.com/ibis-project/ibis

What ibis is

Ibis provides a unified dataframe API that abstracts over SQL-generating backends (PostgreSQL, BigQuery, Snowflake, etc.) and DataFrame backends (DuckDB, Polars, PySpark). It uses expression compilation and mixing of Python and SQL code to enable portability; DuckDB is the default execution engine for local work.

Quickstart

Get the ibis source

Clone the repository and explore it locally.

terminalbash

1git clone https://github.com/ibis-project/ibis.git2cd ibis3# follow the project's README for install & configuration

Need it deployed, integrated, or customized instead? DEV.co ships production installs.

Best use cases

Multi-backend analytics workflows

Develop and test analysis locally on DuckDB or Polars, then deploy the identical code against BigQuery, Snowflake, or Trino in production without refactoring.

SQL generation and compilation

Use Python's expressiveness to construct complex SQL queries that Ibis compiles automatically, while retaining ability to drop into raw SQL when needed.

Data exploration and iteration

Interactive mode with fast local execution via DuckDB enables exploratory data analysis without expensive warehouse queries; transition to remote backends for scaled processing.

Implementation considerations

Each backend requires its own connector/dialect implementation; verify your target backends are documented and actively maintained before committing to deployment.
Default local backend is DuckDB; ensure DuckDB supports your data types, file formats, and query patterns; fallback to Pandas/Polars if limitations emerge.
Lazy evaluation model requires explicit materialization (e.g., `.to_pandas()`, `.execute()`) to trigger computation; this differs from eager pandas semantics and needs training.
Type mapping and NULL handling vary across backends; test schema inference and edge cases (e.g., NULLs, string encoding) on target backends early.
Performance profiling should happen on actual target backend(s), not just local DuckDB, to catch SQL compilation or dialect-specific slowdowns.

When to avoid it — and what to weigh

Real-time streaming pipelines — Ibis is optimized for batch query execution. For sub-second streaming analytics or event-driven architectures, dedicated streaming platforms (Kafka, Flink, Kinesis) are more appropriate.
Complex custom transformations or UDFs — If your logic relies heavily on backend-specific stored procedures or UDFs, Ibis's abstraction layer may constrain capability or force complex workarounds.
Single-backend, performance-critical workloads — If you're locked to one database and need maximum control over execution plans and query optimization, direct SQL or a backend-native driver may offer better performance tuning.
Mature production systems with existing SQL codebase — Migrating large, well-tested SQL systems to Ibis introduces rewrite and validation risk; benefit primarily accrues if multi-backend portability is a genuine future requirement.

License & commercial use

Ibis is licensed under Apache License 2.0 (Apache-2.0), a permissive open-source license that allows commercial use, modification, and distribution with attribution and liability waiver.

Apache-2.0 explicitly permits commercial use in proprietary applications without royalties. No commercial license, usage fees, or vendor lock-in. Ensure compliance with patent and liability clauses; consult legal counsel if bundling or reselling is planned.

DEV.co evaluation signals

Editorial assessment — not user reviews. Directional, with an explicit confidence level.

Signal	Assessment
Maintenance	Active
Documentation	Strong
License clarity	Clear
Deployment complexity	Low
DEV.co fit	Strong
Assessment confidence	High

Security considerations

Ibis itself does not authenticate or authorize database access; it delegates to backend connectors and credentials. No built-in encryption, masking, or audit logging at Ibis layer—security depends entirely on underlying database and credential management. SQL injection risk is low (parameterized queries are generated), but untrusted user input in raw SQL blocks (`.sql()`) must be sanitized. Sensitive credentials (API keys, passwords) should be managed via environment variables or secret management tools, not hardcoded. Review each backend's security model independently.

Alternatives to consider

Pandas + SQLAlchemy

Established Python ecosystem for single-backend workflows; less abstraction overhead but no multi-backend portability or lazy evaluation.

Polars (native)

High-performance in-process dataframe engine with fast execution and eager API; lacks SQL generation and multi-backend support but simpler mental model for local analytics.

dbt (Data Build Tool)

SQL-first transformation layer with version control and lineage; focuses on reproducible SQL workflows rather than Python portability; complementary rather than competitive.

Software development agency

Build on ibis with DEV.co software developers

Ibis enables teams to develop data transformations in Python locally and deploy to any of 20+ SQL or DataFrame backends with a single line change. Review the documentation, test against your target backends, and consider whether portability or performance gains align with your roadmap.

Talk to DEV.co

Related open-source tools

Surfaced by semantic similarity across the DEV.co open-source index.

bruin dbx bdash

Related on DEV.co

Explore the category and the services that help you build with it.

Open-Source Databases AI/Application Development API Development Cloud Deployment Talk to DEV.co

ibis FAQ

Can I use Ibis in a Jupyter notebook or production batch job?

Yes to both. Ibis is a Python library that works in interactive notebooks (with `ibis.options.interactive = True`) and in scripts/batch jobs. Execution model is the same; only the output formatting differs.

Do I have to rewrite my SQL?

No. Ibis supports raw SQL via the `.sql()` method, allowing you to embed SQL strings within Python workflows. You can mix and match Python expressions and SQL incrementally.

What if my target backend is not supported?

Ibis covers 20+ backends (see list in README). If your backend is missing, you can file a GitHub issue or contribute a backend implementation. For niche databases, direct SQL drivers may be more practical.

Is there a performance penalty for using Ibis vs. writing native backend queries?

Ibis compiles to native SQL or DataFrame operations; no runtime overhead. Performance depends on the quality of compiled SQL and backend optimization. Test on your actual target backend; some backends may have dialect-specific quirks.

Custom software development services

Need help beyond evaluating ibis? DEV.co is a software development agency offering software development services and web development for teams of every size. Our software developers and web developers build custom software, web applications, APIs, and open-source databases integrations — and maintain them long-term.

Evaluate Ibis for your multi-backend analytics strategy

Talk to DEV.co Schedule a Consultation