Open-Source Databases · machow

siuba

siuba is a Python library that brings R's dplyr-style syntax to pandas DataFrames and SQL databases. It lets you write the same data manipulation code that works on both local data and remote SQL sources without rewriting logic.

Talk to DEV.co Discuss Implementation

Source: GitHub — github.com/machow/siuba

1.2k

GitHub stars

Forks

Python

Primary language

MIT

License (OSI-approved)

Key facts

Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.

Field	Value
Repository	machow/siuba
Owner	machow
Primary language	Python
License	MIT — OSI-approved
Stars	1.2k
Forks	55
Open issues	107
Latest release	v0.4.4 (2023-09-19)
Last updated	2025-09-24
Source	https://github.com/machow/siuba

What siuba is

siuba provides verb-based data operations (select, filter, mutate, summarize, arrange, group_by) with lazy expression support via the `_` operator. It translates high-level operations into pandas calls or SQL queries, supporting PostgreSQL, Redshift, and SQLite backends.

Quickstart

Get the siuba source

Clone the repository and explore it locally.

terminalbash

1git clone https://github.com/machow/siuba.git2cd siuba3# follow the project's README for install & configuration

Need it deployed, integrated, or customized instead? DEV.co ships production installs.

Best use cases

Cross-source data analysis with unified syntax

Write exploratory analysis once that executes identically against local pandas DataFrames or remote SQL tables, minimizing refactoring when moving from prototyping to production queries.

R developer migration to Python

Teams familiar with dplyr can adopt Python for data work using familiar verb patterns and piping syntax (>> operator), reducing cognitive load and onboarding time.

Data pipeline construction with lazy evaluation

Build composable, chainable transformations that defer execution until needed, enabling query optimization at the SQL layer and efficient memory usage with large datasets.

Implementation considerations

Lazy expressions require understanding of deferred evaluation—expressions are built as ASTs and executed differently on pandas vs. SQL, which can lead to subtle behavioral differences if not documented.
SQL backend support is database-specific; ensure your target database (PostgreSQL, Redshift, SQLite) is tested and compatible before committing to production pipelines.
Testing infrastructure requires Docker (postgres service), so CI/CD setup must account for database initialization and cleanup overhead.
Debugging lazy queries can be opaque; the library provides preview output but full SQL or pandas operation visibility may require custom introspection.
Library is maintained by a single primary author with infrequent releases; dependency on timely security patches or compatibility updates with newer pandas/SQLAlchemy versions is uncertain.

When to avoid it — and what to weigh

Advanced statistical or ML workflows — siuba focuses on tabular transformation verbs; it lacks built-in support for statistical modeling, time-series operations, or machine learning pipelines. Use pandas + scikit-learn or polars for those needs.
Production systems with strict version guarantees — Latest release is v0.4.4 (Sept 2023) with 107 open issues and modest community size (1.2k stars). Stability and backwards compatibility beyond patch releases are not guaranteed.
Multi-database migrations with dialect variations — SQL support is limited to PostgreSQL, Redshift, and SQLite. Projects requiring Microsoft SQL Server, BigQuery, Snowflake, or complex dialect handling will face implementation gaps.
Teams without R or functional programming familiarity — The lazy expression model (`_.col.method()`) and pipe-first design may confuse teams accustomed to imperative pandas chaining. Requires upfront education and consistency enforcement.

License & commercial use

siuba is released under the MIT License, which is a permissive OSI-approved license allowing commercial use, modification, and redistribution with minimal restrictions.

MIT License permits unrestricted commercial use. However, no warranty or liability disclaimers apply, so conduct independent security and compatibility testing before deploying in production systems. Verify that dependencies (pandas, SQLAlchemy, database drivers) also meet your commercial licensing requirements.

DEV.co evaluation signals

Editorial assessment — not user reviews. Directional, with an explicit confidence level.

Signal	Assessment
Maintenance	Moderate
Documentation	Adequate
License clarity	Clear
Deployment complexity	Moderate
DEV.co fit	Good
Assessment confidence	Medium

Security considerations

No explicit security audit or vulnerability disclosure process is mentioned. SQL backend relies on SQLAlchemy parameterization to prevent injection; verify SQLAlchemy versions are current. Lazy expressions are built as ASTs and executed on user data—review expression translation for any unintended data exposure. No built-in audit logging or field-level access controls.

Alternatives to consider

polars

Modern Rust-backed DataFrame library with lazy evaluation, SQL-like syntax, and broader backend support (Parquet, CSV, database connectors). Superior performance and concurrent maintenance. Steeper learning curve if coming from dplyr.

pandas + SQLAlchemy ORM

Mature, widely-adopted combination with extensive documentation and community support. More verbose but offers finer control and predictable behavior. No unified syntax layer, requiring separate code paths for pandas vs. SQL.

ibis

Python dataframe algebra with deferred execution and multi-backend support (Pandas, DuckDB, BigQuery, Snowflake, etc.). More abstract and backend-agnostic than siuba; steeper curve but greater flexibility for enterprise database ecosystems.

Software development agency

Build on siuba with DEV.co software developers

Evaluate siuba for your team's data analysis needs. Test SQL backend compatibility with your databases and assess learning curve impact before adopting in production pipelines.

Talk to DEV.co

Related open-source tools

Surfaced by semantic similarity across the DEV.co open-source index.

Medoo calcite clover

Related on DEV.co

Explore the category and the services that help you build with it.

Open-Source Databases AI development API development Custom software Talk to DEV.co

siuba FAQ

Can I use siuba with my existing SQLAlchemy models?

Partial support via tbl() function, but integration is not explicitly documented. Recommend prototyping with raw table names first, then reverse-engineer from examples if direct ORM model binding is required.

What happens if a SQL query fails or returns unexpected results?

siuba does not provide built-in debugging tools for generated SQL. Access the underlying SQLAlchemy query object via .query attribute (if exposed) or enable SQLAlchemy echo mode to inspect generated SQL. Error messages may be opaque.

Is siuba suitable for real-time or streaming data?

No. siuba is designed for batch tabular transformations with lazy evaluation. For streaming or real-time analytics, use Faust, Kafka Streams, or Apache Spark Structured Streaming instead.

What databases other than PostgreSQL, Redshift, and SQLite are supported?

Unknown. Documentation does not list other database backends. Review source code or file an issue if your database is required; adding support likely requires custom dialect translation.

Work with a software development agency

DEV.co helps companies turn open-source tools like siuba into production software. Our software development services cover the full lifecycle — architecture, web development, integration, and maintenance — delivered by software developers and web developers who ship. Engage our software development agency to implement or customize it for your open-source databases stack.

Ready to unify your data workflows?

Evaluate siuba for your team's data analysis needs. Test SQL backend compatibility with your databases and assess learning curve impact before adopting in production pipelines.

Talk to DEV.co Schedule a Consultation