Open-Source Databases · noborus

trdsql

trdsql is a CLI tool that executes SQL queries directly on CSV, JSON, YAML, LTSV, and TBLN files without requiring a database setup. It supports PostgreSQL and MySQL SQL syntax and outputs results in multiple formats, making it useful for ad-hoc data transformation and analysis.

Talk to DEV.co Discuss Implementation

Source: GitHub — github.com/noborus/trdsql

2.2k

GitHub stars

Forks

Primary language

MIT

License (OSI-approved)

Key facts

Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.

Field	Value
Repository	noborus/trdsql
Owner	noborus
Primary language	Go
License	MIT — OSI-approved
Stars	2.2k
Forks	78
Open issues	14
Latest release	v1.2.3 (2026-05-29)
Last updated	2026-07-05
Source	https://github.com/noborus/trdsql

What trdsql is

Written in Go, trdsql uses SQLite3 (via go-sqlite3 or pure-Go sqlite) as its query engine and can connect to PostgreSQL or MySQL backends. It parses various text-based data formats into SQL-queryable tables, supporting jq expressions for JSON input and offering compression (gzip, bz2, zstd, lz4, xz) for output.

Quickstart

Get the trdsql source

Clone the repository and explore it locally.

terminalbash

1git clone https://github.com/noborus/trdsql.git2cd trdsql3# follow the project's README for install & configuration

Need it deployed, integrated, or customized instead? DEV.co ships production installs.

Best use cases

Ad-hoc CSV/JSON data transformation

Transform and filter CSV or JSON files with SQL joins and aggregations without loading into a database.

Data pipeline preprocessing

Use as a lightweight ETL step in shell scripts or data workflows to normalize multiple input formats into standardized outputs.

SQL-driven data format conversion

Convert between CSV, JSON, YAML, LTSV, and other formats using SELECT queries, with conditional logic and reformatting.

Implementation considerations

NULL handling must be explicitly configured per run (specify -inull and -onull for non-JSON formats); default is empty string = NULL.
Column name and type inference happen during parsing; inconsistent columns in input may cause silent truncation or errors.
CGO dependency on go-sqlite3 can complicate cross-compilation; use CGO_ENABLED=0 with pure-Go sqlite for portability but expect ~10-20% performance penalty.
Memory usage grows linearly with input file size; pre-filter or limit rows with -ilr to avoid OOM on large files.
jq expressions for JSON input require jq syntax familiarity; complex transformations may be easier in post-processing scripts.

When to avoid it — and what to weigh

Large-scale data processing — In-memory SQLite execution limits practical file sizes; not suitable for multi-gigabyte datasets or production data warehouses.
Real-time streaming ingestion — Designed for batch file processing; does not support real-time event streams or incremental updates.
Complex ACID transactions — Single-file execution context provides no multi-table transactions, distributed commits, or rollback semantics.
Proprietary or binary formats — Requires human-readable text input (CSV, JSON, YAML); cannot directly query Avro, Parquet, Protocol Buffers, or databases without export.

License & commercial use

MIT License: permissive, allows commercial use, modification, and distribution with inclusion of license and copyright notice. No patent grant clause.

MIT is a permissive OSI-approved license. Commercial use is legally permitted. No additional licensing fees or restrictions. Verify compliance with your own legal counsel for liability and warranty disclaimers implicit in MIT.

DEV.co evaluation signals

Editorial assessment — not user reviews. Directional, with an explicit confidence level.

Signal	Assessment
Maintenance	Active
Documentation	Adequate
License clarity	Clear
Deployment complexity	Low
DEV.co fit	Good
Assessment confidence	High

Security considerations

No embedded secrets management. Executes user-supplied SQL; no sandboxing—malformed queries or SQL injection via user input is possible if input is untrusted. File-based operation avoids network exposure. No built-in audit logging. Review SQL context if handling untrusted input.

Alternatives to consider

q (harelba/q)

Similar CLI for CSV/TSV queries using SQLite. Lightweight, but less format variety and no PostgreSQL/MySQL syntax support.

textql (dinedal/textql)

Older CLI for CSV→SQL; actively superseded by trdsql. Smaller feature set and less maintained.

DuckDB CLI

Modern OLAP engine with broader format support (Parquet, JSON, CSV) and faster performance on large files. More heavyweight, better for analytics.

Software development agency

Build on trdsql with DEV.co software developers

trdsql is ideal for scripted data transformation and ad-hoc file queries. Explore its CLI and Docker image to integrate into your pipeline.

Talk to DEV.co

Related open-source tools

Surfaced by semantic similarity across the DEV.co open-source index.

sq libsql prql

Related on DEV.co

Explore the category and the services that help you build with it.

Open-Source Databases API Development Custom Software Development DevOps & Data Pipelines Talk to DEV.co

trdsql FAQ

Can trdsql query files > 10 GB?

Theoretically yes, but practical limits depend on available RAM (entire table loaded into SQLite in-memory). For gigabyte+ files, DuckDB or a data warehouse is more appropriate. Use -ilr to limit row reads if memory is tight.

Does trdsql support real-time tailing of log files?

No. It is a batch query tool for static files. Use tools like `tail -f | trdsql -` for streaming, but expect memory issues and lack of transaction semantics.

Can I use trdsql in Docker without CGO?

Yes. Build with CGO_ENABLED=0 and use the pure-Go sqlite library. This simplifies multi-arch deployments but reduces performance.

What is the output if a NULL value is encountered in CSV input?

By default, empty strings are NULL; NULL is output as empty string unless -onull is specified. For JSON, null remains null and is treated as SQL NULL.

Software development & web development with DEV.co

Adopting trdsql is usually one piece of a larger software development effort. As a software development agency, DEV.co provides software development services and web development expertise — pairing senior software developers and web developers with your team to design, build, and operate open-source databases software in production.

Need lightweight SQL processing for text data?

trdsql is ideal for scripted data transformation and ad-hoc file queries. Explore its CLI and Docker image to integrate into your pipeline.

Talk to DEV.co Schedule a Consultation