Open-Source Observability · awslabs

flowgger

Flowgger is a lightweight, single-executable data collector written in Rust that reads logs from multiple input sources (UDP, TCP, TLS, Redis, stdin), parses them in various formats (JSON/GELF, LTSV, Cap'n Proto, RFC5424), and forwards them to downstream systems like Kafka or Graylog. It emphasizes speed, memory efficiency, and safety through Rust's memory guarantees without unsafe code.

Talk to DEV.co Discuss Implementation

Source: GitHub — github.com/awslabs/flowgger

882

GitHub stars

Forks

Rust

Primary language

BSD-2-Clause

License (OSI-approved)

Key facts

Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.

Field	Value
Repository	awslabs/flowgger
Owner	awslabs
Primary language	Rust
License	BSD-2-Clause — OSI-approved
Stars	882
Forks	60
Open issues	15
Latest release	0.3.2 (2024-05-17)
Last updated	2025-06-23
Source	https://github.com/awslabs/flowgger

What flowgger is

A standalone log ingestion pipeline written in Rust that decodes structured and semi-structured logs, validates input rigorously, and re-encodes into target formats for asynchronous delivery to Kafka, Graylog, or other collectors. Avoids JVM overhead and uses async I/O to handle high-volume message streams.

Quickstart

Get the flowgger source

Clone the repository and explore it locally.

terminalbash

1git clone https://github.com/awslabs/flowgger.git2cd flowgger3# follow the project's README for install & configuration

Need it deployed, integrated, or customized instead? DEV.co ships production installs.

Best use cases

High-volume syslog/RFC5424 forwarding to centralized logging

Efficiently parse and forward RFC5424 syslog messages from multiple sources to Graylog or Kafka with minimal latency and resource overhead compared to Java-based alternatives.

Lightweight log aggregation on constrained infrastructure

Deploy as a single binary on edge devices, containers, or VMs with strict memory/CPU budgets; no JVM or heavy dependencies required.

Multi-format log normalization and enrichment

Ingest JSON, LTSV, Cap'n Proto, and syslog in a single pipeline, normalize timestamps/fields, and forward to downstream analysis or storage systems.

Implementation considerations

Configuration is TOML-based; review flowgger.toml schema and the v0.3.0 breaking change from strftime to the `time` crate's format_description syntax.
Validate input protocol and format support matches your sources (stdin, UDP/TCP/TLS for intake; Kafka/Graylog/Redis for egress).
Deploy as a single binary; ensure your infrastructure can execute Rust binaries or use pre-built releases (verify availability for your OS/arch).
No unsafe code is used, but test the version you plan to deploy against your log volume and latency requirements.
Monitor the 15 open issues; assess if any block your use case or warrant a fork/patch.

When to avoid it — and what to weigh

Require extensive built-in transformations or field enrichment — Flowgger focuses on parsing and forwarding; complex conditional logic, lookups, or data enrichment require external orchestration or custom extensions.
Depend on a large ecosystem of plugins and integrations — The project has a narrower integration surface than Logstash or Fluentd; unsupported input/output formats may require code contributions.
Need commercial support or SLA guarantees — AWS Labs project with community-driven maintenance; no official commercial support tier documented.
Expect frequent feature releases and rapid evolution — Release cadence is infrequent (v0.3.2 in May 2024 after ~2 years); breaking changes (e.g., chrono→time) may require configuration updates.

License & commercial use

BSD 2-Clause license. Permissive OSI-approved license allowing commercial use, modification, and distribution with attribution and liability disclaimer.

BSD 2-Clause is permissive and does not restrict commercial use or proprietary deployment. However, AWS Labs maintains the project without formal commercial support. Ensure your legal and compliance teams review the license for your specific use case, and consider the implications of depending on a community-maintained AWS project for production log infrastructure.

DEV.co evaluation signals

Editorial assessment — not user reviews. Directional, with an explicit confidence level.

Signal	Assessment
Maintenance	Moderate
Documentation	Adequate
License clarity	Clear
Deployment complexity	Low
DEV.co fit	Good
Assessment confidence	High

Security considerations

Written in Rust without unsafe code, leveraging memory safety guarantees. Input validation is emphasized as a design goal. No known CVEs are evident from the data provided, but review dependency chain (especially after v0.3.0's migration from chrono to time due to RUSTSEC-2020-0071). TLS is supported for encrypted transport. Requires standard operational security practices: least-privilege service accounts, network segmentation, credential rotation, and regular dependency audits.

Alternatives to consider

Logstash (Elastic)

Full-featured log processing pipeline with extensive plugins; heavier resource footprint (requires JVM). Mature commercial support. Consider if you need advanced transformations or have existing Elastic Stack investment.

Fluentd / Fluent Bit

Lightweight, multi-format log forwarder; Fluent Bit is especially suited for edge/container workloads. More established plugin ecosystem. Consider if you prioritize ecosystem size over binary compactness.

Vector (Timber.io)

Modern Rust-based log/metric/trace collector with broad integrations and built-in transformations. Active commercial backing and rapid release cadence. Consider if you need richer data pipeline orchestration.

Software development agency

Build on flowgger with DEV.co software developers

Test Flowgger with your log volume and format to confirm performance and feature fit. Review the GitHub wiki and open issues for integration details. Contact Devco for guidance on architecture and production deployment.

Talk to DEV.co

Related open-source tools

Surfaced by semantic similarity across the DEV.co open-source index.

slog fluent-bit tracing

Related on DEV.co

Explore the category and the services that help you build with it.

Open-Source Observability DevOps Cloud Deployment API Development Talk to DEV.co

flowgger FAQ

Can Flowgger replace Logstash or Fluentd?

Partially. Flowgger excels at fast, low-overhead log parsing and forwarding for simple pipelines. It lacks Logstash's rich transformation language and Fluentd's plugin ecosystem. Evaluate your transformation needs before replacing.

What are the performance characteristics compared to Logstash?

README claims Flowgger is orders of magnitude faster than Logstash and Fluentd, but independent benchmarks are not provided in the data. Test with your log volume and format before assuming gains.

Is this suitable for production?

Yes, if your use case aligns with Flowgger's focus on log parsing and forwarding. It is battle-tested in AWS Labs environments, but verify maturity and maintenance pace match your SLA expectations.

How do I handle custom log formats?

Supported formats are JSON/GELF, LTSV, Cap'n Proto, and RFC5424. Custom formats require code contribution or fork. Evaluate this constraint before adoption.

Custom software development services

DEV.co is a software development agency delivering custom software development services to companies building on open source. Our software developers and web developers design, integrate, and ship production systems — spanning web development, APIs, AI, data, and cloud. If flowgger is part of your open-source observability roadmap, our team can implement, customize, migrate, and maintain it.

Ready to evaluate Flowgger for your logging pipeline?

Talk to DEV.co Schedule a Consultation