AI Frameworks · sigoden

aichat

AIChat is a unified CLI tool for interacting with 20+ LLM providers (OpenAI, Claude, Gemini, Ollama, etc.) through a single interface. It offers shell command generation, interactive REPL chat, RAG document integration, function calling, and AI agents—all in Rust with an optional lightweight HTTP server.

Talk to DEV.co Discuss Implementation

Source: GitHub — github.com/sigoden/aichat

10.2k

GitHub stars

707

Forks

Rust

Primary language

Apache-2.0

License (OSI-approved)

Key facts

Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.

Field	Value
Repository	sigoden/aichat
Owner	sigoden
Primary language	Rust
License	Apache-2.0 — OSI-approved
Stars	10.2k
Forks	707
Open issues	89
Latest release	v0.30.0 (2025-07-06)
Last updated	2026-02-23
Source	https://github.com/sigoden/aichat

What aichat is

Rust-based CLI supporting multi-provider LLM inference with features including streaming, role-based prompting, session persistence, RAG via embeddings/reranking APIs, function calling for tool integration, and a built-in HTTP server exposing OpenAI-compatible chat/embeddings endpoints. Supports local models (Ollama) and cloud providers via unified configuration.

Quickstart

Get the aichat source

Clone the repository and explore it locally.

terminalbash

1git clone https://github.com/sigoden/aichat.git2cd aichat3# follow the project's README for install & configuration

Need it deployed, integrated, or customized instead? DEV.co ships production installs.

Best use cases

Developer productivity at the terminal

Generate shell commands from natural language descriptions, execute them interactively, and maintain chat context across sessions without leaving the CLI.

Multi-LLM evaluation and comparison

Test and compare outputs across 20+ LLM providers using the Arena feature or side-by-side CLI queries without vendor lock-in.

Lightweight LLM API proxy for teams

Deploy the built-in HTTP server to provide OpenAI-compatible chat/embeddings/rerank APIs on-premises, with configurable per-provider routing and RAG integration.

Implementation considerations

API keys for each provider must be managed via environment variables or config files; no built-in secrets management—integrate with HashiCorp Vault, AWS Secrets Manager, or similar for production.
RAG feature requires manual embedding model selection and vector storage setup; no turnkey local vector DB (e.g., no bundled sqlite-vec or Milvus).
Function calling and AI agents depend on well-formed tool schemas; malformed definitions will fail silently or produce unhelpful LLM responses.
Session and role data stored as local JSON/YAML files; scaling to multi-user deployments requires centralized configuration management.
HTTP server exposes API without authentication by default; must front with API gateway, reverse proxy with auth, or run on isolated network.

When to avoid it — and what to weigh

Need enterprise GUI with RBAC — AIChat is CLI and REPL focused; web UI is basic (playground/arena). Lacks built-in user management, audit logging, or role-based access controls for team environments.
Require offline operation without setup — Local inference (Ollama) requires separate model downloads and GPU/resource provisioning. No batteries-included local model bundling.
Heavily regulated environments without audit trails — No documented compliance features (SOC2, FedRAMP, etc.). Conversation logs are local files; data residency and retention policies are user-managed.
Expect production-grade stability guarantees — v0.30.0 indicates pre-1.0 status. API/config stability is not guaranteed; breaking changes may occur between minor releases.

License & commercial use

Dual licensed under Apache License 2.0 or MIT. Both are permissive OSI-approved licenses allowing commercial use, modification, and distribution with minimal restrictions (Apache 2.0 includes patent grant; MIT is simpler).

Apache 2.0 and MIT are both permissive and widely accepted for commercial software. No royalties or special commercial licensing required. However, review your legal requirements for patent clauses (Apache 2.0 provides explicit patent protection) and ensure compliance with Apache's NOTICE file and MIT's attribution terms when bundling.

DEV.co evaluation signals

Editorial assessment — not user reviews. Directional, with an explicit confidence level.

Signal	Assessment
Maintenance	Active
Documentation	Strong
License clarity	Clear
Deployment complexity	Low
DEV.co fit	Strong
Assessment confidence	High

Security considerations

API keys must be stored securely outside the codebase (environment variables, config file permissions). Local session/role files may contain sensitive conversation data; no built-in encryption at rest. HTTP server exposes API without authentication—operator must implement TLS, API key validation, or firewall rules. Tool execution (shell commands, function calls) runs with user privileges; no sandboxing. No public security audit available; Rust memory safety mitigates some categories of vulnerabilities, but business logic flaws remain possible. Data sent to external LLM providers is subject to their privacy policies.

Alternatives to consider

LangChain CLI / LangServe

Framework-based approach with richer Python ecosystem, multi-agent orchestration, and enterprise observability; steeper learning curve and heavier dependencies than a single Rust binary.

Continue (IDE Plugin)

IDE-integrated LLM assistant with multi-provider support and codebase awareness; better for editor-centric workflows but lacks independent CLI and session persistence.

OpenRouter CLI / API Gateway

Simple unified LLM router with no tool/RAG features; minimal setup but less powerful for local agents and function calling.

Software development agency

Build on aichat with DEV.co software developers

Evaluate AIChat for your use case: single-user productivity, multi-provider comparison, or team API proxy. Review security requirements (auth, secrets, data residency) and test with your provider configuration before production deployment.

Talk to DEV.co

Related open-source tools

Surfaced by semantic similarity across the DEV.co open-source index.

Langchain-Chatchat deepchat bedrock-chat

Related on DEV.co

Explore the category and the services that help you build with it.

AI Frameworks AI Application Development API Development Cloud Deployment Talk to DEV.co

aichat FAQ

Can I use AIChat without internet (offline)?

Partially. Local inference via Ollama (Llama, Mistral, etc.) works offline if models are pre-downloaded and Ollama is running. Cloud provider models (OpenAI, Claude) require internet and API keys.

Does AIChat support streaming responses?

Yes. Streaming is built-in for both CLI and HTTP server endpoints; responses are output incrementally to reduce latency.

How do I run AIChat in production for multiple users?

Use the `--serve` mode to expose an HTTP API, then front it with an authentication layer (API Gateway, Reverse Proxy with Auth) and centralize configuration. Session/role management is currently file-based; consider external config management for scale.

Is conversation data encrypted or logged by default?

No. Sessions are stored locally as unencrypted JSON/YAML files. Data sent to external LLM providers follows their privacy policy. Operator is responsible for securing local files and choosing privacy-respecting providers.

Software development & web development with DEV.co

Need help beyond evaluating aichat? DEV.co is a software development agency offering software development services and web development for teams of every size. Our software developers and web developers build custom software, web applications, APIs, and ai frameworks integrations — and maintain them long-term.

Ready to streamline LLM workflows?

Talk to DEV.co Schedule a Consultation