py-xiaozhi
py-xiaozhi is a Python-based open-source AI assistant framework enabling voice interaction, vision processing, and IoT device control across desktop and embedded platforms (Raspberry Pi, ESP32). It integrates MCP (Model Context Protocol) tools, offline wake-word detection, and supports Windows, macOS, Linux, and ARM architectures.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Repository | huangjunsen0406/py-xiaozhi |
| Owner | huangjunsen0406 |
| Primary language | Python |
| License | MIT — OSI-approved |
| Stars | 3.4k |
| Forks | 710 |
| Open issues | 1 |
| Latest release | v2.0.6 (2026-06-04) |
| Last updated | 2026-07-04 |
| Source | https://github.com/huangjunsen0406/py-xiaozhi |
What py-xiaozhi is
Built on asyncio with event-driven architecture, py-xiaozhi provides real-time Opus codec streaming (sub-20ms latency), Sherpa-ONNX offline speech recognition, multimodal vision-language integration, WebSocket/MQTT dual-protocol communication with TLS encryption, and a plugin-based MCP tool ecosystem (music, camera, weather, app management). Designed for edge computing with dependency injection and layered protocol abstraction.
Get the py-xiaozhi source
Clone the repository and explore it locally.
git clone https://github.com/huangjunsen0406/py-xiaozhi.gitcd py-xiaozhi# follow the project's README for install & configurationNeed it deployed, integrated, or customized instead? DEV.co ships production installs.
Best use cases
Implementation considerations
- Dependency management: Latest pip dependencies must be manually reinstalled after each update per README guidance; plan for testing cycle and breaking changes.
- Audio/hardware prerequisites: Requires microphone, speaker, stable internet connection; optional Sherpa-ONNX models for wake-word detection need downloading and storage (2GB+ total).
- Multi-protocol routing: Support for WebSocket, MQTT, and GPIO requires clear architectural decision on which transport fits your deployment environment and network topology.
- MCP tool customization: Extending the tool ecosystem requires understanding JSON-RPC 2.0 and the plugin system; base tools (music, camera, weather) may need adaptation to your hardware.
- Configuration hierarchy: Dot-notation config system allows dynamic updates; ensure environment and override layers are well-documented for ops and scaling.
When to avoid it — and what to weigh
- Requires proprietary LLM hosting or vendor lock-in — py-xiaozhi integrates with external AI services (Claude, GPT, Gemini via sponsorship APIs) but does not bundle commercial LLM licenses. Deployment still requires third-party API keys or self-hosted models; review data residency and commercial terms separately.
- Need guaranteed production SLA or enterprise support — This is a community-maintained open-source project with no formal enterprise support contract, SLA guarantees, or dedicated incident response. Use only where internal or community-driven support is acceptable.
- Low latency absolute requirement below 5ms — While the project targets sub-20ms latency in audio processing, network round-trips to external LLM APIs will typically exceed 100ms. Not suitable for real-time safety-critical or ultra-low-latency industrial control.
- Unsupported or legacy hardware with Python 3.9 or older — Minimum Python 3.10 required; targets modern AVX-capable CPUs and audio devices with 16kHz support. Older or obscure embedded platforms may require significant porting effort.
License & commercial use
MIT License (permissive). Allows commercial use, modification, distribution, and private use with attribution. No restrictions on proprietary derivative works or closed-source applications built on top. Sponsor acknowledgments are recommended but not legally required.
MIT is a permissive OSI-approved license compatible with most commercial use cases. You may build closed-source commercial products using py-xiaozhi without licensing restrictions from the framework itself. However: (1) external LLM API access requires separate commercial agreements with those vendors (OpenAI, Anthropic, Google, etc.); (2) Sherpa-ONNX and other bundled third-party libraries have their own licenses (verify ONNX Runtime, opus codec); (3) no warranty or indemnification from the project maintainer; (4) use of sponsorship API relays (GitDo.net, Factory.pub) requires their separate ToS. Requires review of supply chain and third-party terms.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Adequate |
| License clarity | Clear |
| Deployment complexity | Moderate |
| DEV.co fit | Strong |
| Assessment confidence | High |
Project implements WSS/TLS encryption for remote communication, device fingerprint recognition, and tool permission management. However: (1) security posture depends on correct TLS setup by deployer (certificate management, cipher selection); (2) local microphone/camera access is inherently privileged—verify wake-word detection and audio capture respect user consent; (3) MCP tool system allows arbitrary command execution (music/app/camera control)—restrict tool permissions in untrusted environments; (4) external LLM API traffic exposes audio/vision data to third parties—review privacy implications; (5) no formal security audit evident; (6) Python runtime and dependencies (asyncio, PySide6, etc.) inherit standard Python supply-chain risks. Suitable for trusted networks and internal deployments; not recommended for multi-tenant or high-security contexts without additional hardening.
Alternatives to consider
Rasa + Mycroft
Rasa provides NLU/dialogue management; Mycroft offers embedded voice assistant stack. Better for conversational NLU-heavy workflows, but less tightly integrated for vision/IoT robotics than py-xiaozhi.
Open WebRTC + PipeWire + custom agent loop
Lower-level building blocks (audio codec + IPC + custom orchestration). Offers maximum flexibility but requires significant engineering to replicate py-xiaozhi's MCP integration and multimodal abstractions.
Home Assistant + ESPHome + voice add-ons
Mature home automation platform with voice integration via Nabu Casa or local TTS/STT. Simpler for smart-home use cases but less suitable for robotics, custom AI agents, or deployments beyond home automation.
Build on py-xiaozhi with DEV.co software developers
py-xiaozhi provides the foundation for embedded AI assistants with offline wake-word detection, vision integration, and hardware control. Explore the GitHub repo, review architecture docs, and start with the Bilibili video tutorial. For production deployments involving robotics or complex integrations, consider consulting on custom MCP tool development and LLM API selection.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
py-xiaozhi FAQ
Can I use py-xiaozhi without external LLM APIs (fully offline)?
Does py-xiaozhi work on Windows 10, or do I need Windows 11?
How do I deploy to Raspberry Pi?
Is the MCP tool ecosystem complete, or do I need to write custom tools?
Software development & web development with DEV.co
From first prototype to production, DEV.co delivers software development services around tools like py-xiaozhi. Our software development agency staffs experienced software developers and web developers for custom software development, web development, integrations, and ongoing support across mcp servers and beyond.
Ready to build intelligent voice-controlled IoT systems?
py-xiaozhi provides the foundation for embedded AI assistants with offline wake-word detection, vision integration, and hardware control. Explore the GitHub repo, review architecture docs, and start with the Bilibili video tutorial. For production deployments involving robotics or complex integrations, consider consulting on custom MCP tool development and LLM API selection.