agentops
AgentOps is a Python SDK that provides observability, monitoring, and debugging for AI agents. It tracks LLM costs, captures agent execution flows, and integrates with popular frameworks like CrewAI, LangChain, and OpenAI Agents SDK.
Key facts
Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.
| Field | Value |
|---|---|
| Repository | AgentOps-AI/agentops |
| Owner | AgentOps-AI |
| Primary language | Python |
| License | MIT — OSI-approved |
| Stars | 5.7k |
| Forks | 604 |
| Open issues | 170 |
| Latest release | 0.4.21 (2025-08-29) |
| Last updated | 2026-06-25 |
| Source | https://github.com/AgentOps-AI/agentops |
What agentops is
MIT-licensed Python package offering session replay, execution graphs, LLM cost tracking, and decorator-based instrumentation for agent workflows. Supports async/await, multi-agent architectures, and self-hosted deployment via dashboard and API backend.
Get the agentops source
Clone the repository and explore it locally.
git clone https://github.com/AgentOps-AI/agentops.gitcd agentops# follow the project's README for install & configurationNeed it deployed, integrated, or customized instead? DEV.co ships production installs.
Best use cases
Implementation considerations
- Decorator-based instrumentation (@session, @agent, @operation, @workflow) supports nested spans and async functions; review nesting semantics to avoid double-counting costs.
- API key management required for cloud backend; self-hosted deployment adds infrastructure complexity (see app/README.md for setup).
- Framework-specific integrations vary: CrewAI is tightly integrated (2-line init), while custom or newer frameworks may require manual instrumentation via decorators.
- Cost tracking is provider-aware (OpenAI, Anthropic, etc.); custom or private LLM endpoints may require custom cost mapping.
- 170 open issues suggest active development; check GitHub issues for known limitations in your target framework version.
When to avoid it — and what to weigh
- Non-Python Agents (Primary Gap) — AgentOps is Python-first; while TypeScript support exists for OpenAI Agents SDK, broader Node.js/JavaScript agent ecosystems have limited or no native integration.
- Offline-Only Requirements — The SDK requires connectivity to AgentOps cloud or self-hosted backend for session replay and dashboarding; local-only observability is not a design goal.
- Zero External Dependencies — The SDK introduces observability infrastructure dependencies; if your agent must run completely standalone with no external SDKs, this adds operational overhead.
- Real-Time Streaming Analytics — AgentOps focuses on post-hoc replay and session analytics; real-time streaming dashboards or low-latency event processing are not explicitly supported.
License & commercial use
MIT License (permissive, OSI-approved). Allows commercial use, modification, distribution, and private use with proper attribution. No restrictions on proprietary use or closed-source derivatives.
MIT license explicitly permits commercial use. AgentOps SDK itself is open source, but the hosted cloud service (dashboard, API, cost tracking backend) is a commercial offering. Self-hosting avoids cloud vendor lock-in but requires operational overhead.
DEV.co evaluation signals
Editorial assessment — not user reviews. Directional, with an explicit confidence level.
| Signal | Assessment |
|---|---|
| Maintenance | Active |
| Documentation | Strong |
| License clarity | Clear |
| Deployment complexity | Low |
| DEV.co fit | Strong |
| Assessment confidence | High |
SDK transmits session data (LLM calls, costs, metadata) to AgentOps cloud or self-hosted backend; sensitive payloads require review of what is recorded. API key must be protected in environment variables. Self-hosted deployments inherit security posture of chosen cloud provider and database. No explicit security audit, penetration test, or SOC 2 certification mentioned in data.
Alternatives to consider
LangSmith (LangChain)
Tightly integrated with LangChain agents; stronger real-time monitoring and dataset management; narrower framework support outside LangChain ecosystem.
Weights & Biases (W&B Weave)
General-purpose ML observability with agent tracing; broader ML context (training, evaluation, registry); less agent-framework-specific.
OpenTelemetry + Custom Backend
Vendor-agnostic, self-hosted observability; requires more engineering effort to instrument agents and build dashboards; maximum control and privacy.
Build on agentops with DEV.co software developers
Get started in 2 lines of code. Track costs, replay sessions, and debug agents across CrewAI, LangChain, OpenAI, and more.
Talk to DEV.coRelated open-source tools
Surfaced by semantic similarity across the DEV.co open-source index.
Related on DEV.co
Explore the category and the services that help you build with it.
agentops FAQ
Can I use AgentOps with proprietary/private LLMs?
Is the full AgentOps dashboard open source?
Does AgentOps work with non-CrewAI agents?
What data is sent to AgentOps cloud servers?
Software developers & web developers for hire
DEV.co helps companies turn open-source tools like agentops into production software. Our software development services cover the full lifecycle — architecture, web development, integration, and maintenance — delivered by software developers and web developers who ship. Engage our software development agency to implement or customize it for your ai frameworks stack.
Monitor Your AI Agents with AgentOps
Get started in 2 lines of code. Track costs, replay sessions, and debug agents across CrewAI, LangChain, OpenAI, and more.