AI Frameworks · instavm

clickclickclick

ClickClickClick is a Python framework that enables autonomous interaction with Android devices and desktop computers using various LLMs (local or remote). It breaks tasks into planning and element-finding phases, supporting models from OpenAI, Google Gemini, and Ollama.

Talk to DEV.co Discuss Implementation

Source: GitHub — github.com/instavm/clickclickclick

692

GitHub stars

Forks

Python

Primary language

MIT

License (OSI-approved)

Key facts

Objective fields from the source. Values we can't verify are shown as “Unknown” rather than guessed.

Field	Value
Repository	instavm/clickclickclick
Owner	instavm
Primary language	Python
License	MIT — OSI-approved
Stars	692
Forks	85
Open issues	3
Latest release	v0.3.0 (2024-12-26)
Last updated	2026-03-17
Source	https://github.com/instavm/clickclickclick

What clickclickclick is

The framework implements a two-stage architecture: a planner LLM decomposes tasks into navigation steps, and a finder LLM locates UI elements via vision models to execute actions. It exposes CLI, REST API, Gradio web interface, and Python API, requiring ADB for Android device communication.

Quickstart

Get the clickclickclick source

Clone the repository and explore it locally.

terminalbash

1git clone https://github.com/instavm/clickclickclick.git2cd clickclickclick3# follow the project's README for install & configuration

Need it deployed, integrated, or customized instead? DEV.co ships production installs.

Best use cases

Automating repetitive mobile app workflows

Execute structured tasks like drafting emails, filling forms, or navigating apps autonomously. Particularly useful for testing, data entry, and workflow automation on Android devices where traditional UI automation is limited.

Cross-platform browser automation with vision intelligence

Autonomous web scraping, task completion, and navigation using visual understanding rather than DOM parsing. Handles dynamic content, complex UIs, and scenarios where CSS/XPath selectors are fragile.

Local-first AI agent infrastructure for organizations handling sensitive data

Deploy with Ollama and local vision models to avoid external API calls. Useful for enterprises processing confidential information on isolated networks or devices.

Implementation considerations

ADB must be installed and configured on the host machine for Android device control; no built-in error handling for missing or misconfigured ADB is documented.
API key management for OpenAI/Gemini required; projects using local Ollama avoid external dependencies but trade off accuracy (qwen3.5 documented as basic navigator, poor UI element detection).
Image quality parameter (1–100) trades processing cost vs. accuracy; README suggests --image-quality=45 for Ollama but no systematic guidance on tuning for other models.
Two-stage architecture (planner + finder) means cost/latency scales with task complexity; no caching, batching, or multi-turn context preservation documented.
Setup requires manual YAML configuration and environment variable export; no infrastructure-as-code or CI/CD integration patterns documented.

When to avoid it — and what to weigh

Requiring guaranteed low-latency execution — LLM inference (planner) and vision model processing (finder) introduce multi-second latencies per decision cycle, making real-time automation infeasible.
Complex multi-step workflows with tight interdependencies — The framework is early-stage and experimental; task planning may fail on chains requiring conditional branching, error recovery, or context-dependent decisions across many steps.
High-volume production deployments without in-house model tuning — Documented performance variance across models (qwen3.5 unreliable as finder, Gemini 3.1 Flash-Lite recommended); tuning and fallback strategies essential for reliability at scale.
Applications where task failure cannot be tolerated — README explicitly states 'highly experimental' and 'use at your own risk.' No guarantees on success rates, and vision-based element detection can fail silently or produce unintended actions.

License & commercial use

Licensed under MIT (permissive open-source license). No copyleft obligations, modifications permitted, redistribution allowed with license notice.

MIT license permits commercial use, including proprietary modifications and redistribution. However, the framework is marked 'highly experimental.' Users assume full risk for production deployments. Requires review of liability and support expectations, as no commercial backing or SLA is evident.

DEV.co evaluation signals

Editorial assessment — not user reviews. Directional, with an explicit confidence level.

Signal	Assessment
Maintenance	Active
Documentation	Adequate
License clarity	Clear
Deployment complexity	Moderate
DEV.co fit	Good
Assessment confidence	Medium

Security considerations

No explicit security audit, threat model, or hardening guidelines documented. REST API has no authentication layer (localhost-only default). Vision models process screenshots containing potentially sensitive UI data; users must assess data residency and model privacy. ADB requires device trust relationships. No input validation or injection attack prevention documented. Local model path (Ollama) avoids external API exposure but shifts risk to local infrastructure.

Alternatives to consider

Claude/Anthropic Computer Use API

Closed-source, fully managed, proprietary model with higher vision accuracy. Higher cost, vendor lock-in, no local deployment option; Anthropic's official tool but less flexible.

OpenAI Operator / GPT-4o vision API

Managed service with production-grade APIs, strong vision performance, higher cost. No open-source control, API-dependent, no local inference; better for teams prioritizing reliability over cost.

UI Automator / Appium + custom LLM agents

Mature open-source mobile automation frameworks; can be extended with LLM vision modules. Steeper learning curve, requires more integration work, but proven stability for specific platforms (Android, iOS).

Software development agency

Build on clickclickclick with DEV.co software developers

ClickClickClick offers a flexible, open-source foundation for autonomous agent development. Assess your use case, model costs, and reliability tolerances—then prototype on Android or desktop before scaling.

Talk to DEV.co

Related open-source tools

Surfaced by semantic similarity across the DEV.co open-source index.

autoclip openagi llama_index

Related on DEV.co

Explore the category and the services that help you build with it.

AI Frameworks AI Application Development API Development Custom Software Development Talk to DEV.co

clickclickclick FAQ

Can I run this entirely offline without external APIs?

Yes, using Ollama with local vision models (e.g., qwen3.5). However, README notes qwen3.5 is unreliable as a finder; best results require Gemini 3.1 Flash-Lite, which needs a Google Cloud API key.

What are the main failure modes documented?

Vision model failures in element detection, planner task decomposition errors, and ADB connectivity issues. README does not provide systematic error recovery or retry logic; failures are not gracefully reported.

Does this support iOS or only Android?

README mentions Android and macOS (osx platform flag). iOS support not mentioned; unclear if macOS covers all desktop workflows or only native apps.

What's the cost of running this with remote models?

Depends on chosen models. Gemini 3.1 Flash-Lite and GPT-4o Vision incur per-image/request charges. No cost estimation tool or pricing comparison provided; users must calculate based on task complexity and image volume.

Software development & web development with DEV.co

Need help beyond evaluating clickclickclick? DEV.co is a software development agency offering software development services and web development for teams of every size. Our software developers and web developers build custom software, web applications, APIs, and ai frameworks integrations — and maintain them long-term.

Ready to automate UI workflows with AI vision?

Talk to DEV.co Schedule a Consultation