Computer use & desktop agents
Computer use and desktop agents control a desktop or operating system directly, reading the screen and issuing clicks and keystrokes rather than calling APIs. Open Interpreter turns plain language into executed computer actions, while OmniParser parses screenshots into structured elements so vision-only agents can act on what they see.
727 agents · most starred first
Top 100 of 727
A natural language interface for computers
The Open-Source Multimodal AI Agent Stack: Connecting Cutting-Edge AI Models and Agent Infra
Scale computer-use 2.0 with open-source drivers, cross-OS sandboxes, and benchmarks for training, evaluation, and data generation.
Open-source Grok Bot alternative with a virtual machine that bots can use
Computer use CLI for AI agents
A simple screen parsing tool towards pure vision based GUI agent
YC (S26) | Screenpipe finds the work worth automating | Continuously record computer work locally, discover recurring workflows, and hand the result to any agent you already use.
A local MCP server that gives your coding agent a desktop toolbox: downloads from YouTube, torrents and every site yt-dlp supports, media conversion, files, chats and system cleanup. One line connects Claude Code, Codex, Cursor or Gemini CLI, and sensitive actions wait for your click. Free and open source for Windows, macOS and Linux.
Self-evolving agent: grows skill tree from 3.3K-line seed, achieving full system control with 6x less token consumption
The open, self-hostable alternative to closed AI bots. Production-ready, fully customizable, and plugs into every stack: any agent, any model, any tool.
Agent S: the state-of-the-art open source computer use agents
Pioneering Automated GUI Interaction with Native Agents
Bytebot is a self-hosted AI desktop agent that automates computer tasks through natural language commands, operating within a containerized Linux desktop environment.
A framework to enable a multimodal model to operate a computer.
UFO³: Weaving the Digital Agent Galaxy
MagenticLite is an experimental agent that works across the browser and local file system
Give your agent a computer 👾
On-screen aware AI assistant for your desktop. Uses current app context, multiple LLMs, and MCP tools to help you act across apps.
Fara1.5 – A family of frontier computer use agent models
A simple SWE-style browser+desktop agent framework that achieves SOTA results on long-horizon CUA tasks
Peekaboo is a macOS CLI & optional MCP server that enables AI agents to capture screenshots of applications, or the entire system, with optional visual question answering through local or remote AI models.
Agent-ready RPA suite with out-of-the-box automation tools. Built for individuals and enterprises.
An autonomous, context-aware AI desktop companion. Built with Python, featuring real-time screen vision, custom ONNX voice synthesis, active window tracking, and a dynamic floating UI with reactive facial expressions.
Like Manus, Computer Use Agent(CUA) and Omniparser, we are computer-using agents.AI-driven local automation assistant that uses natural language to make computers work by themselves
[NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments
Harness Anything - AI agent control hub: WPS, MS Office, Zotero, Photoshop, 47 CLI commands, 27 academic skills, SVG-to-PPTX
This is the official website for TuriX Computer-use-Agent
👾 Open Computer Use – Open-Source Alternative to Codex Computer Use
Mano-P: Open-source GUI-VLA agent for edge devices. #1 on OSWorld (specialized, 58.2%). Runs locally on Apple M4 Mac mini/MacBook — no data leaves your device.Mano-P 是一个开源 GUI-VLA 项目,支持在 Mac mini/MacBook 上或通过算力棒本地运行推理,实现纯视觉驱动的跨平台 GUI 自动化操作。数据完全本地处理,支持复杂多步骤任务规划与执行。
Control Any Computer Using LLMs.
The Cradle framework is a first attempt at General Computer Control (GCC). Cradle supports agents to ace any computer task by enabling strong reasoning abilities, self-improvment, and skill curation, in a standardized general environment with minimal requirements.
AI computer use powered by open source LLMs and E2B Desktop Sandbox
STEP-GUI: The top GUI agent solution in the galaxy. Developed by the StepFun-GELab team and powered by StepFun’s cutting-edge research capabilities.
👾 Open Computer Use – Open-Source Alternative to Codex Computer Use
Reliable AI Skill + MCP toolkit for agent-driven desktop CAD automation.
Browser and desktop automation with an MCP server for AI assistants. Record macros or write JavaScript to automate websites, extract data, and control desktop apps with OCR and image recognition.
OpenBitFun combines a high-performance agent runtime written in Rust with a polished desktop application. It pairs the depth of a Code Agent with open, general-purpose capabilities for work beyond software development.
Make Mac apps accessible for AI agents
Out-of-the-box (OOTB) GUI Agent for Windows and macOS
Learn how to use CUA (our Computer Using Agent) via the API on multiple computer environments.
For EHRs, payer portals, and any app your team clicks through. A person or an agent shows the task; OpenAdapt turns it into a local program that repeats the steps with no model calls, checks each save, and stops when something doesn't match.
The long-horizon computer-use harness. Run AI agents across desktop apps and the CLI for extended periods while preserving task state and making reliable progress on complex workflows. Features fresh-context execution, durable verified state, independent auditing, recoverable progress, and native Claude Code / Codex / OpenClaw integration.
[CVPR 2025] Open-source, End-to-end, Vision-Language-Action model for GUI Agent & Computer Use.
Agent Desktop gives any agent reliable computer use on the desktop. Built with Rust, it sees any app's real UI structure through OS accessibility trees and operates it — refs stay stable and actions stay safe to retry, instead of guessing from pixels.
The micro-agent orchestrator that monitors anything for you. Running completely locally. Open Source.
MAI-UI: Real-World Centric Foundation GUI Agents ranging from 2B to 235B
extremely fast web remote desktop streaming
An self-improving embodied conversational agent seamlessly integrated into the operating system to automate our daily tasks.
Ouroboros — self-creating AI agent. Born Feb 16, 2026.
Full computer-use for AI agents. Self-learning workflows. Native macOS. No screenshots required.
The Open-Source AI Agent OS inspired by OpenAI Dots, Meta Muse, OpenClaw, Hermes & Mem0. Autonomous 3-tier headhunting, Receipt & Verifier loops, scoped memory, and BYOM for Claude Code, Codex & Cursor.
Build, Evaluate, and Deploy GUI Agents — online RL training, standardized benchmarks, and real-device deployment in one framework.
The first open-source Artificial Narrow Intelligence generalist agentic framework Computer-Using-Agent that fully operates graphical-user-interfaces (GUIs) by using only natural language. Uses Visualization-of-Thought and Chain-of-Thought reasoning to elicit spatial reasoning and perception, emulates, plans and simulates synthetic HID interactions.
Computer use for about $0.0002 a step: OCR the screen, classify the next action with TypeSafe, click. macOS.
A browser-based desktop where AI Agent operates every app through natural language.
识屏 Shiping:截屏 + AI,看懂屏幕上的任何内容。代码、报错、题目、图表、外文资料,按快捷键截下图就有解答。可作为桌面工具常驻后台,随时框选识别。对话模式实时听懂问题、给出回答要点。可用作桌面工具、面试助手、考试助手、刷题助手。
AI agent using GPT-4V(ision) capable of using a mouse/keyboard to interact with web UI
Open, source-backed catalog of computer-use agents: reviewed papers, GUI models, datasets, benchmarks, tools, and reusable JSONL/SQLite data.
gradio region:us
Windows Agent Arena (WAA) 🪟 is a scalable OS platform for testing and benchmarking of multi-modal AI agents.
Amazon Nova Act is an AWS service for building and deploying highly reliable AI agents that automate UI-based workflows at scale.
Aiden — an autonomous AI agent and work engine built solo. It can operate your browser, terminal, files, apps, APIs, skills and tools, remember context, recover from failures, automate workflows, and verify the result. Open source · AGPL-3.0.
A fork of Anthropic Computer Use that you can run on Mac computers to give Claude and other AI models autonomous access to your computer.
Let your AI agents paint big arrows, boxes and text on your Mac screen. One CLI, click-through, gone by itself. Skill for Claude Code and Codex. MIT.
Surf is a computer use AI agent powered by OpenAI that interacts with a E2B's virtual desktop environment through natural language instructions
[NeurIPS 2025 Spotlight] OpenCUA: Open Foundations for Computer-Use Agents
Autonomous Android and computer use using any LLM (local or remote)
让 AI 实时看见你的屏幕:按变化量筛选帧、多帧拼进一张图,静止画面几乎不花 token
Desktop app to control your computer with AI using your terminal, browser, mouse & keyboard
Agent! now supports macOS 14.6 or later, Apple Silicon and Intel, latest release. The Best AI agent for Mac: drives any app via Accessibility, codes/builds in Xcode, automates AppleScript, JXA, Swift and SMAppService shell (user/root). 23 LLM providers, local/cloud, plus on-device Apple Intelligence and Jev guard. Auto-Pilot, MCP, Avatar.
Autonomous self-evolving agents. Vision-grounded layered memory and self-written skills for LLM agents that operate your computer.
Wayland - The AI Agent That Perceives. Reasons. Acts. Evolves.
Bops: a team of AI bots that run your business ops, each with its own computer, email and phone number. bops.bot
Mac Agent for macOS 26: the agentic AI harness for your Mac Desktop. Computer use, automation, scripting, coding, and more. Powered by 18+ providers across local and cloud LLMs.
Real-time multimodal desktop agent evolving toward a persistent AI OS interface (0.17 α).
Hardware mouse MCP server for AI agents on Windows x64. USB HID control and voice dictation; requires compatible uMouse hardware and firmware. 面向 AI 智能体的硬件鼠标 MCP:USB HID 控制与语音听写,需要兼容的 uMouse 设备与固件。
QuickDesk is the first AI-native remote desktop — an open-source, free application with a built-in MCP (Model Context Protocol) Server that lets any AI agent see and control remote computers.
Open Duet card feed: agents finish work, Magnus decides with one click
The model, data and code for the visual GUI Agent SeeClick
An AI Agent that interacts with Windows OS at GUI level.
The only general AI agent that does NOT requires extra API key, giving you full control on your local and remote MacOs from Claude Desktop App
This is the repo for the paper "OS Agents: A Survey on MLLM-based Agents for Computer, Phone and Browser Use" (ACL 2025 Oral).
Hermes Desktop - OS1 Edition: native macOS workspace for Hermes Agent on Orgo cloud computers and SSH hosts
An open-source stack for generalist computer-use agents, combining code execution, GUI interaction, and cross-device control.
A curated collection of resources, tools, and frameworks for developing GUI Agents.
OS-ATLAS: A Foundation Action Model For Generalist GUI Agents
One workspace for everything you do with AI: Claude Code, Codex and seven more coding agents side by side, one assistant that runs them, voice, and docs. Open source, on your computer, on your own plans and keys. Formerly Personal Jarvis.
Open-source resources on agents for computer use.
One agent. Every kind of work.
Open-source, local-first alternative to Cowork-style computer assistants: real PTY terminal ops, explicit approvals, JSONL audit logs. Windows + Linux + macOS. Model agnostic.
OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks
[NeurIPS'25] GUI-Actor: Coordinate-Free Visual Grounding for GUI Agents
Tactile: an accessibility-first operating layer for agents.
Native Ubuntu control center for agent usage, system utilities, media, clipboard, emoji and GNOME integration.
Open-sourced, Fast and Context-aware Action Grounding from GUI Instructions for GUI/Computer-use Agents
Local MCP server that gives any AI agent safe desktop control on Windows, macOS and Linux. Reads the screen as text and structure (accessibility + OCR) and acts by element name, with screenshots as the last resort. Multi-monitor aware, verifies every action, one safety gate.
Official repo for paper DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning.
[ICML2025] Aguvis: Unified Pure Vision Agents for Autonomous GUI Interaction
Turn ChatGPT into a local agent runtime. Operate your computer, use local tools, discover MCP servers, and delegate work to your own models through one MCP connection.
Reliable Automation Agents at Scale