vercel-labs/agent-desktop

This page introduces an external open-source repository. It is not an HDATF product.

What it does

This Rust CLI and C library lets AI agents read and operate desktop apps through OS accessibility trees rather than guessed coordinates. Automation developers select an app and window and receive a JSON snapshot with references for actionable elements. An agent requests clicks or text entry using those references, then takes another snapshot to check the result. Default actions use accessibility APIs without moving the pointer or bringing the app forward; physical input needs a separate mode. Entry points include npm packages and platform release archives, with Accessibility permission on macOS and a graphical session with AT-SPI on Linux. Windows callers must also consider process integrity levels and target-app conditions. Stale references and ambiguous targets return errors and require fresh observation. This is not a complete assistant with its own model or planner, and the native CLI requires no remote model, unlike the optional JEV helper. Snapshots, clipboard contents and screenshots may expose private information, and capabilities differ by OS.

License

Apache-2.0 Permissive, with a patent grant. Commercial use is allowed; keep the notices and state your changes.

More in this category

• CopilotKit/OpenBot
  OpenBot is a beta AI-coworker platform template intended for companies to adapt and operate in their own environments. Rather than a finished hosted service or model, it supplies a React interface, API server, PostgreSQL, browser computers and AG-UI agent connections. Conversations and task requests flow to agents and onward to browser or file operations, result interfaces and audit records. The server gateway is designed to resolve targets, evaluate policy, write an audit record and only then forward permitted actions. Deny rules take precedence in the policy engine, but the shipped startup policy permits everything, so installation alone does not establish restrictive business permissions. Setup needs Docker, Bun, database and encryption configuration, separate model credentials, and a configured CopilotKit Intelligence connection and entitlement. Conversations and memory integrate with Intelligence, while app connectors and MCP tools require their own credentials and grants. Default single-user mode treats requests as one administrator, so authentication, roles and network isolation must be configured before others gain access. Browser logins and components controlling Docker require protection, and operators remain responsible for completing and validating the template for actual business use.
• jacke-pp/open-yoki
  open-yoki is a self-hosted AI livestream host foundation for operators who want chat responses, voice and an animated avatar in one application. It accepts website comments or externally forwarded room events and produces spoken replies, subtitles, avatar behavior and a livestream feed. A fast language model handles conversation, while an optional longer-running agent keeps diary and persona files and prepares social posts. Local memory stores visitor notes and show summaries, and an outbox records the reported results of external posting. Headless Chrome renders the broadcast scene and ffmpeg sends it through RTMP, a protocol for sending live video to a platform. The documented starting environment uses Node.js and ffmpeg, with an x86_64 Ubuntu server for full deployment and services such as OpenRouter or DeepSeek, Volcengine voice and RTC, and Vidu for live avatar operation. Operators must provide authorized images, recordings, credentials, platform comment ingestion and each broadcast's streaming details because the original host's assets and accounts are not included. The release does not contain wallet or trading functions, and its one-server architecture does not mean offline operation or a fully automatic platform login and room-opening workflow.
• wangzan101/Mika
  Mika is a personal Android assistant that turns natural-language requests into app and web operations and shows results with execution history. It opens apps on a separate background display and reads interfaces through accessibility data, OCR and an optional vision model. Its built-in browser reads web material and handles files, while supported devices provide a Linux sandbox per conversation for computation and document processing. Installing the APK alone does not enable phone control; debugging pairing, permissions and a tool-capable model connection must be configured. Installation requires Android 11 or later, phone-app tasks require Android 13 or later, Android 14 is recommended, and the sandbox is arm64-only. Models connect through separate APIs or supported login flows, and screen content can reach the selected provider, so on-device execution does not mean fully offline processing. Login and verification challenges are handed to a person, but the default asking mode does not request approval for every message send or deletion. Autonomous mode also reduces payment-related confirmation, a distinction that matters for business accounts. This is a public beta with device compatibility limits, protected-screen visibility limits and third-party terms or account-restriction risks.
• amosblomqvist/learnharness
  LearnHarness is an AI learning application that connects a learner's present understanding to a target through lessons and practice. Topic goals, answers, confidence and reasoning inform prerequisite checks, plans, explanations and questions. It accumulates propositions with their premises and conclusions in a knowledge graph rather than storing only right-or-wrong outcomes. Earlier sessions can inform later teaching and practice, and personal notes can be imported. Official architecture documentation separates extraction from teaching prose, identity review against existing knowledge and deterministic storage merging. The graph-tool source also exposes different information to diagnostic and extraction-review roles. Desktop releases for macOS, Windows and Linux start with provider API keys and model selection. Dictation and web search need separate service keys, and configured model providers receive task context. Chats, graphs and settings are local files, not proof of entirely offline learning. Neither reasoning accuracy nor educational effectiveness is established by the project description alone, so model output still needs checking.
• ccch1mneyyy/dsh-Go
  DSH Go is an unofficial Android client for using DeepSeek Harness conversations and work. It can run the engine inside a phone environment containing Ubuntu, Node.js and DSH, or connect to a computer's engine through SSH or DSH Remote. It passes text, images and voice input and displays answers, tool approval requests and work status. Running the engine on a phone does not mean running the language model there, since model requests go to the configured provider. Android 8 or newer and an appropriate 64-bit package are required, followed by separate local-runtime or remote-host setup and model credentials. Phone screen control requires the local engine and additional permissions, while the documentation says sensitive actions such as payments and passwords return to the person. DSH Remote limits non-image attachments, the in-app web mode, DS Bot and conversation-log export, so capabilities vary by connection. Screen information may reach the model and anonymous usage statistics are enabled by default, making privacy settings important. The application identifies AGPL-3.0 and the SDK and examples identify MIT, but names and mascots have separate terms, so code and visual assets must be treated separately.

Only repositories in the ranked Trendshift lists are included, and the lists are used only to find candidates. We do not copy their ranks. Descriptions, licenses and star counts come from each GitHub repository. The notes are our own reading. We have not tested these projects, and a place on a trending list does not prove quality.