What is Extra?
Extra gives AI agents hands and eyes on your Windows desktop.
Connect Claude Desktop, Google Antigravity, Cursor, Windsurf, AGY CLI, or your own autonomous agent and let it interact with real Windows applications.
Unlike traditional pyautogui-style automation or slow cloud-based computer-use systems, Extra works close to the Windows graphics, UI, and input layers for fast, reliable desktop automation.
🚀 Why Extra is Different
⚡ <3ms Frame Capture via DXGI
Captures the Windows desktop through the DirectX Desktop Duplication API instead of legacy GDI screen capture, enabling extremely fast visual feedback.
🎯 Semantic UI Access via Windows UI Automation
Extra can inspect the Windows accessibility/UI tree and identify actual controls, elements, and bounding boxes instead of relying entirely on visual guessing.
🔢 Set-of-Mark Visual Badges
Interactable UI elements can be automatically labelled with numbered visual markers, making it easier for vision-based agents to select the correct target.
⌨️ Fast Hardware-Level Input
Uses native Windows SendInput with keyboard scan codes and Unicode input for fast, reliable typing and mouse interaction.
🛡️ Perceptual Stall Detection
Extra compares screen states between actions to detect frozen workflows, unresponsive dialogs, or actions that didn't actually happen—helping agents recover instead of wasting tokens.
🖥️ Multi-Monitor & DPI Support
Handles Windows DPI scaling and multi-monitor setups, including common 125%, 150%, and 4K configurations, helping prevent coordinate drift.
yantraOS 