Microsoft unveils its major Copilot update with Vision and Voice features across platforms

Microsoft is introducing new capabilities for Copilot across multiple platforms, including Windows, iOS, Android, and the web, in its most significant redesign to date. Key features include Copilot Vision, a new feature that gives the AI assistant the ability to interact with on-screen elements, and a new voice feature for natural spoken interactions akin to OpenAI’s recently launched Advanced Voice Mode.

The updated Copilot apps now have a more conversational UI style with warmer tones. Copilot Vision, available through the Copilot Labs opt-in program for Pro users, allows the assistant to view content on users' PCs, primarily via Microsoft Edge, and answer related questions. It can analyze text and images from selected web pages, though it excludes paywalled and sensitive content. Microsoft emphasizes that Copilot Vision deletes data immediately after conversations, ensuring no data is stored for model training.

In the other hand, Copilot Voice supports spoken interactions with four synthetic voices that respond based on the user's tone, with usage time limited and more minutes available to Pro subscribers. The new "Think Deeper" feature also allows Copilot to provide step-by-step answers to complex problems using advanced reasoning models from OpenAI, initially available to select users in Australia, Canada, New Zealand, the U.S., and the U.K.

The plataform has also announced new integrations with WhatsApp and Microsoft OneDrive, offering features like document summaries and complex question answering. Personalized recommendations based on past interactions will also be introduced, with the updated features rolled out gradually.