ChatGPT adds video and screenshare capabilities to Advanced Voice Mode for visual context

OpenAI has expanded ChatGPT Advanced Voice Mode (AVM) by adding video and screenshare capabilities, building upon the initial audio features introduced with GPT-4o in May. This update enables users to use their phone cameras for visual recognition, allowing ChatGPT to finally understand visual context, and "see" the user's surroundings to interact accordingly.

During a livestream, OpenAI's Chief Product Officer Kevin Weil showcased these new features by demonstrating how AVM could assist in making pour-over coffee, recognizing the coffee maker and offering step-by-step guidance.

The update, which also includes playful elements likea seasonal Santa voice and the ability to interpret screenshared messages, is currently available to ChatGPT Plus and Pro users, with broader access planned for January.