Microsoft wants to bring generative AI to Windows — and the PCs that run it.
In a pair of keynotes during its annual Build developer conference this week, the company unveiled a new lineup of Windows machines it calls Copilot+ PCs, as well as creative AI-powered features like Recall, which lets users find apps , helps to find files and other content. seen in the past. Copilot, Microsoft's brand of generative AI, will soon be more deeply integrated into the Windows 11 experience. And new Microsoft Surface devices are on the way.
We've rounded up all the big announcements from Monday and Tuesday here.
Volumetric apps
Microsoft is bringing Windows volumetric apps — essentially spatially aware, interactive VR apps — to the MetaQuest headset. Through the partnership with Meta, Microsoft says it will bring Windows 365 and native PC connectivity to the Quest headset, enabling developers to extend their apps into 3D space.
During Tuesday's keynote, Microsoft showed a digitally exploded 3D view of an Xbox controller from the perspective of the MetaQuest 3 headset — a digital object that the wearer can manipulate with their hands. “We're deepening our partnership with Meta to make Windows a first-class experience on Quest devices,” Pawan Davalori, CVP of Windows and Microsoft devices, said during the demo.
Developers can sign up for a preview to get access to Microsoft's new volumetric API.
Copilot+ PCs
Copilot+PCs is Microsoft's vision for AI-first, flagship Windows hardware. All of them include dedicated chips called NPUs that power AI experiments like Recall. And they ship with at least 16GB of RAM, paired with SSD storage.
The first Copilot+ PCs will pack Qualcomm's Snapdragon X Elite and Plus chips, which Microsoft claims offer up to 15 hours of web browsing and 20 hours of video battery life. Chipmakers Intel and AMD are also committed to developing processors for Copilot+ devices in partnership with a number of manufacturers including Acer, Asus, Dell, HP, Lenovo and Samsung.
Copilot+ PCs start at $999, and some are available for pre-order today.
Surface Pro and Surface Laptop
Microsoft's new Surface devices, the Surface Laptop and Surface Pro, focus on performance and battery.
The latest Surface Laptop — available with a 13.8- or 15-inch display — has been redesigned with “modern lines” and thinner screen bezels. The company says it lasts up to 22 hours on a single charge and is 86 percent faster than the Surface Laptop 5. It also supports Wi-Fi 7 and has a haptic feedback touchpad.
As for the new Surface Pro, Microsoft says it's 90 percent faster than the previous generation Surface Pro (Surface Pro 9), and it's a new OLED with an HDR display, Wi-Fi 7 (and optional 5G). plays Upgraded ultra-wide front camera. In addition, its detachable keyboard – reinforced with extra carbon fiber – now has haptic feedback.
to remember
Windows 11's upcoming Recall feature can “remember” apps and content a user accessed on their PC weeks or months ago, for example helping them find a Discord chat where they Was thinking of buying clothes. Users can use Recall's timeline to “scroll back” to see what they were working on in the recent past and drill down into files like PowerPoint presentations to potentially narrow down their search. But relevant information can be made public.
Microsoft says Recall can make associations between colors, images and more so users can search for virtually anything on their PC in natural language (not unlike startup Rewind's Tech); Developers will be able to improve Recall by adding relevant information to their apps. And Microsoft claims that all user data associated with Recall is kept private and on-device — and, importantly, not used to train AI models.
Here's more from Microsoft: “Your snapshots are yours; they live locally on your computer. You can delete individual snapshots, adjust time limits in Settings, and delete them.” , or at any point from the icon in the system tray on your taskbar. You can also filter apps and websites from being saved.”
Image editing and live translation
Windows now has more AI than ever before, and some of it is exclusive to the new Copilot+ PCs.
A new feature called Super Resolution can restore old photos by automatically upscaling them. And Copilot can now analyze images to give users ideas for creative compositions. Through a feature called Cocreator, users can create images and tell the AI model what they're drawing to change or restyle the image.
Elsewhere, Live Captions with Live Translation translates any audio that passes through the PC — whether from YouTube or a native file — into the language of the user's choice. Live Translations will initially support around 40 languages, including English, Spanish, Mandarin and Russian.
A separate but related new feature in Microsoft Edge offers real-time video translation on sites like LinkedIn, YouTube, Coursera, Reuters, CNBC, Bloomberg and more. Set to be available in the near future, the feature—which supports Spanish to English and English to German, Hindi, Italian, Russian, and Spanish—live translation of spoken content through both dubbing and subtitles. she does.
Team Copilot and Extensions
Team Copilot is the latest extension of Microsoft's growing Copilot suite of generative AI tech. It integrates with the company's video conferencing app Teams, to help organize meeting agendas and take notes that anyone in a meeting can be a co-author. And it extends to Microsoft's collaboration and planning platforms, Loop and Planner, to create and assign tasks, track deadlines and notify team members when input is needed.
In somewhat related Copilot news, Microsoft has launched (in private preview) Copilot Extensions, which allow developers to extend GitHub's code-generating tool GitHub Copilot with third-party apps and skills. Launch partners include DataStax, Docker and LambdaTest. Extensions will live in the GitHub marketplace, but developers will also be able to create their own custom extensions to integrate with their internal systems and APIs.
Windows Copilot Runtime
Powering capabilities like Recall and Super Resolution is the Windows Copilot Runtime, a collection of ~40 generative AI models that Microsoft describes as “a new layer” of Windows. Combined with Semantic Index, a local vector-based system for an individual Copilot+PC, Windows Copilot Runtime allows generative AI-powered apps — including third-party apps — to run without requiring an Internet connection.
“[The runtime] Ready-to-use AI APIs such as Studio Effects, Live Captions Translation, OCR, Recall with User Activity and [more]which will be available to developers in June,” Davoluri said on Tuesday.
CapCut, the popular video editor from TikTok owner ByteDance, will use the Windows Copilot Runtime, along with a new Windows Copilot Library, a set of APIs and AI dev tools, to accelerate its AI features, Microsoft says. And Meta will add the aforementioned studio effects to WhatsApp to provide features like background blur and eye contact during video calls.
Upgraded bot builders
Azure AI Studio, the toolset within Microsoft's Azure OpenAI service that lets users integrate AI models and build an app that makes “more sense” of that data, will soon let developers pay-as-you-go. Go Inference APIs – will allow building apps using APIs. through which developers can access and tune generative AI models hosted on Azure infrastructure. Microsoft calls it a “model-as-a-service,” and it's starting with models from Nucastla and Core42.
In the adjacent Copilot Studio product suite, Microsoft is launching Copilot agents, which the company describes as AI bots that can “independently configure tasks according to specific roles and functions.” (CoPilot Studio provides tools to connect Copilot for Microsoft 365, the AI-powered “copilot” in apps like Excel and Word, with third-party data. Learning from user feedback and when those situations When faced with asking for help, they don't know how to handle it.
Snapdragon Dev Kit
There's a new dev kit from Qualcomm that aims to help developers build apps for the Arm chip-packing Copilot+ PCs.
The $899.99 Snapdragon Dev Kit for Windows — which has the same width, height and length as Apple's Mac Mini, incidentally — packs Qualcomm's Snapdragon X Elite chip paired with 32GB of RAM, 512GB of storage and plenty of I/O. The Dev Kit supports Wi-Fi 7 and Bluetooth 5.4, and, through its various USB-C and HDMI ports, it can drive three 4K monitors simultaneously.
Phi-3
Microsoft announced an addition to its creative AI model family Phi, Phi-3-Vision, which can perform general visual analysis and reasoning tasks, such as answering questions about charts and images. The model can read both text and images and is efficient enough to run on a mobile device.
Phi-3-Vision is available in preview, while the model's previously announced text-only counterparts — Phi-3-mini, Phi-3-small and Phi-3-medium — are now generally available.
Partnership with Khan Academy
Microsoft is donating access to cloud computing infrastructure in collaboration with Khan Academy, allowing Khan Academy to offer teachers in the US free access to Khan Academy's AI-powered tools. The two companies will also collaborate to explore opportunities to improve AI apps for teaching math through generative AI, Microsoft said on Tuesday.
We're launching an AI newsletter! Sign up Here June 5th to start receiving it in your inboxes.