Microsoft Build 2024: AI‑Powered Copilot PC, New Windows Copilot Runtime, GPT‑4o

Microsoft’s annual developer conference, Microsoft Build, opened in Seattle on May 21 with the theme “Boosting developer productivity with AI.”

Opening remarks

CEO Satya Nadella said that AI will become distributed and that the richest AI experiences will combine cloud and edge computing.

Copilot + PC

Microsoft announced the Copilot + PC, a new AI‑focused Windows computer capable of more than 400 trillion operations per second and integrated with large language models. The device includes a Prism simulation layer for seamless compatibility with x86 apps and is built in partnership with Intel, Qualcomm, AMD and OEMs.

More than 40 AI models open to developers

Microsoft revealed that over 40 AI models now run on Windows 11 through the new Windows Copilot Runtime, an API library that lets developers embed AI capabilities such as Studio Effects, Live Captions, OCR, Recall, and Phi‑Silica into their apps.

New Windows Copilot Runtime

The runtime provides ready‑to‑use AI APIs and a semantic index that stores user activity locally, enabling features like Recall, which records and searches a user’s PC activity.

Developers can also use the runtime’s API library to build custom AI pipelines and integrate with Azure services.

Recall

Recall lets users search and revisit everything they did on a PC, powered by the local semantic index. Microsoft plans to expose a Vector Embeddings API so developers can build their own vector stores and retrieval‑augmented generation (RAG) solutions.

Fabric

Microsoft announced new AI features in Microsoft Fabric, including real‑time intelligence for faster business decisions and a Workload Development Kit for extending applications within Fabric.

GitHub Copilot extensions

New extensions let developers customize Copilot with Azure, Docker, Sentry and other services. GitHub Copilot for Azure enables managing Azure resources, troubleshooting, and accessing logs directly from the Copilot chat.

Team Copilot

Team Copilot extends Microsoft 365 Copilot to Teams, Loop, Planner and other collaboration spaces, helping with agenda management, note‑taking, action‑item tracking and project oversight.

Microsoft Copilot Studios also added an agent capability that lets developers build assistants that react to specific data and events.

GPT‑4o launch

GPT‑4o is now available as an API in Azure AI Studio, and the Phi‑3‑vision model is also accessible on Azure.

Microsoft also announced a partnership with Cognition, bringing the Devin AI software to Azure.

All of these improvements run on the upcoming ARM‑based driver system and upcoming NPU hardware from AMD and Intel, positioning Windows as an open AI platform for the next decade.