How the NPU is paving the way toward a more intelligent Windows

Inside Copilot+ PCs is a tiny but powerful AI engine, an innovation originally used in smartphones and now applied to Windows PCs: a neural processing unit Microsoft developed with silicon partners for its latest line of personal computers.

These NPU chips speed up AI tasks locally - which means they happen on the device - and can enhance the Copilot+ PC's digital experiences. What sets Copilot+ PCs apart is the requirement for the NPU to be capable of 40+ Trillion Operations per Second. (TOPS measures trillions of operations a system can perform per second.)

NPUs specialize in processing machine learning and small language models efficiently to power features such as image creation and smart camera effects while drawing less power from other areas of the PC, helping extend battery life when unplugged.

"AI features available on Windows create novel experiences, and we developed the hardware needed to support those experiences," says Steven Bathiche, a corporate vice president and technical fellow who established the Applied Sciences team that develops much of the AI technology in Windows.

An NPU is built with an architecture that allows it to process vast amounts of data in parallel and perform trillions of operations per second. Unlike a general-purpose processor (CPU) or even a graphics processor (GPU), an NPU excels at machine learning calculations while using far less power. This means an NPU can handle AI workloads very efficiently, freeing up the CPU to focus on multitasking and GPU to focus on graphics, animation and gaming.

NPUs allow Microsoft and other manufacturers to offer sophisticated AI experiences on devices with a more affordable price point. What once required compute that cost thousands of dollars can now be done on a device with options that cost hundreds, making the technology more accessible to a wider cohort of people, the company says.

And because of its architecture, the NPU puts Copilot+ PCs in the best position to be ready for when more advanced developments become available.

For instance, NPUs on Copilot+ PCs make it possible to find and do things faster on PCs. This includes (semantic) improved Windows search, which allows people to use natural language to get more accurate search results across files, settings and photos, as well as visual intelligence that enables a variety of easy actions on screen, such as copying text from an image or summarizing content from text on screen.

The NPU allows for some local processing, which can then be used in conjunction with cloud processing that potentially allows for running larger models. When you are connected to the internet, the NPU can work in tandem with the cloud to deliver more extensive experiences.

It's a big milestone for Microsoft, which turned 50 this year, and for Windows, as it celebrates its 40th anniversary in November.

How Copilot+ PCs evolved the NPU

The road to developing the NPU for Copilot+ PCs started years ago with the Surface Hub 2 Smart Camera. Its ability to perform sophisticated AI tasks, like reframing video and correcting perspective locally with a dedicated processor provided a compelling proof-of-concept that powerful on-device AI processing could improve user experiences in real time, such as eye contact, enhanced background blur and AI audio denoising - which means removing background noise. Microsoft took the lessons from that smart camera and applied them to mainstream computing in collaboration with its silicon partners. Copilot+ PCs, equipped with NPUs from AMD, Intel and Qualcomm under the hood, are a direct outcome of that evolution.

Those advances laid the groundwork for the AI features now seen in Copilot+ PCs.

Recall is an opt-in explorable timeline that taps into what you've seen before on a Copilot+ PC across apps, by retracing steps in a visual way. Photos, links, messages and more can be a fresh point to continue from. Click to Do provides shortcuts on a PDF or any text on screen, using those images to summarize, rewrite, copy text or search. More actions are starting to roll out in Click to Do, including the option to create a bulleted list from selected text.

The NPU even enables Copilot+ PCs to provide professional-grade AI editing tools with no subscription required. New to the Microsoft Photos app, relight adds dynamic lighting controls to pictures, simplifying complex lighting adjustments into a visual, intuitive experience for correcting poor lighting or applying creative effects.

Ultimately, one of the biggest benefits of the NPU is being able to run multiple AI-driven applications like these and others simultaneously. While other generative AI is based on large language models and need the cloud to work, small language models are what run on an NPU.

Bathiche says that the Windows platform has the flexibility and adaptability to shift workloads depending on the configuration of the machine. For example, the bigger the GPU, the more the PC can shift graphics-heavy workloads to it. But to save power and efficiency, the NPU can work together with the CPU and GPU, running AI processes in the background.

The future: Reaching more people in more intuitive ways

"If you think about it, for the past 60 years, how we used a computer really hasn't changed," says Bathiche, referring to the mouse and keyboard. With AI agents - intelligent software assistants that can understand user requests and with their permission perform tasks on their behalf - he says, there's an opportunity for a future with a lot less clicking.