Microsoft is enhancing its Copilot artificial intelligence system to grant it greater control over Windows and user files, a move signaled by the introduction of "Hybrid Intelligence." This new approach allows Copilot to access local data and execute commands directly within the operating system, a capability demonstrated through a tax filing scenario.
The Hybrid Intelligence framework enables Copilot to utilize a mix of local and cloud-based AI models, aiming for more efficient task processing. During a demonstration, an executive asked Copilot's Autopilot function to assist with tax preparation. The AI then accessed various folders, located necessary documents, updated them, compressed them into a single folder, and drafted an email to the accountant with the attached files.
These new capabilities are expected to roll out to Copilot+ PCs in the coming months. The integration builds upon existing Copilot experiences such as Home, Code, and Autopilot, by incorporating local device information and enabling more on-device processing. For developers, Copilot's Code functionality will allow for the creation of native Windows applications from single prompts and more secure code execution with support for Microsoft Execution Containers (MXC).
Microsoft also detailed security measures for agent access, with organizational controls available through Microsoft Execution Containers, which enforce limits on file and network access for agents. This system provides various containment options, from process isolation to virtual machines, ensuring that agents operate within defined parameters.
The company indicated that while Copilot will continue to use cloud inference for complex tasks, it can run prompts on local models for scenarios prioritizing cost or privacy. This local processing aims to make tokens go further, reducing the need for constant cloud communication. GitHub's HydraFusion routing system is also being extended to local Windows models, with an experimental preview expected later in October for the GitHub Copilot app, its command-line tool, and Visual Studio Code.
New hardware is also being introduced to support these advancements. The Surface Laptop Ultra, featuring Nvidia RTX Spark, is priced at $2,599 and begins shipping on October 16. A Surface RTX Spark Dev Box will ship to U.S. customers in November.
Microsoft's approach to Hybrid Intelligence aims to make AI a more integrated part of users' daily computer interactions. The company stated that this integration allows users to express their needs, and with permission, their Windows PC can act on their behalf. The updated Copilot features are designed to bring information from the device into each Copilot experience, allowing more work to be completed directly on the PC.