Microsoft wants to show how AI assistants can help and support Windows users with their tasks. It has developed the standard Windows Agent Arena for this purpose.
The standard Specifically, it tests the performance of AI assistants on Windows computers. It tests the accuracy with which tasks are performed and the speed with which the AI agent can interact with commonly used Windows applications. The objects tested include the Microsoft Edge and Google Chrome web browsers, system functions like Explorer, and applications like Visual Studio Code, Notepad, Paint, and the clock. The test includes 150 different actions.
AI agents are not ready yet
To convince Windows users that AI agents for PCs are a great help, it seems the technology needs to evolve further. Microsoft Research, the developers of the benchmark, compiled the Navi agent themselves. The AI agent achieved an overall score of only 19.5 percent, while the success rate for humans was 74.5 percent. Windows Agent Arena is a good benchmark for AI agent developers regarding the performance of their latest developments.
Rogerio Bonatti, lead author of the study He says“Windows Agent Arena provides a realistic and comprehensive environment for pushing the boundaries of AI agents. By making our standard open source, we hope to accelerate research in this critical area within the AI community.
Developing high-performance AI agents is also important for Microsoft to launch sales of Copilot+ PCs. The latest models from PC manufacturers still have the capabilities to run AI applications. However, for the user to benefit from this, the applications must also be at a high level.
Read also: These are the new Copilot+ computers from Lenovo, Samsung, ASUS, and Acer
