General access, costly action chains
Any human-accessible application becomes reachable, but simple goals can require many visual steps. Native text is rendered into pixels, then re-encoded for a language model.
Position
Digital agents need an agent-native computer, not another single-purpose interface.
Today's agent environments are a patchwork of incompatible worlds. Some expose low-level GUI control; others expose benchmark-specific text tools. Real digital tasks often need both: visual inspection and verification, plus symbolic operations for files, code, web data, and documents.
Efficient actions, incomplete coverage
Bash, Python, web tools, and APIs align with LLM strengths, but many workflows still depend on proprietary apps, GUI-only interactions, visual layout, and presentation checks.
Shared state across complementary views
An Agent-Native Computer lets agents choose the right surface for each subtask while every action updates one coherent sandboxed OS state.
Context
Agentic work has outgrown text-only benchmarks and isolated tools.
Modern LLM agents operate in a perception-decision-action loop: they observe a task context, choose actions, inspect the consequences, and continue. As tasks move toward software engineering, web work, scientific discovery, deep research, and economically valuable GDPVal-style office tasks, the environment becomes a core part of the system.
The paper asks a simple design question: what environment does an agent need to complete any given digital task? The answer is not a human GUI alone, and not a bag of text APIs alone. Agents need a computer whose interaction surfaces are modular, semantically rich, and synchronized through the same operating-system state.
Conceptual Framework
Agent-Native Computer
An Agent-Native Computer is a unified environment for arbitrary digital tasks. It integrates GUI-based and text-based interaction surfaces, keeps them grounded in the same sandboxed OS state, and factors interaction into configurable Environment Views.
What agents can do
GUI actions, OS actions such as open_app and switch_window, app-specific actions, shell commands, code execution, and mounted tool calls.
What agents can see
Screenshots, accessibility trees, window status, application state, file contents, execution results, logs, and mounted environment outputs.
Environment Views
Each view pairs a task-relevant observation space with a matching action space, such as a screen view, OS-level view, app-level view, or external mounted environment.
Context-bound and free-form actions
Some actions operate inside a specific view and depend on its current state; others are stateless calls that can be registered independently.
Shared OS state across interfaces
A file downloaded by a web tool can be opened in the GUI; a Python edit appears in the native application; every interface changes the same computer.
Evidence
Layering agent-facing interfaces improves OSWorld performance.
The experiment holds the hierarchical agent architecture fixed and changes only the environment. On the OSWorld workflow subset, performance rises monotonically as text tools, an OS-level view, and code execution are added to the GUI baseline.
| Setting | Tools | OS View | Code | Result |
|---|---|---|---|---|
| Baseline | 41.54 | |||
| + Tools | Yes | 46.72 | ||
| + OS View | Yes | Yes | 47.87 | |
| + Code | Yes | Yes | Yes | 51.30 |
+9.76
points over the GUI-only baseline
The largest gain comes from mounted tools; OS views and code add complementary reliability by reducing screenshot parsing, navigation, batch-editing, and file-operation overhead.
Demo Session
Step through a complete task execution.
The demo shows how a task trajectory moves across setup, GUI state, text-based operations, repeated repairs, and final deliverables inside one shared environment.
Case Studies
Real workflows need modality switching.
Two GDPVal tasks and one GAIA task show the same pattern: GUI interaction grounds visual understanding and verification, while text-based actions provide efficient manipulation, computation, and error recovery.
Medical Secretary
Task
Create three lab-specific Excel bulk forms and three email templates from a reference oncology testing spreadsheet.
Insight
Python efficiently parses, filters, sorts, and generates files, but GUI verification reveals that embedded logos can be lost and must be repaired through native spreadsheet interaction.
Takeaway
The computer is part of the agent system.
As LLM agents move toward economically valuable digital work, the operating environment should stop forcing a choice between human-facing GUIs and narrow tool APIs. Just as GUIs emerged to match human perception and control, agent-native interfaces should become first-class software surfaces for agents.
Alternative Views
Why not just use one interface?
The paper addresses two natural objections: future agents may master GUIs, or APIs and wrappers may eventually cover all digital work. Both objections confuse possible access with sufficient and efficient interaction.
GUIs Alone Are Sufficient
One view argues that agent-specific interface layers are temporary scaffolds. As vision-language models improve, agents may reach human-level GUI proficiency, making text-based abstractions unnecessary overhead.
Our response: GUI-only interaction confuses capability with optimality. Even perfect visual grounding would still make bulk spreadsheet edits, file operations, and code changes unnecessarily long and error-prone.
APIs Alone Are Sufficient
A second view argues that text-based actions alone will suffice once APIs, wrappers, and accessibility-tree tools are comprehensively developed. Under this view, GUI interaction is only a temporary tooling gap.
Our response: complete API coverage is not realistic. Proprietary applications lack comprehensive programmatic interfaces, accessibility trees are incomplete, and many tasks require visual verification. Agent-native computers treat text and GUI interaction as coequal views.
Citation
BibTeX
@inproceedings{wu2026agentnativecomputers,
title={Position: Digital Agents Require Unified Agent-Native Computers},
author={Wu, Yiran and Liu, Jiale and Zhang, Jieyu and Zhang, Yaolun and Liu, Shilong and Wang, Chi and Wang, Mengdi and Wang, Huazheng and Wu, Qingyun},
booktitle={International Conference on Machine Learning},
year={2026}
}