NSF Project Pitch & Research Framework

Collaboration and Shared Awareness in AI Systems

Investigating computational foundations, coordination mechanisms, and quantitative metrics for heterogeneous multi-agent artificial intelligence systems.

Introduction

As AI systems become increasingly specialized, future applications will increasingly depend on collections of AI agents rather than a single model. A vision agent may identify an event, a language agent may interpret its significance, and a planning agent may determine an appropriate response. Yet current multi-agent systems often coordinate through loosely structured messages and repeated reconstruction of context. The result can be redundant computation, conflicting decisions, and poor awareness of what other agents are doing.

Recent practical work by David Levine highlights this problem in real-time multi-agent systems. In a LiveKit-based multi-agent environment, agents can independently process the same audio, talk over one another, and lack a shared understanding of “who is doing what right now.” Levine proposes treating agent state, turn-taking, fairness, and interruption as explicit coordination problems rather than relying on timing or implicit inference. He further points toward low-latency awareness mechanisms and shared contextual state as a way for specialized agents to coordinate without repeatedly reprocessing an entire conversation.

“How can independent AI systems develop and maintain a useful shared awareness of a dynamic environment when their knowledge, capabilities, confidence, and observations differ?”

This project investigates the computational foundations of shared awareness among heterogeneous AI systems. We hypothesize that collaboration can be substantially improved when agents maintain an explicit representation of shared awareness that captures not only information, but also its provenance, confidence, recency, ownership, and unresolved conflicts. Rather than requiring every agent to independently reconstruct the state of a problem, agents will selectively share information and coordinate responsibilities through this common representation.

Research Objectives

Research Objective 1: Representing Shared Awareness

The project will develop computational representations of what a collection of AI agents collectively knows, does not know, and believes to be uncertain. The representation will incorporate evidence, provenance, confidence, temporal validity, agent state, and unresolved disagreement. A key research question is how much information must be shared to establish useful common ground without requiring every agent to receive or process the complete underlying data stream.

Research Objective 2: Coordinating Specialized Agents

The project will develop algorithms that allow agents to determine when they should contribute, request additional information, challenge another agent's conclusion, defer to a more capable agent, or remain silent. David Levine's work on turn-taking suggests an important generalization: communication itself can be treated as a coordinated resource, with explicit policies governing eligibility, priority, and interruption rather than allowing every agent to respond whenever it has something to say.

The research will investigate whether similar principles can govern broader forms of agent behavior, including task allocation, information exchange, and decision authority.

Research Objective 3: Measuring Collaboration

Current AI evaluation largely measures individual-model performance. This project will develop quantitative measures of shared-awareness accuracy, collaboration gain, communication efficiency, conflict resolution, information freshness, and responsibility allocation.

A central metric will be collaboration gain: the measurable improvement achieved by coordinated agents compared with the performance of the best individual agent operating independently.

Phase I Experimental Approach

Phase I will evaluate the proposed methods in controlled multi-agent environments in which specialized agents receive different, incomplete, and potentially conflicting observations. The proposed shared-awareness approach will be compared against independent agents, conventional natural-language message passing, and centralized reasoning.

Experiments will measure decision accuracy, latency, communication volume, computational cost, robustness to stale information, and the system's ability to recognize and resolve conflicting evidence.

The project will also investigate an important distinction suggested by David's real-time agent work: shared awareness does not require every agent to share everything. A practical coordination layer can instead expose selective, low-latency state and role-specific context while allowing specialized agents to retain their own internal reasoning.

Intellectual Merit

The proposed research addresses a fundamental limitation in current multi-agent AI: agents can exchange information without necessarily developing a consistent understanding of what the collective system knows, does not know, or needs to know. The project treats shared awareness as an explicit computational object rather than an incidental consequence of message exchange.

The research will establish representations, coordination algorithms, and evaluation methods for studying collective intelligence in heterogeneous artificial systems. The work could reveal fundamental relationships between information sharing, coordination, communication cost, uncertainty, and collective performance.

Broader Impacts

Reliable collaboration among AI systems could enable more capable and efficient applications in scientific research, robotics, healthcare, industrial automation, and human-AI interaction. The resulting benchmarks and metrics could provide researchers with standardized methods for evaluating multi-agent collaboration beyond individual-model performance.

The research will also support future systems in which humans interact with collections of specialized AI agents. In such systems, maintaining shared awareness is not simply a matter of providing more information; it requires determining what each participant needs to know, when it needs to know it, and when an agent should act or remain silent.

Expected Outcome

The project will produce:

A computational framework for shared awareness among heterogeneous AI agents.
Algorithms for real-time coordination, selective information exchange, and conflict resolution.
Quantitative benchmarks for measuring collaborative AI performance.
Experimental evidence identifying when explicit shared awareness provides advantages over conventional multi-agent communication.

Together, these results will establish whether shared awareness can serve as a fundamental coordination mechanism for collections of AI systems, providing a foundation for future research in collaborative and collective artificial intelligence.