From Quasi-Agents to Governed Artificial Systems: Extending Chalmers through Mindful Machine Theory

1. Background and Problem

As artificial intelligence moves from single-turn language generation toward persistent systems that remember, use tools, coordinate agents, modify workflows, and act in the world, the foundation model is no longer a sufficient unit of philosophical or engineering analysis. Three questions increasingly overlap but should remain distinct. First, what cognitive organization might support consciousness or subjectivity? Second, what entity, if any, persists across interactions with a language model? Third, what architecture allows an artificial system to remain governable and accountable while its models, resources, workflows, policies, and topology change?

David Chalmers’ work addresses the first two questions. His analysis of large language model consciousness emphasizes architectural features such as recurrent processing, global workspace, unified agency, and self-modeling rather than inferring consciousness from fluent behavior [1,3]. His more recent account of the LLM “interlocutor” shifts the unit of analysis beyond the foundation model, characterizing the conversational entity as a virtual, memory-thread-bound quasi-agent rather than simply as an abstract model or a particular hardware instance [2]. This paper argues that this move toward persistence and system-level identity opens a further problem: memory continuity is not yet governance continuity.

2. Objective

The objective is to develop a conceptual bridge from Chalmers’ quasi-agent and memory-thread account to the operational notion of governed identity developed in Mindful Machine theory. The central claim is that artificial agency should be analyzed at three architectural levels: cognitive architecture, interlocutor identity, and governed operational identity. None logically entails the next. A system may display sophisticated cognition without persistent identity; persistent memory without legitimate authority; or operational continuity without phenomenal consciousness.

3. Conceptual Framework: From Memory Thread to Commitment Thread

Chalmers’ memory-thread account provides a criterion for asking, “Is this the same interlocutor?” Governed systems require the stronger question, “Is this still the same accountable operational entity?” A memory thread can preserve conversational history while the rules authorizing action silently change. Conversely, an enterprise process can preserve its governing commitments while replacing an underlying model and losing stylistic continuity.

We therefore distinguish conversational identity from governed identity. Let conversational identity at time be represented as

where denotes retained memory and conversational context. Governed operational identity requires a richer state,

where denotes commitments, authority, policies and invariants, event and consequence history, and recovery obligations.

The notation is not intended as a complete metaphysics of identity; it exposes an engineering distinction. Governed identity is the continuity of purpose, authority, commitments, admissibility constraints, provenance, consequences, and recovery obligations across changes in executors, models, topology, and resources. Memory contributes to such identity but does not define it. This distinction is central because accountability attaches not merely to the component producing tokens, but to the process that authorizes, executes, records, and remains responsible for consequential commitments.

4. Architectural Method: Mindful Machine Theory

Mindful Machine theory treats governance-relevant knowledge as executable system state rather than documentation external to the runtime [12–15]. Its Digital Genome represents functional and non-functional requirements, topology, policies, commitments, historical context, and recovery conditions. The Autopoietic Management and Orchestration System (AMOS) observes execution, compares proposed actions and system states against this knowledge, authorizes or constrains changes, and coordinates adaptation independently of any particular model or infrastructure substrate.

This creates a key separation between proposal and commitment. In many agentic systems, the same model that proposes an action may also interpret the prompt-level policy intended to constrain it. Mindful Machine architecture instead places a governance/control path between proposal and durable action. An LLM, symbolic program, database, service, external tool, or human may propose or contribute to an action, but the proposal is evaluated against machine-readable commitments before execution. The governance layer may authorize, deny, abstain, enter a declared degraded mode, or escalate to legitimate human authority. Model output is therefore evidence or proposal, not final authority.

The architecture further stratifies change. An outer stratum contains purpose, invariants, and authority conditions that ordinary execution cannot amend. A middle stratum contains policies, routing choices, degraded modes, and structural options that may change under supervised authority. An inner stratum contains operational self-regulation such as scaling, failover, migration, recovery, and executor selection. This distinction separates self-regulation from self-amendment: the capability to change behavior does not imply legitimate authority to change the rules that justify behavior.

5. Relationship to Contemporary Agent Architectures

Current agent research already moves toward system architecture. ReAct couples reasoning with environmental action [4]; Generative Agents add memory, reflection, and planning [5]; Reflexion uses episodic linguistic feedback [6]; MemGPT introduces operating-system concepts for persistent memory [7]; AIOS adds scheduling, context, storage, tools, and access control [8]. Recent agent-operating-system work explicitly separates a Control and Governance Plane from a Runtime and Coordination Plane [9].

These advances are important, but they solve different parts of the problem. Memory supports persistence; reflection supports behavioral improvement; tool interfaces support action; access control limits resources; orchestration coordinates execution. Considered independently, none specifies the durable commitments of the whole system, the authority by which those commitments may change, or the consequence record by which accountability can be reconstructed. The architecture ladder examined in the source paper therefore suggests that increasing agent capability does not automatically produce governance continuity.

Mindful Machine theory complements rather than replaces contemporary agent architectures by making commitment continuity a first-class systems property.

6. Cognitive Loops versus Governance Loops

A further distinction follows between cognitive and governance loops. A cognitive loop asks: What should I try next? It may plan, act, observe, reflect, and retry. A governance loop asks: What may I commit, under whose authority, with what evidence, and with what obligations if the action fails? These loops must interact but should not collapse into one another.

This distinction has a cybernetic precedent. Ashby’s law of requisite variety frames regulation in terms of a regulator having sufficient response variety to handle disturbances [10], while the good-regulator theorem links effective regulation to possession of a model of the regulated system [11]. In a governed AI system, memory and models increase informational variety; policy and authorization constrain admissible responses; consequence history enables comparison between intended and realized outcomes.

The key design rule is therefore that the component proposing an action should not, by that fact alone, possess authority to authorize the action or rewrite the rules under which it acts. Cognitive reflection may improve proposed behavior, but reflection must not silently become self-authorization.

7. Evidence and Evaluation

Existing Mindful Machine implementations provide bounded evidence for elements of this architecture. Reported demonstrations include policy-driven auto-failover, auto-scaling, live migration, associative memory, and event-driven history [13]; integration of a persistent Digital Genome and AMOS in a credit-default application [14]; and topology-as-data with explicit consistency/availability commitments in a distributed microservice/database testbed [15].

These results demonstrate the architectural feasibility of integrating governance knowledge, resource regulation, event history, and structural adaptation into executable systems. They do not establish phenomenal consciousness, general intelligence, or general superiority over state-of-the-art agent platforms. The evidence supports specific mechanisms in bounded testbeds and therefore motivates comparative rather than metaphysical conclusions.

The proposed research program consequently evaluates governed identity under disturbance, rather than relying only on nominal-path performance. Relevant probes include model substitution during execution, adversarial or poisoned memory, prompt injection through tools, stale policy, partial service failure, topology change, irreversible actions, and attempted modification of purpose or invariants.

Expected evidence includes preservation of policy and authorization semantics; provenance of model substitutions and actions; prevention of untrusted content from becoming de facto authority; declared degraded modes during partial failure; stronger authorization gates for irreversible actions; preservation of commitments during migration and scaling; and ratified change paths for modifications to constitutional constraints. Governance claims should therefore be supported by disturbance-injection evidence rather than successful demonstrations under normal conditions alone.

8. Discussion

The strongest philosophical connection between Chalmers and Mindful Machine theory is therefore not a claim about consciousness but a shared problem of continuity. Chalmers asks what persists as the interlocutor when the model is multiply instantiated and conversations acquire memory. Mindful Machine theory asks what persists as the accountable entity when executors, resources, topology, policies, and workflows change.

The proposed answer is a commitment thread: a versioned governance context that preserves purpose, authority, admissibility, provenance, consequences, and recovery obligations regardless of which executor generated the proposal. Future systems may dynamically combine multiple foundation models, specialized agents, humans, databases, tools, and external services. In such systems, asking “Which model acted?” may become progressively less informative. The more consequential question becomes: Which governed context authorized the action and retains the obligation created by it?

This also clarifies what the argument does not establish. Governance and autopoietic regulation do not imply phenomenal consciousness. Cognitive architecture and governance architecture remain methodologically distinct research programs. Yet as future systems acquire greater recurrence, persistent memory, compositional agency, embodied action, and structural adaptation, the two programs increasingly meet at the level of complex systems architecture. Their dependent variables remain different: subjective experience on one side, and safe, accountable continuity on the other.

9. Conclusion

A capable agent can reason and act. A persistent agent can remember and adapt. A managed agent can coordinate resources under an operating runtime. A governed artificial system must additionally preserve explicit commitments, distinguish proposal from permission, maintain an authority chain, record consequences, survive structural change, abstain when authority or evidence is insufficient, and recover when reality diverges from intent.

The central contribution is therefore the concept of governed identity as a third architectural layer beyond cognitive organization and interlocutor persistence. Chalmers’ quasi-agent and memory-thread account helps identify the entity that speaks and persists in interaction. Mindful Machine theory extends the analysis toward the entity that is authorized to act and remains answerable for what follows.

The philosophical question of artificial agency consequently expands from “What are we talking to?” to “What operational entity is permitted to commit an action, under whose authority, and what makes it remain accountable through change?” This shift neither resolves the problem of machine consciousness nor requires such a resolution. It instead identifies a distinct philosophical and engineering requirement for increasingly autonomous artificial systems: continuity must encompass not only memory and cognition, but also purpose, authority, commitment, consequence, and responsibility.

References

1. Chalmers, D. J. (2023; rev. 2024). Could a Large Language Model Be Conscious? arXiv:2303.07103. https://doi.org/10.48550/arXiv.2303.07103.

2. Chalmers, D. J. (2026). What We Talk to When We Talk to Language Models. PhilArchive, version 2.

3. Butlin, P., Long, R., Bayne, T., Bengio, Y., Birch, J., Chalmers, D., et al. (2026). Identifying indicators of consciousness in AI systems. Trends in Cognitive Sciences, 30(6), 488–501.

4. Yao, S., Zhao, J., Yu, D., et al. (2023). ReAct: Synergizing Reasoning and Acting in Language Models. ICLR.

5. Park, J. S., O’Brien, J. C., Cai, C. J., et al. (2023). Generative Agents: Interactive Simulacra of Human Behavior. UIST 2023.

6. Shinn, N., Cassano, F., Berman, E., et al. (2023). Reflexion: Language Agents with Verbal Reinforcement Learning. NeurIPS, 36, 8634–8652.

7. Packer, C., Wooders, S., Lin, K., et al. (2023). MemGPT: Towards LLMs as Operating Systems. arXiv:2310.08560.

8. Mei, K., Zhu, X., Xu, W., et al. (2024; rev. 2025). AIOS: LLM Agent Operating System. arXiv:2403.16971.

9. Sharma, A., & Shah, D. (2026). The Agent Operating System: A Reference Operating Architecture for Distributed Agentic Systems. arXiv:2608.03214.

10. Ashby, W. R. (1956). An Introduction to Cybernetics. Chapman & Hall.

11. Conant, R. C., & Ashby, W. R. (1970). Every good regulator of a system must be a model of that system. International Journal of Systems Science, 1(2), 89–97.

12. Mikkilineni, R. (2022). A New Class of Autopoietic and Cognitive Machines. Information, 13(1), 24.

13. Mikkilineni, R., Kelly, W. P., & Crawley, G. (2024). Digital Genome and Self-Regulating Distributed Software Applications with Associative Memory and Event-Driven History. Computers, 13(9), 220.

14. Mikkilineni, R., & Kelly, W. P. (2025). From Static Prediction to Mindful Machines: A Paradigm Shift in Distributed AI Systems. Computers, 14(12), 541.

15. Mikkilineni, R., & Kelly, W. P. (2026). The Missing Layer in Modern IT: Governance of Commitments, Not Just Compute and Data. Computers, 15(5), 275.