Understanding Understanding: A Structural Account Across Human and Machine Cognition

Understanding is one of the most persistently invoked yet least precisely defined notions in both everyday and philosophical discourse, and the rise of large-scale machine learning systems that answer difficult questions fluently and often correctly has given the question new urgency: do these systems understand what they are doing, or have they simply found a way to perform well without any of the properties that make human understanding valuable? This paper argues that understanding is best treated not as a distinctive mental state or as a species of propositional knowledge, but as a structural achievement: the reliable ability to navigate a space of implications, counterfactuals, and transformations surrounding a domain, rather than merely to produce one correct output for one input.

We situate this proposal against three existing lines of work. First, the philosophical case for distinguishing understanding from knowledge [Kvanvig, 2003, Elgin, 2017, Zagzebski, 2001, Grimm, 2006], which shows that understanding tolerates idealisation and resists reduction to any finite list of true propositions. Second, the ability view of understanding [Ryle, 1949, Hills, 2016, de Regt, 2017], which treats understanding as a cluster of cognitive skills tested by generalisation to novel cases rather than by reproduction of memorised examples. Third, the competence and performance distinction from cognitive science [Chomsky, 1965, Firestone, 2020], which separates an agent’s underlying capacity from its observed behaviour under practical constraints.

From these strands we develop the notion of structural robustness: an agent understands a domain to the extent that its correct performance is stable under transformations that preserve the domain’s underlying structure while varying its surface features, generalising the abstraction consistency criterion used in benchmarks such as the Abstraction and Reasoning Corpus [Chollet, 2019]. We further argue that failures of understanding are frequently failures of structured ignorance, that is, failures to explore particular regions of a domain’s implication space rather than simple absences of information, a diagnosis with clear analogues in heuristics and biases research [Tversky and Kahneman, 1974].

Applying this framework to machine understanding, we argue that superhuman performance on a reasoning benchmark is not, by itself, evidence of understanding; what matters is whether performance is robust under meaning-preserving transformation, and whether failures reflect genuine gaps in the hypotheses explored rather than execution errors on hypotheses already identified. We then consider two objections: the Chinese Room worry that behavioural robustness cannot settle questions of semantic content [Searle, 1980], and the relation between objectual and propositional understanding. We close by raising a further concern of our own: that the class of transformations counted as meaning-preserving is itself selected by human judgement, and so risks recapitulating human inductive biases under the guise of domain-neutral structure. Drawing on Holm and Banerjee’s 2025 argument that comparisons of intelligence across humans, animals, and machines are already anthropocentric, we suggest that a fully adequate structural account of understanding must confront the same difficulty, and we leave open how the relevant transformations can be specified without introducing a human-centred standard.

We have deliberately left open the deeper metaphysical question of whether structural robustness, however extensive, is sufficient for the fullest sense of understanding, or only a necessary condition and reliable signature of it. We take this openness to be appropriate to the current state of the debate, and we suggest that progress on the empirical question of how to measure structural robustness in both human and artificial reasoners can proceed productively even while these questions remain unsettled.

References

François Chollet. On the measure of intelligence. arXiv preprint arXiv:1911.01547, 2019.

Noam Chomsky. Aspects of the Theory of Syntax. MIT Press, Cambridge, MA, 1965.

Henk W. de Regt. Understanding Scientific Understanding. Oxford University Press, Oxford, 2017.

Catherine Z. Elgin. True Enough. MIT Press, Cambridge, MA, 2017.

Chaz Firestone. Performance vs. competence in human-machine comparisons. Proceedings of the National Academy of Sciences, 117(43):26562–26571, 2020. doi: 10.1073/pnas.1905334117.

Stephen R. Grimm. Is understanding a species of knowledge? British Journal for the Philosophy of Science, 57(3):515–535, 2006. doi: 10.1093/bjps/axl015.

Alison Hills. Understanding why. Noûs, 50(4):661–688, 2016. doi: 10.1111/nous.12092.

Jonathan L. Kvanvig. The Value of Knowledge and the Pursuit of Understanding. Cambridge University Press, Cambridge, 2003.

Gilbert Ryle. The Concept of Mind. Hutchinson, London, 1949.

John R. Searle. Minds, brains, and programs. Behavioral and Brain Sciences, 3(3):417–457, 1980. doi: 10.1017/S0140525X00005756.

Amos Tversky and Daniel Kahneman. Judgment under uncertainty: Heuristics and biases. Science, 185(4157):1124–1131, 1974. doi: 10.1126/science.185.4157.1124