Abstract

Despite widespread discussion of AGI , there is no clear framework for measuring progress toward it. This ambiguity fuels subjective claims, makes it difficult to track progress, and risks hindering responsible governance. As a starting point to address this gap, we present a framework for understanding system capabilities in relation to human cognitive abilities. Drawing from decades of research in psychology, neuroscience, and cognitive science, we introduce a Cognitive Taxonomy that deconstructs general intelligence into 10 key cognitive faculties. We then propose a rigorous evaluation protocol in which a system’s performance is measured across a suite of targeted, held-out cognitive tasks, generating a ‘cognitive profile’ that can be used to understand a system’s strengths and weaknesses. We hope this framework will provide a practical roadmap and an initial step toward more rigorous, empirical evaluation of AGI .

Bio: Dr Ryan Burnell is a Staff Research Scientist in the Gemini evaluations team at Google DeepMind in London. His work focuses on developing rigorous and innovative frameworks to evaluate AI systems, understand their reasoning capabilities, and ensure their safety and reliability. His work has been published in top journals, including Science, and featured in popular media outlets such as the BBC , The Atlantic, and the Telegraph.