OpenAI recently unveiled a five-tier system for assessing its progress toward artificial general intelligence (AGI) development, according to an OpenAI spokesperson who spoke to Bloomberg. The company shared the new classification system with employees during an all-hands meeting on Tuesday, which aims to provide a clear framework for understanding AI development. However, the system describes hypothetical technology that does not yet exist and is probably best interpreted as a marketing ploy to garner investment dollars.
OpenAI has previously said that AGI — a loose term for a hypothetical concept that means an AI system that can perform new human-like tasks without special training — is currently the company's primary goal. The acquisition of technology that can replace humans in much of the intellectual work drives much of the sustained hype at the firm, even though such technology would be enormously disruptive to society.
OpenAI CEO Sam Altman has previously expressed his belief that AGI can be achieved within the decade, and a large part of the CEO's public messaging has been about how the company (and the general public How can society handle the disruption caused by AGI? Along these lines, a classification system makes sense to communicate internally achieved AI milestones on the path to AGI.
The five levels of OpenAI—which she plans to share with investors—range from existing AI capabilities to systems that can potentially manage entire organizations. The company believes its technology (such as the GPT-4o that powers ChatGPT) currently sits at Level 1, which includes AI that can engage in the conversational process. However, OpenAI executives reportedly told staff that they were well on their way to Level 2, known as “Reasoners.”
Bloomberg lists OpenAI's five “phases of artificial intelligence” as follows:
- Level 1: Chatbots with conversational language, AI
- Level 2: Reasoning, human-level problem solvers
- Level 3: Agents, systems that can take action.
- Level 4: Innovators, AIs that can help invent.
- Level 5: Organizations, AI that can do the work of an organization.
A Level 2 AI system will reportedly be able to solve the same basic problem as a human with a doctorate degree but no access to external devices. During the all-hands meeting, OpenAI's leadership reportedly demonstrated a research project using its GPT-4 model that researchers believe could be used in the future, according to a person familiar with the discussion who spoke to Bloomberg. There are signs of reaching the reasoning ability like this human being.
The upper levels of the OpenAI hierarchy describe increasingly powerful hypothetical AI capabilities. Level 3 “agents” can work autonomously on tasks for days. Level 4 systems will create new innovations. The apex, Level 5, envisions AI taking over entire organizations.
This classification system is still in progress. OpenAI plans to collect feedback from employees, investors, and board members, potentially refining the levels over time.
Ars Technica asked OpenAI about the ranking system and the accuracy of Bloomberg's report, and a company spokesperson said they had “nothing to add.”
The problem with ranking AI abilities
OpenAI isn't alone in trying to fix the level of AI capabilities. As Bloomberg notes, OpenAI's system feels similar to automated driving levels mapped out by automakers. And in November 2023, researchers at Google DeepMind proposed their own five-level framework for assessing AI development, indicating that other AI labs are also trying to figure out how these things rank. How to ban those that do not exist yet.
OpenAI's classification system is also somewhat similar to Anthropic's “AI Safety Levels” (ASLs) first published in September 2023 by the maker of the cloud AI assistant. Both systems aim to rank AI capabilities, although they focus on different aspects. Anthropic's ASLs are more clearly focused on security and catastrophic threats (such as ASL-2, which refers to “systems displaying early signs of threat capabilities”), while OpenAI's level tracks general capabilities.
However, any AI classification system raises questions about whether it is possible to meaningfully quantify AI progress and what constitutes progress (or even what constitutes a “dangerous” AI system, as in the case of Anthropic). The tech industry has a history of over-promising AI capabilities so far, and linear growth models like OpenAI fuel potentially unrealistic expectations.
There is currently no consensus in the AI research community on how to measure progress toward AGI or even if AGI is a well-defined or achievable goal. As such, OpenAI's five-level system should be viewed as a communication tool to attract potential investors that reflects a company's aspirational goals rather than a scientific or even technological measure of progress.