In September, OpenAI introduced A.I. technology that could “reason” through tasks involving math, coding and science.
Now, this technology can tackle similar tasks that involve images, including sketches, posters, diagrams and graphs.
On Wednesday, the company unveiled two new versions of its reasoning technology called OpenAI o3 and OpenAI o4-mini. Each can handle tasks that involve both images and text.
These systems can “manipulate, crop and transform images in service of the task you want to do,” said Mark Chen, head of research at OpenAI, in announcing the new system during an internet livestream.
OpenAI also said these systems could generate images, search the web and use other digital tools.
Unlike early versions of its ChatGPT chatbot, these reasoning systems spend a significant amount of time “thinking” about a question before answering, rather than providing an instant response.
The systems are part of a wider effort to build A.I. that can reason through complex tasks. Companies like Google, Meta and DeepSeek, a Chinese start-up, are developing similar technologies.
The goal is to build systems that can solve a problem through a series of steps, each one building on the last, similar to how humans reason. These technologies can be particularly useful to computer programmers who use A.I. systems to write code.
The reasoning systems are based on a technology called large language models, or L.L.M.s. To build reasoning systems, companies put L.L.M.s through an additional process called reinforcement learning. During this process, a system learns behavior through extensive trial and error.
By working through various math problems, for instance, it can learn which methods lead to the right answer and which do not. If it repeats this process with a large number of problems, it can identify patterns.
OpenAI’s latest systems have learned to handle problems that involve both images and text.
Experts point out that reasoning systems do not necessarily reason like a human. And like other A.I. technologies, they can get things wrong and make stuff up — a phenomenon called hallucination.
OpenAI also unveiled a new tool, Codex CLI, that is designed to further facilitate computer programming tasks that involve systems like o3 and o4-mini. Called an A.I. agent, it provides ways of using these A.I. systems in tandem with existing code stored on a programmer’s personal machine.
The company said it was open sourcing this tool, meaning it is freely sharing its underlying technology with programmers and businesses, allowing them to modify and build on the technology.
OpenAI said that, beginning Wednesday, these new systems would be available to anyone who subscribed to ChatGPT Plus, a $20-a-month service, or ChatGPT Pro, a $200-a-month service that provides access to all of the company’s latest tools.
(The New York Times has sued OpenAI and its partner, Microsoft, for copyright infringement of news content related to A.I. systems. Both companies have denied the claims.)
