OpenAI’s next flagship model may not represent as much progress as its predecessors, according to a new report in The Information.
Employees who tested the new model, codenamed Orion, found that while its performance outperformed OpenAI’s existing model, it did not see the same improvement seen with the transition from GPT-3 to GPT-4. It is reported that they discovered that.
In other words, the rate of improvement appears to be slowing. In fact, in some areas, such as coding, Orion may not be reliably better than previous models.
In response, OpenAI established a foundation team to consider how models can continue to improve as the supply of new training data dwindles. These new strategies reportedly include training Orion on synthetic data generated by AI models and further improving the models in a post-training process.
OpenAI did not immediately respond to a request for comment. In response to previous reports about the company’s plans for its flagship model, the company said, “We have no plans to release a model codenamed Orion this year.”
