The just-released AI Safety Index rated six leading AI companies on their risk assessment efforts and safety procedures…topping the class was Anthropic, with an overall score of C. The remaining five companies – Google DeepMind, Meta, OpenAI, xAI and Zhipu AI – were rated below D+, and Meta was a complete failure.
“The purpose of this is not to shame anyone,” said Max Tegmark, a professor of physics and director of the Future of Life Institute at the Massachusetts Institute of Technology, who published the report. “It’s to give companies an incentive to improve.” He hopes corporate executives look at the index the same way universities look at U.S. News and World Report rankings. They may not like being graded, but once their grades are out there and they get attention, they’ll be more motivated to do better next year.
He also wants to support researchers working on safety teams at these companies. Tegmark says that if a company doesn’t feel external pressure to meet safety standards, “other people in the company will see you as just a nuisance, someone who’s trying to slow things down and throw gravel in the machine.” I would consider it to be.” But if these safety researchers suddenly become responsible for improving a company’s reputation, they gain resources, respect, and influence.
The Future of Life Institute is a nonprofit organization dedicated to helping humanity avoid the truly negative consequences of powerful technologies, with a recent focus on AI. In 2023, the group released a document, later known as a “pause letter,” calling on the AI Institute to pause development of advanced models for six months and use that time to develop safety standards. . Celebrities like Elon Musk and Steve Wozniak signed the letter (to date, 33,707 people have signed it in total), but the companies didn’t stop.
This new report may also be ignored by the companies in question. IEEE Spectrum reached out to all companies for comment, but only Google DeepMind responded with the following statement: “While this index incorporates some of Google DeepMind’s AI safety efforts and reflects industry-adopted benchmarks, our comprehensive approach to AI safety We are beyond that. We remain committed to continually evolving our safety measures as technology advances.”
How companies are rated using the AI Safety Index
The index evaluated companies’ performance in six categories: risk assessment, current hazards, safety framework, survival safety strategy, governance and accountability, and transparency and communication. We used publicly available information such as relevant research papers, policy documents, news articles, and industry reports. The judges also sent out a survey to each company, but only xAI and Chinese company Zhipu AI (which currently has the most qualified Chinese LLMs) responded to the survey, making them less transparent. The scores of these two companies increased.
The scoring was carried out by seven independent reviewers, including luminaries such as University of California, Berkeley professor Stuart Russell and Turing Award winner Joshua Bengio. He said it could lead to. The reviewers include AI leaders who have focused on short-term harms of AI, such as algorithmic bias and harmful language, such as Carnegie Mellon University’s Atusa Kashirzadeh and Encode Justice founder Sneha Revanur. Included.
And overall, reviewers were not impressed. “The findings of the AI Safety Index Project suggest that although there is a lot of activity going on under the heading of ‘safety’ in AI companies, it is still not very effective. ” says Russell. The activity provides quantitative safety guarantees of all kinds. And given the current approach to AI via giant black boxes trained on unimaginably vast amounts of data, it seems impossible to provide such guarantees. And as these AI systems get bigger, it’s going to get even harder. In other words, the current direction of technology may never be able to support the necessary safety guarantees, in which case it is truly a dead end. ”
Anthropic received the highest overall and specific scores, and the only B- for its current harm efforts. The report states that Anthropic’s models have the highest scores on key safety benchmarks. The company also has a “Responsible Scaling Policy,” which requires it to evaluate its models for their potential to cause catastrophic harm and not deploy models it deems too risky.
All six companies performed particularly poorly on survival safety strategies. Reviewers found that while all companies have expressed an intention to build artificial general intelligence (AGI), few have articulated any strategies to ensure that AGI remains aligned with human values. I pointed out that Anthropic, Google DeepMind, and OpenAI are the only ones. “The truth is, no one knows how to control new species that are much smarter than us,” Tegmark says. “The review committee felt that even if (the company) had some kind of early-stage strategy, it was not sufficient.”
Although the report does not make any recommendations to AI companies or policymakers, Tegmark said its findings could be used for regulatory oversight, the equivalent of the U.S. Food and Drug Administration approving AI products before they go to market. I feel strongly that this clearly demonstrates the need for government agencies. .
“I feel like the leaders of these companies, no matter how kind-hearted they are, are in a race to the bottom that no one can escape from,” Tegmark says. Currently, companies are reluctant to slow down their safety testing because they don’t want to lose out to competitors in the market, he said. “On the other hand, if you have a safety standard, there is instead a commercial pressure on who can meet the safety standard first, because then they can be the first to sell and the first to make money. Because you can.”
From an article on your site
Related articles on the web
