AI safety and research company Anthropic calls for proposals to evaluate advanced models

Anthropic, a company that does research into AI safety, is calling for proposals on ways to evaluate advanced learning models.

The rapid growth of AI means there are new AI providers and models all the time. Every major tech company has its own model, while there are dozens more smaller ones as well. That means that the industry has a rising benchmarking problem, making it tough to accurately evaluate how well an AI model performs.

Not only is it hard to state how effective a model is, but it’s also hard to evaluate the risks involved with AI safety. Anthropic is calling for proposals to plug this gap in AI evaluation.

“Developing high-quality, safety-relevant evaluations remains challenging, and the demand is outpacing the supply,” the company writes. “To address this, today we’re introducing a new initiative to fund evaluations developed by third-party organizations that can effectively measure advanced capabilities in AI models.”

The main focus of the proposals should be centered around: AI Safety Level assessments; advanced capability and safety metrics; and infrastructure, tools, and methods for developing evaluations. There should be details on tests that can assess an AI model’s ability to accomplish tasks ranging from cyberattacks, working on weapons of mass destruction, including nuclear weapons, and creating deepfakes or misinformation).

How to submit a proposal on AI safety evaluation

You can read more details on each of the sections that Anthropic is calling for proposals on and submit a proposal on the research company’s website. The team is reviewing submissions on a rolling basis and will follow up with select proposals to discuss next steps.

Anthropic has allocated various levels of funding to help get those research options to their next stages, as well as offering the possibility to talk directly with experts from across their in-house research and safety teams.

Featured image: Ideogram