Обзор бенчмарков

Crowd-sourced pairwise preference ranking from live human evaluations.

Strict instruction-following compliance across formatting and constraint tasks.

57-subject test of world knowledge and problem solving across STEM, humanities, social sciences.

Expert-level questions across physics, chemistry, biology that resist search-engine retrieval.

Competition-level mathematics problems requiring multi-step reasoning.

High school and competition math problems across algebra, geometry, number theory.

Python function synthesis from docstrings — measures pass@1 accuracy.

Real-world GitHub issue resolution across popular open-source repositories.

Contamination-free code generation from recent competitive programming problems.

College-level multimodal reasoning across art, business, science, health, humanities, tech.