OpenAI releases GeneBench-Pro for testing AI in biology

OpenAI introduced GeneBench-Pro, a benchmark for testing whether AI agents can make hard research judgments in biology. It does not only check whether a model can follow a clean recipe; it tests whether the model can inspect messy data, choose a sound analysis path, revise weak assumptions, and decide whether a result is ready to support a real decision. The benchmark has 129 questions across 10 areas, including genomics, quantitative biology, clinical diagnostics, cancer genomics, and protein analysis.

The problems use synthetic data, so the builders know the true answer structure and can grade results more reliably. Some problems were reviewed by outside experts to check whether they felt realistic and difficult enough. Each AI agent gets a short prompt, data files, and a workspace with Python and basic biology analysis tools, then has to do the work end to end.

OpenAI says GPT-5.6 Sol passed 28.7% of the questions at its highest reasoning setting and 31.5% in Pro mode, compared with less than 5% for GPT-5 when the earlier GeneBench work began. Even the best model still solves fewer than one third of the tasks, so today’s AI looks more useful as a research assistant than as a replacement for expert scientists. Anthropic’s Claude Science launch points in the same direction: major AI labs are moving from general chatbots toward specialized scientific workspaces.