APEX-Agents · Law

World 420_LB_06

0/2Fail

APEX-Agents task World 420_LB_06 in AI Agents for FDA and Pharma Promotional Compliance. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for FDA and Pharma Promotional ComplianceLaw World 420Dual harnessGrader: rubric

Task IDtask_3ff7f71e606840d09b4d64bd88398799

WorldLaw World 420

Expected outputmessage_in_console

Published runs7 models · dual config

Task prompt

What the agent was asked to do

You are an attorney. Please review the document regarding circulation issues associated with Bencontra. Does Livyra's dissemination of this document to a doctor who prescribes Bencontra appear to meet the standard for misbranding under 21 U.S.C. 352(a)? And does it appear to provide support for a misbranding finding under 21 U.S.C. 352(f)? Please give a "yes" or "no" answer to these questions; you do not need to elaborate on your reasoning. Output your answers as a message in the console. Also, for purposes of these questions, you should assume that (1) the document is being distributed in response to a doctor's question about circulation issues associated with Bencontra and (2) any factual information in the document is true. In addition to the the document regarding circulation, please consider the following sources: 1. 21 U.S.C. 352 2. FDA approval letter for Bencontra 3. 21 CFR 202.1 4. FDA First Amendment memo

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

Model	Harness	Score	Result	Links
GPT-5.5showcase	dual	0/2	Fail	Share page Public trace
fireworks models Kimi K2	dual	2/2	Pass	Share page Public trace
Gemini 3 Flash	dual	2/2	Pass	Share page Public trace
Gemini 3.1 Pro	dual	2/2	Pass	Share page Public trace
GPT-5.4	dual	2/2	Pass	Share page Public trace
GPT-5.4 mini	dual	2/2	Pass	Share page Public trace
GPT-5.4 nano	dual	1/2	Fail	Share page Public trace

Grading rubric

Criteria and grader verdict (showcase run)

States Yes, Livyra’s dissemination of the document to a doctor who prescribes Bencontra appears to meet the 21 U.S.C. 352(a) misbranding standard
Fail
Evidence: <TEXT_RESPONSE> states, “21 U.S.C. 352(a): No.” Assessment: The criterion requires stating Yes that dissemination appears to meet the 21 U.S.C. 352(a) misbranding standard. The response states the opposite, so fail.
States Yes, Livyra’s dissemination of the document to a doctor who prescribes Bencontra appears to provide support for a finding of misbranding under 21 U.S.C. 352(f)
Fail
Evidence: <TEXT_RESPONSE> states, “21 U.S.C. 352(f): No.” Assessment: The criterion requires stating Yes that dissemination appears to provide support for a misbranding finding under 21 U.S.C. 352(f). The response states the opposite, so fail.