APEX-Agents · Law

World 420_ZT_02

1/1Pass

APEX-Agents task World 420_ZT_02 in AI Agents for FDA and Pharma Promotional Compliance. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for FDA and Pharma Promotional ComplianceLaw World 420Dual harnessGrader: rubric

Task IDtask_4fe676ac270b404c9104c758bd3964dc

WorldLaw World 420

Expected outputmessage_in_console

Published runs7 models · dual config

Task prompt

What the agent was asked to do

Review the attached email (stored as a docx file), as well as the cases available in the world. Write a message to me, outlining whether John’s actions constitute unlawful off-label promotion.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

Model	Harness	Score	Result	Links
GPT-5.5showcase	dual	1/1	Pass	Share page Public trace
fireworks models Kimi K2	dual	0/1	Fail	Share page Public trace
Gemini 3 Flash	dual	0/1	Fail	Share page Public trace
Gemini 3.1 Pro	dual	0/1	Fail	Share page Public trace
GPT-5.4	dual	0/1	Fail	Share page Public trace
GPT-5.4 mini	dual	1/1	Pass	Share page Public trace
GPT-5.4 nano	dual	1/1	Pass	Share page Public trace

Grading rubric

Criteria and grader verdict (showcase run)

States that John's actions do not constitute an unlawful off-label promotion
Pass
Evidence: The response states John’s email is “not, by itself, a clear ‘smoking gun’ of unlawful off-label promotion” and concludes: “John’s email alone likely does not establish completed unlawful off-label promotion… rather than a definitive legal violation on the email alone.” Assessment: The criterion asks whether it states that John's actions do not constitute unlawful off-label promotion. Pass, because the response clearly gives that bottom-line conclusion, albeit with compliance-risk qualifications.