APEX-Agents · Law
Task p5y453db
APEX-Agents task Task p5y453db in AI Agents for FDA and Pharma Promotional Compliance. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
Livyra has scrapped the "Focus on What Matters" campaign. They intend to replace it with a new campaign advertising to the general population which states normal approved audio messages based on depression treatment in adults but is supplemented with images of happy children hugging their parents. Looking at the guidance from the FDA, "Guidance from FDA on Unapproved Uses.pdf", as well as case law such as US v. Caronia, "US v. Caronia, 703 F. 3d 149 - Court of Appeals, 2nd Circuit 2012.pdf", will Livyra be in violation of FDA regulations regarding off-label marketing?
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 0/1 | Fail | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 0/1 | Fail | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 1/1 | Pass | Share pagePublic trace |
| GPT-5.4 | dual | 0/1 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 0/1 | Fail | Share pagePublic trace |
| GPT-5.4 nano | dual | 0/1 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States that Livyra's proposed campaign does not violate the FDA guidelines on communications with medical professionals
FailEvidence: The response opens, “Short answer: likely yes — or at least the proposed replacement campaign creates a serious FDA off-label-promotion/misbranding risk.” It also says the guidance “applies to firm-initiated scientific information shared with HCPs, not general-population advertising” and that the proposed campaign is “outside the FDA guidance’s narrow scientific-information safe harbor.” Assessment: The criterion requires the response to state that Livyra’s campaign “does not violate the FDA guidelines on communications with medical professionals.” The response does not state non-violation; it concludes likely violation/risk and distinguishes HCP guidance as inapplicable/outside the safe harbor. Fail.