APEX-Agents · Investment Banking

World228_JP_02

0/2Fail

APEX-Agents task World228_JP_02 in AI Agents for FDA and Pharma Promotional Compliance. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for FDA and Pharma Promotional ComplianceInvestment Banking World 228Dual harnessGrader: rubric

Task IDtask_6a1f6778265e4fc99811f7ebf71c205c

WorldInvestment Banking World 228

Expected outputmessage_in_console

Published runs4 models · dual config

Task prompt

What the agent was asked to do

Assume that 10% of Aptar's Asian pharma revenue will shift entirely into European pharma revenue. Retroactively apply this to 2024 financials. Refer to the 2024 ATR report. Return a short message with the values for Asia Pharma and Europe Pharma for their corresponding revenue under this set of assumptions. Round dollar amounts to the nearest thousand.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

Model	Harness	Score	Result	Links
GPT-5.5showcase	dual	0/2	Fail	Share page Public trace
Gemini 3.1 Pro	dual	2/2	Pass	Share page Public trace
GPT-5.4 mini	dual	2/2	Pass	Share page Public trace
GPT-5.4 nano	dual	2/2	Pass	Share page Public trace

Grading rubric

Criteria and grader verdict (showcase run)

States the revenue corresponding to Asia is $209,502,000
Fail
Evidence: <TEXT_RESPONSE> states “Asia Pharma revenue: $162,672,000.” Assessment: Criterion requires stating Asia revenue is $209,502,000; the response gives a different value, so it fails.
States the revenue corresponding to Europe is $829,884,000
Fail
Evidence: <TEXT_RESPONSE> states “Europe Pharma revenue: $823,219,000.” Assessment: Criterion requires stating Europe revenue is $829,884,000; the response gives a different value, so it fails.