APEX-Agents · Investment Banking

World228_SM_Task07

0/1Fail

APEX-Agents task World228_SM_Task07 in AI Agents for FDA and Pharma Promotional Compliance. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for FDA and Pharma Promotional ComplianceInvestment Banking World 228Dual harnessGrader: rubric

Task IDtask_a13be532a65d4938b2397ce17aa071d8

WorldInvestment Banking World 228

Expected outputmessage_in_console

Published runs4 models · dual config

Task prompt

What the agent was asked to do

In The Aptar Group's 2024 annual report, extract global revenue for the Pharma segment in 2023 and 2024. Compute the aggregate dollar change in thousands of USD adjusted for 7.1% 2024 global inflation and express the result in USD, rounded to the nearest thousand. Print back to me what you find please.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

Model	Harness	Score	Result	Links
GPT-5.5showcase	dual	0/1	Fail	Share page Public trace
Gemini 3.1 Pro	dual	0/1	Fail	Share page Public trace
GPT-5.4 mini	dual	0/1	Fail	Share page Public trace
GPT-5.4 nano	dual	0/1	Fail	Share page Public trace

Grading rubric

Criteria and grader verdict (showcase run)

States the aggregate dollar change in revenue from FY2023-FY2024 is $13,299,000
Fail
Evidence: TEXT_RESPONSE states “Rounded to nearest thousand: **$14,168,000**” and “Result: ... approximately $14,168,000.” Assessment: The criterion requires stating the aggregate dollar change is “$13,299,000”; the response states a different value, so it fails.