APEX-Agents · Management Consulting
world130_HO_08
APEX-Agents task world130_HO_08 in AI Agents for Employment Law Analysis. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
The client sent us employee wage data (attached), so we need to update our assumptions in the financial analysis section of the survey analysis report to display the updated annual productivity loss figures (in 000s). Find the average hourly salary of employee roles and use that to update the annual productivity loss estimate (rounded to 1 decimal place). Assume average hourly wages in the data are fully-loaded costs. Assume the following activities are non-productive: manual data entry, searching for data, and fixing errors. Report final answers here, written out in a short message.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 0/2 | Fail | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 1/2 | Fail | Share pagePublic trace |
| GPT-5.4 | dual | 0/2 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 1/2 | Fail | Share pagePublic trace |
| GPT-5.4 nano | dual | 1/2 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States the updated annual productivity loss is $50,494,000
FailEvidence: <TEXT_RESPONSE> states “Total annual productivity loss: $43,580.5 ($000s)” (i.e., $43,580,500), with component figures listed above it. Assessment: The criterion requires stating the updated annual productivity loss is $50,494,000; the response states a different total, so this fails.
States the average fully-loaded hourly wage used to calculate the annual productivity loss is $28.90
FailEvidence: <TEXT_RESPONSE> states “Using the role-count-weighted loaded hourly wage of $24.94/hr.” Assessment: The criterion requires stating the average fully-loaded hourly wage used is $28.90; the response states $24.94, so this fails.