APEX-Agents · Management Consulting
World 129_CY_Task 2
APEX-Agents task World 129_CY_Task 2 in AI Agents for Divestiture and Spin-Off Analysis. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
Refer to the latest (v1.0) customer segmentation data and the final pricing model. Calculate the % change of Customer Segmentation expected revenue (use target customers and average ARR for the pricing) relative to Configuration A expected revenue (use target customers and effective pricing) for each segment. Account for churn in both scenarios based on each scenario's respective data source. Reply with your results here, with final numbers rounded to the nearest 0.01%.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 0/3 | Fail | Share pagePublic trace |
| Gemini 3 Flash | dual | 3/3 | Pass | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 3/3 | Pass | Share pagePublic trace |
| GPT-5.4 | dual | 3/3 | Pass | Share pagePublic trace |
| GPT-5.4 mini | dual | 3/3 | Pass | Share pagePublic trace |
| GPT-5.4 nano | dual | 3/3 | Pass | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States that the percentage change of Customer Segmentation expected revenue relative to Configuration A expected revenue for SMB is 51.82%
FailEvidence: <TEXT_RESPONSE> table lists SMB “% Change vs. Config A” as “-96.16%.” Assessment: Criterion requires stating SMB percentage change is 51.82%; the response gives a different value, so fail.
States that the percentage change of Customer Segmentation expected revenue relative to Configuration A expected revenue for Mid-Market is -49.17%
FailEvidence: <TEXT_RESPONSE> table lists Mid-Market “% Change vs. Config A” as “-90.82%.” Assessment: Criterion requires stating Mid-Market percentage change is -49.17%; the response gives a different value, so fail.
States that the percentage change of Customer Segmentation expected revenue relative to Configuration A expected revenue for Enterprise is -91.40%
FailEvidence: <TEXT_RESPONSE> table lists Enterprise “% Change vs. Config A” as “-90.54%.” Assessment: Criterion requires stating Enterprise percentage change is -91.40%; the response gives a different value, so fail.