APEX-Agents · Law
World434_AH_05
APEX-Agents task World434_AH_05 in AI Agents for Tax Due Diligence. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
Determine the current Capital Account for Pacific Pension Fund in HarborView Fund I, LP. The following financial events have occurred since inception of the Fund (unless otherwise stated, assume Gross Asset Value is identical to adjusted cost basis): - aggregate Profit of $10,000,000 in 2024 - aggregate Loss of $10,000,000 in 2025 - reduction of Gross Asset Value of $10,000,000 for an asset ("Asset 1") that was not reflected in taxable income - assume no management or other fees need to be applied, as they are reflected in Profit and Loss disclosed above. The above includes the following: - gain on sale of Asset 1 above declared taxable gain in the amount of $10,000,000 - gain on the sale of Asset 2 of $5,000,000. Now, reply to me here with your answer.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 0/1 | Fail | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 1/1 | Pass | Share pagePublic trace |
| Gemini 3 Flash | dual | 1/1 | Pass | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 0/1 | Fail | Share pagePublic trace |
| GPT-5.4 | dual | 1/1 | Pass | Share pagePublic trace |
| GPT-5.4 mini | dual | 0/1 | Fail | Share pagePublic trace |
| GPT-5.4 nano | dual | 0/1 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States the current Capital Account for Pacific Pension Fund is US$300,000,000
FailEvidence: The response states, “Pacific Pension Fund’s current Capital Account is $298,333,333.33” and concludes “$300,000,000 - $1,666,666.67 = $298,333,333.33.” Assessment: The criterion requires that it “States the current Capital Account for Pacific Pension Fund is US$300,000,000.” The response states a different amount, so it fails.