Raycaster / evalsBack to AI Agents for Tax Due Diligence

APEX-Agents · Investment Banking

WORLD223_ES_01

1/6Fail

APEX-Agents task WORLD223_ES_01 in AI Agents for Tax Due Diligence. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for Tax Due DiligenceInvestment Banking World 223Dual harnessGrader: rubric
Task IDtask_60519c5af2d44d06a5f6885622195d50
WorldInvestment Banking World 223
Expected outputmake_new_sheet
Published runs4 models · dual config

Task prompt

What the agent was asked to do

Estimate impact to EPS of MMM divesting 1/3, 2/3 or 100% of its holdings in SOLV. Using the accretion dilution model, calculate the EPS impact of selling the different amounts of shares. Assume that MMM has to pay a 20% tax on the proceeds from the divestiture of the first 1/3 of the shares, a 25% tax on the next 1/3 and a 30% tax on the last 1/3 of the shares. Assume that net proceeds from the divestiture are invested in an instruments that pays an 8% interest rate upfront and that interest income is taxed at 35%. Tell me: 1- Percentage of SOLV Divested 2- EPS Accretion / Dilution 3- Pro forma net income Pro forma net income should incorporate the impact of investment of net proceeds from divestiture at 8% and after tax. All figures should be presented with two decimals. Output the final results by making a new Spreadsheet file.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

ModelHarnessScoreResultLinks
GPT-5.5showcasedual1/6FailShare pagePublic trace
Gemini 3.1 Produal2/6FailShare pagePublic trace
GPT-5.4 minidual1/6FailShare pagePublic trace
GPT-5.4 nanodual2/6FailShare pagePublic trace

Grading rubric

Criteria and grader verdict (showcase run)

  1. States that EPS accretion when 1/3 of SOLV shares are divested is 1.06%

    Fail

    Evidence: TEXT_RESPONSE summary and workbook Results page show 33.33% EPS Accretion / Dilution = 1.09%. Assessment: Criterion requires stating EPS accretion for 1/3 divested is 1.06%; the response/file state 1.09%, so fail.

  2. States that EPS accretion when 2/3 of SOLV shares are divested is 2.05%

    Fail

    Evidence: TEXT_RESPONSE summary and /root/workspace/.../MMM_SOLV_Divestiture_EPS_Impact.xlsx Results page show 66.67% EPS Accretion / Dilution = 2.12%. Assessment: Criterion requires 2.05%; stated value is 2.12%, so fail.

  3. States that EPS accretion when 100% of SOLV shares are divested is 2.98%

    Fail

    Evidence: TEXT_RESPONSE summary and workbook Results page show 100.00% EPS Accretion / Dilution = 3.08%. Assessment: Criterion requires 2.98%; stated value is 3.08%, so fail.

  4. States that Pro Forma Net Income when 1/3 of SOLV shares are divested is $3,437 million

    Fail

    Evidence: TEXT_RESPONSE summary and workbook Results page show Pro Forma Net Income for 33.33% = $3,438.22mm. Assessment: Criterion requires $3,437 million; the stated/file value is not $3,437 million (nor a rounding equivalent), so fail.

  5. States that Pro Forma Net Income when 2/3 of SOLV shares are divested is a value other than $3,471 million

    Pass

    Evidence: TEXT_RESPONSE summary and workbook Results page show Pro Forma Net Income for 66.67% = $3,473.11mm. Assessment: Criterion requires that the stated value be other than $3,471 million; $3,473.11mm is other than $3,471mm, so pass.

  6. States that Pro Forma Net Income when 100% of SOLV shares are divested is $3,502 million

    Fail

    Evidence: TEXT_RESPONSE summary and workbook Results page show Pro Forma Net Income for 100.00% = $3,505.67mm. Assessment: Criterion requires $3,502 million; the stated/file value is $3,505.67mm, so fail.