APEX-Agents · Law

World425_tas_04

3/4Fail

APEX-Agents task World425_tas_04 in AI Agents for Tax Due Diligence. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for Tax Due DiligenceLaw World 425Dual harnessGrader: rubric

Task IDtask_b9b58e483f384c5990900ef2d8c9fe17

WorldLaw World 425

Expected outputmake_new_doc

Published runs7 models · dual config

Task prompt

What the agent was asked to do

Harbor Bridge has recently expressed concern that certain provisions of Summit's Amended and Restated Shareholder Agreement may be inconsistent with Summit's S-Corp election under 26 U.S.C. § 1361 and 26 C.F.R. § 1.1361-1(l). Please examine the shareholder agreement and draft a concise amendment to the agreement to correct any potential deficiencies which may invalidate Summit's S-Corp election. Put everything in a new docx file.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

Model	Harness	Score	Result	Links
GPT-5.5showcase	dual	3/4	Fail	Share page Public trace
fireworks models Kimi K2	dual	2/4	Fail	Share page Public trace
Gemini 3 Flash	dual	3/4	Fail	Share page Public trace
Gemini 3.1 Pro	dual	2/4	Fail	Share page Public trace
GPT-5.4	dual	3/4	Fail	Share page Public trace
GPT-5.4 mini	dual	2/4	Fail	Share page Public trace
GPT-5.4 nano	dual	2/4	Fail	Share page Public trace

Grading rubric

Criteria and grader verdict (showcase run)

States that amended Section 2.3 to require strictly pro rata distributions, in accordance with each Shareholder’s percentage ownership of the Company’s stock
Pass
Evidence: In `/root/workspace/filesystem/Summit Filing Solutions Inc./Summit_First_Amendment_S-Corp_Corrections.docx`, Section 1 replaces 2.3 with: “Any dividend… shall be made… strictly pro rata in accordance with the number of shares owned by each Shareholder…” and liquidation proceeds “distributed strictly pro rata in accordance with share ownership.” Assessment: The criterion requires amended Section 2.3 to require strictly pro rata distributions by percentage/share ownership; pass.
States that Section 2.5 is deleted in its entirety
Pass
Evidence: In the docx, Section 3 is titled “Replacement of Section 2.5” and states: “Section 2.5 of the Agreement is deleted in its entirety and replaced with the following.” Assessment: The criterion requires stating that Section 2.5 is deleted in its entirety; pass, even though the amendment also supplies replacement language.
States that except as specifically amended, the original agreement shall remain in full force and effect
Pass
Evidence: In the docx, Section 7 “Effect of Amendment; Conflicts” states: “Except as expressly amended by this Amendment, the Agreement remains in full force and effect.” Assessment: The criterion requires stating that except as specifically amended, the original agreement remains in full force and effect; pass.
States that the amendment will be governed and construed by the laws of the State of Delaware
Fail
Evidence: The docx text includes “Summit Filing Solutions, Inc., a Delaware corporation,” but no governing-law clause stating the amendment “will be governed and construed by the laws of the State of Delaware.” Assessment: The criterion specifically requires a Delaware governing/construction law statement; fail because the artifact lacks such a provision.