APEX-Agents · Law
World425_tas_07
APEX-Agents task World425_tas_07 in AI Agents for Tax Due Diligence. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
We have been asked to determine whether the S-corporation election of Summit Filing Solutions, Inc. ("Summit"), which Summit claims was effective on 1/1/19, was terminated. Please analyze Summit's Shareholder Agreement, Summit's Income Schedule, and the Office Lease Agreement between Summit and Anderson Instruments. Please disregard issues arising under 26 U.S.C. sec. 1361(b)(1). Please prepare a concise memorandum in a new DOCX file you make, briefly explaining your conclusions and citing to appropriate authority.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 6/9 | Fail | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 3/9 | Fail | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 4/9 | Fail | Share pagePublic trace |
| GPT-5.4 | dual | 6/9 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 3/9 | Fail | Share pagePublic trace |
| GPT-5.4 nano | dual | 4/9 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States that Summit's S-corporation election was terminated
PassEvidence: DOCX brief answer says, “Subject to the accumulated-earnings-and-profits caveat below, Summit's S election terminated effective January 1, 2022,” and conclusion repeats it terminated “if Summit had accumulated C corporation earnings and profits.” Assessment: Criterion asks whether it states Summit's S-corporation election was terminated. Pass, though the memo states the conclusion conditionally.
States that under 26 U.S.C. § 1362(d)(3), where a corporation has retained earnings and profits derived during a period preceding its S-corporation election for each of three consecutive taxable years and has, during each of those three years, passive investment income which exceeds 25% of its gross receipts, its S-corporation election is terminated
PassEvidence: Governing law section states termination applies when “for each of three consecutive taxable years, the corporation (i) has accumulated earnings and profits at year-end and (ii) has gross receipts more than 25 percent of which are passive investment income,” citing I.R.C. § 1362(d)(3)(A), and that it is effective after the third year. Assessment: Criterion asks for this rule under 26 U.S.C. § 1362(d)(3). Pass; the memo clearly states the three-year accumulated E&P/passive income rule, despite using “accumulated earnings and profits” rather than the rubric’s “retained” wording.
States that under 26 C.F.R. § 1.1362-2(c)(5), passive investment income includes gross receipts derived from property rental
PassEvidence: Governing law section says, “Passive investment income generally includes gross receipts derived from rents. I.R.C. section 1362(d)(3)(C)(i); Treas. Reg. section 1.1362-2(c)(5)(i).” Assessment: Criterion asks whether it states that passive investment income includes gross receipts derived from property rental under 26 C.F.R. § 1.1362-2(c)(5). Pass.
States that although 26 C.F.R. § 1.1362-2(c)(5)(ii)(B)(2) provides that rents derived in the active trade or business of renting property are generally excepted from passive income, the exception applies to circumstances in which the corporation provides significant services or incurs substantial costs in its rental activities and excludes “net” leases
PassEvidence: The memo states the rental exception “requires significant services or substantial costs in the rental business,” and that regulations provide such services/costs generally are not present “in connection with net leases.” Assessment: Criterion asks for the active rental exception, significant services/substantial costs, and exclusion of net leases. Pass.
States that Summit’s Gross Income Schedule shows that Summit had earnings and profits from tax year 2018, which preceded its S-Corporation election
FailEvidence: Application section says, “The reviewed documents establish a pre-S C corporation period in 2018 and show substantial 2018 gross receipts, but they do not by themselves establish Summit's accumulated earnings and profits...” Assessment: Criterion asks whether it states Summit’s Gross Income Schedule shows earnings and profits from tax year 2018. Fail; the memo expressly distinguishes gross receipts from E&P and says the schedule does not establish E&P.
States that Summit did all of the following which retained Summit's 2018 earnings and profits: minimize expenses, paying no payroll to shareholder-employees until 2024, and making no distributions to shareholders until early 2025
FailEvidence: The DOCX contains no statement that Summit minimized expenses, paid no payroll to shareholder-employees until 2024, or made no distributions until early 2025; instead it says E&P requires confirmation from “Form 1120, E&P workpapers, distribution records, or other corporate-tax records.” Assessment: Criterion asks for all listed actions retaining 2018 E&P. Fail.
States that Summit would have retained its 2018 earnings and profits for at least three years after its S-Corporation election
FailEvidence: The memo says “Accumulated E&P is the remaining factual condition” and that reviewed documents “do not by themselves establish Summit's accumulated earnings and profits at the close of 2019, 2020, and 2021.” Assessment: Criterion asks whether it states Summit would have retained 2018 E&P for at least three years after the S election. Fail; the memo states this only as an unconfirmed condition, not a conclusion.
States that Summit’s gross receipts derived from property rental exceeded 25% of Summit’s gross receipts for each of the three years following 2018
PassEvidence: Application section table lists passive rent percentages for 2019, 2020, and 2021 of 32.4%, 32.0%, and 31.3%, and says “The base-rent percentage exceeded 25 percent in each of those years.” Assessment: Criterion asks whether it states property-rental gross receipts exceeded 25% for each of the three years following 2018. Pass.
States that the triple-net provisions of the Office Lease Agreement lead to the conclusion that the exception of 26 C.F.R. § 1.1362-2(c)(5)(ii)(B)(2) does not apply to the rental income derived through that lease
PassEvidence: Brief answer states the lease is “a triple-net arrangement” and rent is passive unless services/costs exception applies; Application states additional-rent provisions make it “a net lease in substance” and “Summit is not providing significant services or incurring substantial unreimbursed costs,” so base rent is passive. Assessment: Criterion asks whether triple-net lease provisions lead to conclusion that the regulatory exception does not apply. Pass.