APEX-Agents · Law

World415_im_02

0/3Fail

APEX-Agents task World415_im_02 in AI Agents for SEC Disclosure Analysis. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for SEC Disclosure AnalysisLaw World 415Dual harnessGrader: rubric

Task IDtask_729336fef1614d1a91692ac2bb442bd5

WorldLaw World 415

Expected outputmessage_in_console

Published runs7 models · dual config

Task prompt

What the agent was asked to do

Review only the following: CrowdStrike Form 8-K 2024.pdf Form 8-K Standard.pdf DELTA AIR LINES, INC. 8-K.pdf Today, August 9, 2024, we discovered that the Falcon sensor outage may have been caused by a cyberattack. Our cybersecurity firm is processing the data and has advised they will have a conclusive determination in 7 days, but for now, believes that there is a 30% chance that a cyberattack was the underlying cause. Determine if the Company should file a Form 8-K, and if so, what date and under what item the disclosure should be made. Respond back to me here with your findings.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

Model	Harness	Score	Result	Links
GPT-5.5showcase	dual	0/3	Fail	Share page Public trace
fireworks models Kimi K2	dual	3/3	Pass	Share page Public trace
Gemini 3 Flash	dual	3/3	Pass	Share page Public trace
Gemini 3.1 Pro	dual	0/3	Fail	Share page Public trace
GPT-5.4	dual	2/3	Fail	Share page Public trace
GPT-5.4 mini	dual	0/3	Fail	Share page Public trace
GPT-5.4 nano	dual	2/3	Fail	Share page Public trace

Grading rubric

Criteria and grader verdict (showcase run)

States that CrowdStrike should file a Form 8-K
Fail
Evidence: TEXT_RESPONSE says, “the Company does **not** have a mandatory Form 8-K filing obligation today” and “**Do not** file an Item 1.05 Form 8-K today,” with only an “Optional disclosure before confirmation.” Assessment: The criterion requires stating CrowdStrike should file a Form 8-K. The response does not clearly state that; it recommends no mandatory filing and frames any filing as optional. Fail.
States that CrowdStrike should file the Form 8-K by August 15, 2024
Fail
Evidence: TEXT_RESPONSE states, “Filing deadline: **No later than Thursday, August 22, 2024**” if confirmed/material, and “file under **Item 1.05 by August 22, 2024**.” Assessment: The criterion requires stating the Form 8-K should be filed by August 15, 2024. The response gives August 22 instead and does not mention August 15 as the deadline. Fail.
States that CrowdStrike should disclose the incident under Item 8.01 "Other Events" of Form 8-K
Fail
Evidence: TEXT_RESPONSE says, “If the Company decides it should update the market now… the appropriate route would likely be a **voluntary Item 8.01 — Other Events** filing,” but its recommendation is “Do **not** file an Item 1.05 Form 8-K today.” Assessment: The criterion requires stating CrowdStrike should disclose the incident under Item 8.01 “Other Events.” The response identifies Item 8.01 only as a conditional/optional route, not as the required disclosure item. Fail.