APEX-Agents · Law
World416_TK_02
APEX-Agents task World416_TK_02 in AI Agents for Tariff and Trade Law Analysis. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
TAC has just informed us that they expect a 20% drop in gross margin due to import tariffs. TAC has posted the information on their website as they have done in the past. Write me back a brief message, explaining whether this will trigger an additional 8K filing.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 0/1 | Fail | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 1/1 | Pass | Share pagePublic trace |
| Gemini 3 Flash | dual | 0/1 | Fail | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 0/1 | Fail | Share pagePublic trace |
| GPT-5.4 | dual | 0/1 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 1/1 | Pass | Share pagePublic trace |
| GPT-5.4 nano | dual | 0/1 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States that a forward-looking tariff impact does not trigger a mandatory Form 8-K submission
FailEvidence: The response says, “I do not think the website posting alone creates a separate Regulation FD 8-K obligation,” but also states, “a 20% gross-margin decline is likely material” and “TAC should furnish an additional Form 8-K—most likely under Item 7.01….” Assessment: The criterion requires stating that a forward-looking tariff impact does not trigger a mandatory Form 8-K submission. The response does not clearly make that point and instead suggests an additional 8-K may/should be furnished if it updates guidance, so it fails.