APEX-Agents · Law

World416_TK_04

1/2Fail

APEX-Agents task World416_TK_04 in AI Agents for Tariff and Trade Law Analysis. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for Tariff and Trade Law AnalysisLaw World 416Dual harnessGrader: rubric

Task IDtask_5eec5f9ff1234067859443ab0a6c402d

WorldLaw World 416

Expected outputmessage_in_console

Published runs7 models · dual config

Task prompt

What the agent was asked to do

Review the two supply agreement templates, Master Supply Agreement Template 1.docx and Master Supply Agreement Template 4.pdf, along with the attached files (UCC §2-209 and Restatement (Second) of Contracts §89) to determine whether the supplier’s tariff-based request for a mid-term price increase would be a valid modification. Reply to me here with your assessment.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

Model	Harness	Score	Result	Links
GPT-5.5showcase	dual	1/2	Fail	Share page Public trace
fireworks models Kimi K2	dual	1/2	Fail	Share page Public trace
Gemini 3 Flash	dual	1/2	Fail	Share page Public trace
Gemini 3.1 Pro	dual	1/2	Fail	Share page Public trace
GPT-5.4	dual	1/2	Fail	Share page Public trace
GPT-5.4 mini	dual	1/2	Fail	Share page Public trace
GPT-5.4 nano	dual	1/2	Fail	Share page Public trace

Grading rubric

Criteria and grader verdict (showcase run)

States that a tariff-driven mid-term price-increase request is an unenforceable modification under Master Supply Agreement Template 1
Pass
Evidence: The response says, “Template 1: likely not valid unless Buyer expressly accepts in a signed writing,” and “A tariff-based mid-term price increase is not valid unilaterally under Template 1.” Assessment: The criterion asks whether it states the request is an unenforceable modification under Template 1. Pass, because the response clearly concludes it is not valid/enforceable absent Buyer’s signed written acceptance.
States that a tariff-driven mid-term price-increase request is an unenforceable modification under Master Supply Agreement Template 4
Fail
Evidence: The response says, “Template 4: likely valid only to the extent the tariff increase is handled through the template’s existing price-adjustment or duty-reimbursement provisions,” and “A tariff-based increase is likely enforceable if it is a good-faith, documented pass-through...” Assessment: The criterion asks whether it states the request is an unenforceable modification under Template 4. Fail, because the response’s main conclusion is that Template 4 likely permits/enforces the tariff increase in specified circumstances, not that it is unenforceable.