APEX-Agents · Law

World421_js_01

4/4Pass

APEX-Agents task World421_js_01 in AI Agents for Cross-Border Regulatory Review. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for Cross-Border Regulatory ReviewLaw World 421Dual harnessGrader: rubric

Task IDtask_df4f7e06025945458402b6e3e712ea64

WorldLaw World 421

Expected outputmessage_in_console

Published runs7 models · dual config

Task prompt

What the agent was asked to do

Please check these four faxes (sent on December 4, 2025) and for each one that isn't compliant, state which subsection(s) of the TCPA it violates. Note that every recipient consented to the receipt of the faxes. Respond to me with the information that I requested.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

Model	Harness	Score	Result	Links
GPT-5.5showcase	dual	4/4	Pass	Share page Public trace
fireworks models Kimi K2	dual	2/4	Fail	Share page Public trace
Gemini 3 Flash	dual	4/4	Pass	Share page Public trace
Gemini 3.1 Pro	dual	4/4	Pass	Share page Public trace
GPT-5.4	dual	4/4	Pass	Share page Public trace
GPT-5.4 mini	dual	2/4	Fail	Share page Public trace
GPT-5.4 nano	dual	4/4	Pass	Share page Public trace

Grading rubric

Criteria and grader verdict (showcase run)

States Fax2.pdf violates 47 U.S.C. § 227(d)(1)(B)
Pass
Evidence: In the table, the row for `Fax2.pdf` says "Noncompliant? Yes" and "TCPA subsection(s) violated | 47 U.S.C. § 227(d)(1)(B)." Assessment: Criterion asks whether the response states Fax2.pdf violates 47 U.S.C. § 227(d)(1)(B); pass because it explicitly does so.
States Fax3.pdf violates 47 U.S.C. § 227(d)(1)(B)
Pass
Evidence: In the table, the row for `Fax3.pdf` says "Noncompliant? Yes" and "TCPA subsection(s) violated | 47 U.S.C. § 227(d)(1)(B)." Assessment: Criterion asks whether the response states Fax3.pdf violates 47 U.S.C. § 227(d)(1)(B); pass because it explicitly does so.
States Fax4.pdf violates 47 U.S.C. § 227(d)(1)(B)
Pass
Evidence: In the table, the row for `Fax4.pdf` says "Noncompliant? Yes" and "TCPA subsection(s) violated | 47 U.S.C. § 227(d)(1)(B)." Assessment: Criterion asks whether the response states Fax4.pdf violates 47 U.S.C. § 227(d)(1)(B); pass because it explicitly does so.
States Fax5.pdf violates 47 U.S.C. § 227(d)(1)(B)
Pass
Evidence: In the table, the row for `Fax5.pdf` says "Noncompliant? Yes" and "TCPA subsection(s) violated | 47 U.S.C. § 227(d)(1)(B)." Assessment: Criterion asks whether the response states Fax5.pdf violates 47 U.S.C. § 227(d)(1)(B); pass because it explicitly does so.