APEX-Agents · Law
Law_World_419_WA_02
APEX-Agents task Law_World_419_WA_02 in AI Agents for Maritime and Environmental Liability. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
Draft a pre-litigation legal memorandum that addresses CJ's status, financial exposure, and potential defenses under the Oil Protection Act of 1990. Create a new docx file, containing your memo.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 2/9 | Fail | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 0/9 | Fail | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 0/9 | Fail | Share pagePublic trace |
| GPT-5.4 | dual | 6/9 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 0/9 | Fail | Share pagePublic trace |
| GPT-5.4 nano | dual | 6/9 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States that CJ is a "responsible party" under 33 U.S.C. § 2701(32) based on charterer/operator status
FailEvidence: /root/workspace/CJ_OPA90_Pre_Litigation_Memorandum.docx, p.1 Executive Summary says “C/J has a strong argument that it is not the OPA 90 ‘responsible party’”; p.3 states “C/J should take the position that Star Tankers, not C/J, is the statutory vessel responsible party” and that C/J was a “voyage charterer.” Assessment: Criterion requires stating CJ is a responsible party under §2701(32) based on charterer/operator status. The memo states the opposite, so fail.
States that being a responsible party triggers strict liability for removal costs and damages under § 2702(a)
PassEvidence: /root/workspace/CJ_OPA90_Pre_Litigation_Memorandum.docx, p.3 Governing OPA Framework states “OPA 90 imposes strict liability on each responsible party… 33 U.S.C. § 2702(a)” and “Covered costs include… removal costs… and specified categories of damages.” Assessment: Criterion requires stating responsible-party status triggers strict liability for removal costs and damages under §2702(a). This is clearly included, so pass.
States that § 2704(a)(1) establishes a liability limit of $2,500 per gross ton
FailEvidence: /root/workspace/CJ_OPA90_Pre_Litigation_Memorandum.docx, p.3 says the record used “29,847 GT x $1,200/GT” and the supplied excerpt states “$1,900/GT”; p.8 Appendix B lists “Alternative § 2704 excerpt calculation: 29,847 GT x $1,900/GT.” Assessment: Criterion requires stating §2704(a)(1) establishes $2,500 per gross ton. The memo does not state $2,500/GT and instead gives $1,200 or $1,900/GT, so fail.
States that § 2704(a)(1) includes a statutory liability minimum of $21,521,000
FailEvidence: /root/workspace/CJ_OPA90_Pre_Litigation_Memorandum.docx, p.3 discusses vessel limits but gives “$35,816,400” and “$56,709,300”; p.8 Appendix B repeats those results. Assessment: Criterion requires stating §2704(a)(1) includes a statutory liability minimum of $21,521,000. That figure/minimum is not stated in the memo, so fail.
States that the gross ton liability limit is $74,617,500
FailEvidence: /root/workspace/CJ_OPA90_Pre_Litigation_Memorandum.docx, p.3 states a statutory-excerpt calculation “would produce $56,709,300”; p.8 Appendix B lists “NPFC/COFR vessel limit… $35,816,400” and “Alternative… $56,709,300.” Assessment: Criterion requires stating the gross ton liability limit is $74,617,500. The memo states different limits and not $74,617,500, so fail.
States that the gross ton liability is greater than $21,521,000
FailEvidence: /root/workspace/CJ_OPA90_Pre_Litigation_Memorandum.docx, p.3 gives vessel limits of “$35,816,400” and “$56,709,300”; p.8 Appendix B repeats those amounts. Assessment: Criterion requires stating that the gross ton liability is greater than $21,521,000. Although the stated amounts are numerically greater, the memo does not state or compare against the $21,521,000 statutory minimum, so this required statement is missing; fail.
States that CJ's liability cap is $74,617,500
FailEvidence: /root/workspace/CJ_OPA90_Pre_Litigation_Memorandum.docx, p.4 table says if CJ loses status defense, “case-specific $35.8M vs. statutory-excerpt $56.7M”; p.8 Appendix B lists $35,816,400 and $56,709,300. Assessment: Criterion requires stating CJ's liability cap is $74,617,500. The memo does not state this cap and uses other figures, so fail.
States that the third-party sole negligence under Section 2703(a)(3) is the strongest defense
FailEvidence: /root/workspace/CJ_OPA90_Pre_Litigation_Memorandum.docx, p.2 Short Answers labels “complete third-party defense if C/J is treated as a responsible party” as weak; p.6 states “Do not rely on an OPA § 2703 complete third-party defense… this defense is unlikely to be clean even as a fallback.” Assessment: Criterion requires stating third-party sole negligence under §2703(a)(3) is the strongest defense. The memo says the opposite, so fail.
States that the third-party defense requires the defendant to prove that it had no contractual relationship with a third party affecting vessel operation
PassEvidence: /root/workspace/CJ_OPA90_Pre_Litigation_Memorandum.docx, p.3 states the third-party defense “excludes employees, agents, and third parties whose acts or omissions occur in connection with a contractual relationship with the responsible party”; p.6 applies this, noting “Given C/J's contracts with Star and BLPL, this defense is unlikely.” Assessment: Criterion requires stating the third-party defense requires proving no relevant contractual relationship with the third party. The memo conveys this contractual-relationship bar, so pass.