Raycaster / evalsTask overviewAPEX-AgentsCategory

APEX-Agents · GPT-5.4 mini · dual

World 419_UM_01

6/10Fail

GPT-5.4 mini on APEX-Agents: World 419_UM_01 (dual harness). Browse score, rubric, and public trace.

6/10 · Fail
Law
AI Agents for Maritime and Environmental Liability
Law World 419

Grader rubric

Criteria verdict

  1. States that the Marine Excess Liability Insurance Policy will cover Jack Sparrow's claim

  2. States that the Marine Pollution Legal Liability Insurance Policy will cover Jack Sparrow's claim

  3. States that the Marine General Liability Insurance Policy will not cover Jack Sparrow's claim

  4. States that Article I of the Marine Excess Liability Insurance Policy is relevant to Jack Sparrow's claim

  5. States that Article IV of the Marine Excess Liability Insurance Policy is relevant to Jack Sparrow's claim

  6. States that Article VIII of the Marine Excess Liability Insurance Policy is relevant to Jack Sparrow's claim

  7. States that Article XV of the Marine Excess Liability Insurance Policy is relevant to Jack Sparrow's claim

  8. States that Article I of the Marine Pollution Legal Liability Insurance Policy is relevant to Jack Sparrow's claim

  9. States that Article II of the Marine Pollution Legal Liability Insurance Policy is relevant to Jack Sparrow's claim

  10. States that Article V of the Marine Pollution Legal Liability Insurance Policy is relevant to Jack Sparrow's claim

Prompt excerpt

Task context

We are faced with a claim from one Jack Sparrow. He alleges that his fishing boat and equipment were damaged by the M/V Red Room's pollution incident and is demanding compensation from Cooper/Jeffries. Can you go over our insurance policies and see if such a claim is covered under any of our policies? If it is covered, state what sections would be relevant for the claim. Reply to me here with all of the info I requested.

Response trace

Agent response, tools, files, and edits

Open full trace

On mobile, open the full trace for the best view of the agent response, tool calls, file reads, edits, and grading context.

Open mobile trace