APEX-Agents · Law
World434-TK-01
APEX-Agents task World434-TK-01 in AI Agents for Privacy and GDPR Compliance. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
Post-closing, the Fire Department issued a fine for pre-completion non-compliance at one of the facilities, and the buyer paid the fine to avoid operational disruption. Take a look at the Secure Box SPA indemnity structure. Can we recover that amount from the seller under the specific indemnity, or is there a Hong Kong public-policy issue with indemnifying regulatory penalties? I want to know whether the indemnity in the contract works or whether this is a gap. Summarize your answers briefly and reply with a message here.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 1/3 | Fail | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 0/3 | Fail | Share pagePublic trace |
| Gemini 3 Flash | dual | 1/3 | Fail | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 3/3 | Pass | Share pagePublic trace |
| GPT-5.4 | dual | 2/3 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 1/3 | Fail | Share pagePublic trace |
| GPT-5.4 nano | dual | 0/3 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States that under Hong Kong law, there is no public-policy prohibition on indemnification between private parties
PassEvidence: The response says, “I would not treat public policy as a categorical bar on these facts” and “public policy is a secondary enforcement risk, not the primary reason recovery is uncertain.” Assessment: The criterion asks whether it states that under Hong Kong law there is no public-policy prohibition on indemnification between private parties. This is conveyed in substance, despite some qualification about regulatory wrongdoing risk. Pass.
States that the SPA directs the seller to reimburse the buyer for a loss suffered as a result of a pre-completion non-compliance fine
FailEvidence: The response states the opposite: “the fire-safety indemnity appears to cover remediation-type costs, not fines,” and “the fire-safety specific indemnity likely does not cleanly recover the fine.” Assessment: The criterion asks whether it states that the SPA directs the seller to reimburse the buyer for a loss suffered as a result of a pre-completion non-compliance fine. It does not; it identifies only a possible, less clean general indemnity route and a drafting gap. Fail.
States that the indemnity works as drafted, under Hong Kong law
FailEvidence: The response concludes, “unless Schedule 8.02(e) expressly covers FSD fines/penalties, the fire-safety specific indemnity likely does not cleanly recover the fine” and “This is a drafting gap.” Assessment: The criterion asks whether it states that the indemnity works as drafted, under Hong Kong law. The response states the indemnity likely does not work cleanly as drafted. Fail.