APEX-Agents · Law
Task Seed #13
APEX-Agents task Task Seed #13 in AI Agents for Privacy and GDPR Compliance. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
During the first 48 minutes of the EU production outage, Northstar's engineering team exported one or two bundled sets of EU production event logs containing personal data to the U.S. analytics vendor. However, no ongoing or continuous log streaming had yet been configured. Reply back to me here and explain if, Under Northstar's own policies, it can reasonably treat the one or two log exports as consistent with Article 49?
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 1/1 | Pass | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 0/1 | Fail | Share pagePublic trace |
| Gemini 3 Flash | dual | 1/1 | Pass | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 1/1 | Pass | Share pagePublic trace |
| GPT-5.4 | dual | 1/1 | Pass | Share pagePublic trace |
| GPT-5.4 mini | dual | 1/1 | Pass | Share pagePublic trace |
| GPT-5.4 nano | dual | 1/1 | Pass | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States Yes, Northstar can reasonably treat the log exports as consistent with Article 49 under Northstar's own policies
PassEvidence: TEXT_RESPONSE says, “Yes — on the facts you gave, Northstar can reasonably treat the one or two bundled log exports as potentially consistent with Article 49 under its own policies,” and concludes “Northstar can reasonably characterize the one or two bundled outage-response exports as Article 49-consistent under its own policies.” Assessment: The criterion asks whether it states Yes, Northstar can reasonably treat the log exports as consistent with Article 49 under its own policies. Pass; the response clearly states this, with appropriate qualifications.