APEX-Agents · Law
World423_JS_03
APEX-Agents task World423_JS_03 in AI Agents for Privacy and GDPR Compliance. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
Northstar is reviewing its policies for notifying American data subjects in case there is another unauthorized data transfer. Specifically, a lot of Northstar's data subjects reside in the State of New York. Would Northstar be required, under New York Law, to notify affected New York residents, if the data contained in the Northstar health-related product identifiers were transferred to an unauthorized person? Respond to me me with a yes or no answer. Also, give a single explanation.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 2/2 | Pass | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 0/2 | Fail | Share pagePublic trace |
| Gemini 3 Flash | dual | 2/2 | Pass | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 0/2 | Fail | Share pagePublic trace |
| GPT-5.4 | dual | 2/2 | Pass | Share pagePublic trace |
| GPT-5.4 mini | dual | 2/2 | Pass | Share pagePublic trace |
| GPT-5.4 nano | dual | 2/2 | Pass | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States Yes, Northstar is required, under New York Law, to notify affected New York residents, if the data contained in the Northstar health-related product identifiers were transferred to an unauthorized person
PassEvidence: The response begins “**Yes.**” and explains that “§ 899-aa requires notice to any New York resident whose private information ‘was, or is reasonably believed to have been, accessed or acquired by a person without valid authorization,’” so “an unauthorized transfer ... would require notice to affected New York residents.” Assessment: The criterion asks whether it states Yes, Northstar is required under New York law to notify affected New York residents if the health-related product identifier data were transferred to an unauthorized person; pass.
States that the data contained in the Northstar health-related product identifiers is private information
PassEvidence: The response says New York law treats “private information” to include personal information combined with “medical information,” and concludes that the Northstar health/vital-sign records are “identifiable health-related records” covered by the notice requirement. Assessment: The criterion asks whether it states the data contained in the Northstar health-related product identifiers is private information; pass, as the response clearly characterizes the data as private information under the law.