APEX-Agents · Law
World433_JS_03
APEX-Agents task World433_JS_03 in AI Agents for Healthcare and Senior Living Legal Risk. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
Magnolia Gardens provides resident John Marshall with three meals. They are planned out two and a half weeks in advance, as well as two snacks each day. Is Magnolia Gardens in compliance with Part 487 of the New York Codes Rules and Regulations? Give me a clear answer back in here, and concisely explain your reasoning.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 3/3 | Pass | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 3/3 | Pass | Share pagePublic trace |
| Gemini 3 Flash | dual | 3/3 | Pass | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 3/3 | Pass | Share pagePublic trace |
| GPT-5.4 | dual | 3/3 | Pass | Share pagePublic trace |
| GPT-5.4 mini | dual | 3/3 | Pass | Share pagePublic trace |
| GPT-5.4 nano | dual | 3/3 | Pass | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States No, Magnolia Gardens is not in compliance with Part 487 of the New York Codes Rules and Regulations
PassEvidence: TEXT_RESPONSE says, “No. Magnolia Gardens is **not in compliance** with Part 487 as stated.” Assessment: Criterion requires stating no/not in compliance; pass because the response clearly states this.
States that Part 487 of the New York Codes Rules and Regulations requires menus to be planned a minimum of three weeks in advance
PassEvidence: TEXT_RESPONSE says, “the same rule also requires menus to be ‘planned a minimum of three weeks in advance’.” Assessment: Criterion requires stating Part 487 requires menus planned a minimum of three weeks in advance; pass.
States that Magnolia Gardens does not plan menus at least three weeks in advance
PassEvidence: TEXT_RESPONSE says, “Planning meals only two and a half weeks in advance is 0.5 weeks short,” and “the facility is not compliant on menu planning.” Assessment: Criterion requires stating Magnolia Gardens does not plan menus at least three weeks in advance; pass.