APEX-Agents · Law
LawWorld433_ANB_03
APEX-Agents task LawWorld433_ANB_03 in AI Agents for Healthcare and Senior Living Legal Risk. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
Does 15A NCAC 18A .1600 apply to food service in Magnolia Gardens' North Carolina facilities? Create a new documents document and respond in there.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 3/3 | Pass | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 1/3 | Fail | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 2/3 | Fail | Share pagePublic trace |
| GPT-5.4 | dual | 2/3 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 1/3 | Fail | Share pagePublic trace |
| GPT-5.4 nano | dual | 1/3 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States No, 15A NCAC 18A .1600 does not apply to Magnolia Gardens' North Carolina facilities
PassEvidence: In `/root/workspace/filesystem/Magnolia Documents/15A NCAC 18A .1600 Applicability Memo.docx`, Executive Answer says “15A NCAC 18A .1600 should not be treated as the controlling food-service sanitation rule for Magnolia Gardens' standard North Carolina...facilities” and Bottom Line says “No - not for the Magnolia North Carolina facilities described...” Assessment: The criterion requires stating No, .1600 does not apply to Magnolia Gardens' NC facilities; pass, with the document’s stated assumptions/qualification for 13+ resident adult care homes.
States that 15A NCAC 18A .1600 does not apply to an adult care home with 13 or more residents
PassEvidence: The document states “facilities with licensed capacity of 13 or more must comply with 15A NCAC 18A .1300” and “it would not apply to food service in a 13-plus-bed Magnolia adult care home.” Assessment: The criterion requires stating .1600 does not apply to an adult care home with 13 or more residents; pass.
States that in North Carolina, Magnolia Gardens facilities have 20-30 residents
PassEvidence: The document’s Executive Answer says the reviewed materials include “a 20-30 resident North Carolina facility model,” and Application section says “the North Carolina pricing menu discusses a 20-30 resident facility.” Assessment: The criterion requires stating that in North Carolina, Magnolia Gardens facilities have 20-30 residents; pass, as the memo identifies the NC facility model as 20-30 residents.