APEX-Agents · Law
World433_JS_04
APEX-Agents task World433_JS_04 in AI Agents for Healthcare and Senior Living Legal Risk. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
Magnolia Gardens has Adult Care Home facilities in North Carolina. One of the North Carolina Adult Care Home facilities has 25 residents. It has a 400-square-foot enclosed indoor recreational area in the center of the single building facility. Does the recreational area comply with Title 10A, Subchapter 13F, Section .0305 of the North Carolina Administrative Code (10A NCAC 13F .0305)? Provide an answer with a single-sentence explanation.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 2/2 | Pass | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 0/2 | Fail | Share pagePublic trace |
| Gemini 3 Flash | dual | 2/2 | Pass | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 2/2 | Pass | Share pagePublic trace |
| GPT-5.4 | dual | 2/2 | Pass | Share pagePublic trace |
| GPT-5.4 mini | dual | 2/2 | Pass | Share pagePublic trace |
| GPT-5.4 nano | dual | 1/2 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States that Title 10A, Chapter 13F, section .0305 of the North Carolina Administrative Code provides that recreational areas must have windows with views to the outside
PassEvidence: TEXT_RESPONSE says the separate requirement is that the required area “shall have windows with views to the outside.” Assessment: The criterion asks whether it states that 10A NCAC 13F .0305 provides recreational areas must have windows with outside views; this is clearly stated, so pass.
States that the recreational area does not comply with 10A NCAC 13F .0305
PassEvidence: TEXT_RESPONSE begins, “No,” and explains that an enclosed recreational area in the center of the building “does not satisfy” the outside-window requirement. Assessment: The criterion asks whether it states the recreational area does not comply with 10A NCAC 13F .0305; the response clearly does, so pass.