APEX-Agents · Law
World433_JS_07
APEX-Agents task World433_JS_07 in AI Agents for Real Estate Lease Review. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
We are conducting a compliance review of Magnolia Gardens' General Resident Agreement. We are currently determining whether the General Resident Agreement is in compliance with Kentucky Law. Please review the General Resident Agreement and determine if it is in compliance with Kentucky Revised Statutes (KRS) section 194A.713(8). Please provide a short paragraph in reply to me right here, explaining your reasoning.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 3/3 | Pass | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 3/3 | Pass | Share pagePublic trace |
| Gemini 3 Flash | dual | 3/3 | Pass | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 3/3 | Pass | Share pagePublic trace |
| GPT-5.4 | dual | 2/3 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 3/3 | Pass | Share pagePublic trace |
| GPT-5.4 nano | dual | 3/3 | Pass | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States that under Kentucky Revised Statutes (KRS) section 194A.713(8) a lease agreement shall include a minimum thirty (30) day move-out notice provision for resident nonpayment
PassEvidence: The response says KRS 194A.713(8) “requires a lease agreement to include a ‘Minimum thirty (30) day move-out notice provision for resident nonpayment.’” Assessment: This directly states the criterion that the statute requires a minimum 30-day move-out notice provision for resident nonpayment; pass.
States that the General Resident Agreement includes a provision that requires the facility to provide a minimum of thirty (30) days notice for discharge due to non-payment
PassEvidence: The response states the Agreement lists “Non-payment of fees” as a discharge ground and provides that “you will be given at least 30 days advanced notice prior to the date of discharge.” Assessment: This satisfies the criterion by stating the Agreement includes at least 30 days’ notice for discharge due to non-payment; pass.
States that the General Resident Agreement complies with KRS section 194A.713(8)
PassEvidence: The response opens: “The General Resident Agreement appears substantially compliant with KRS 194A.713(8)” and later says “the core 30-day notice requirement for nonpayment is present.” Assessment: The response states compliance with KRS 194A.713(8), albeit with a drafting improvement noted; pass.