APEX-Agents · Law
World433_JS_05
APEX-Agents task World433_JS_05 in AI Agents for Real Estate Lease Review. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
We are doing a compliance review to ensure that the General Resident Agreement Template complies with all applicable State laws and regulations. Please review the General Resident Agreement Template and determine if it complies with Kentucky Revised Statutes 194A.713(14). Provide an answer right here in a short paragraph, explaining your reasoning, based on the General Resident Agreement Template.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 4/6 | Fail | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 3/6 | Fail | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 3/6 | Fail | Share pagePublic trace |
| GPT-5.4 | dual | 4/6 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 4/6 | Fail | Share pagePublic trace |
| GPT-5.4 nano | dual | 6/6 | Pass | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States that the lease agreement must contain grievance policies that minimally address issues related to confidentiality of complaints under Kentucky Revised Statute (KRS) section 194A.713(14)
PassEvidence: TEXT_RESPONSE says, “KRS 194A.713(14) requires the lease agreement to include grievance policies that ‘minimally address issues related to confidentiality of complaints…’” Assessment: The criterion requires stating the lease must contain grievance policies addressing confidentiality of complaints; this is clearly stated. Pass.
States that the lease agreement must contain grievance policies that minimally address issues related to the process for resolving grievances between the resident and the assisted living community under KRS section 194A.713(14)
PassEvidence: TEXT_RESPONSE says the statute requires policies addressing “the process for resolving grievances” and specifies “between the resident and assisted living community.” Assessment: The criterion requires stating the lease must contain grievance policies minimally addressing the process for resolving grievances between those parties; the response does so. Pass.
States that the General Resident Agreement Template does not directly contain grievance policies that minimally address issues related to confidentiality of complaints
PassEvidence: TEXT_RESPONSE says the template “does **not** address confidentiality of complaints” and “that statutory element is missing.” Assessment: The criterion requires stating the template does not directly contain grievance policies minimally addressing confidentiality of complaints; the response clearly conveys that. Pass.
States that the General Resident Agreement Template incorporates a resident's rights policy
FailEvidence: TEXT_RESPONSE discusses a grievance process and contacts but does not mention a resident’s rights policy or incorporation of such a policy. Assessment: The criterion requires stating that the General Resident Agreement Template incorporates a resident’s rights policy; this is absent. Fail.
States that the resident's rights policy has not been provided
FailEvidence: TEXT_RESPONSE does not say that any resident’s rights policy “has not been provided”; it only says the template lacks confidentiality language. Assessment: The criterion requires stating that the resident’s rights policy was not provided; this is not included. Fail.
States that the General Resident Agreement Template may not comply with KRS section 194.713(14)
PassEvidence: TEXT_RESPONSE begins “No” and concludes the template “does not comply with KRS 194A.713(14) as drafted.” Assessment: The criterion requires stating the template may not comply with KRS section 194.713(14); the response states noncompliance more definitively, which satisfies the required position. Pass.