APEX-Agents · Law
LawWorld417_NAF_02
APEX-Agents task LawWorld417_NAF_02 in AI Agents for Employment Law Analysis. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
Our client, William Ito, shares custody of a child with Sarah Rodriguez. Can you please review his email (pasted below) and tell me how much child support needs to be paid and by whom? Assume the gross to net income conversion table and income share schedules for 2025 are the same as 2026. Please also give me the basic child support obligation. State all amounts to the nearest whole dollar value (but don't round any values when calculating). Then, in a single paragraph, please outline the key factual assumptions you made for the calculation based on the relevant employment documents on file and the relevant laws. Reply back to me with your answers as a message in here. ** Here is the part of William's email I'd like to know how much child support I should pay, since Sarah's likely gonna be unemployed for the next while (we were both at Chasing). I want her to take her time since she's a bit burnt out. At any rate, I think it'll probably take her three years to find a new job since there are limited positions and minimal turn-around for head of production roles; it's also unlikely that she'll find the same role. We agreed to be bound by Illinois family law in our separation agreement, and I'm currently on the following temporary parenting time schedule: every Monday and Tuesday overnight, and every other Friday to school on Monday morning.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 3/10 | Fail | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 4/10 | Fail | Share pagePublic trace |
| Gemini 3 Flash | dual | 7/10 | Fail | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 1/10 | Fail | Share pagePublic trace |
| GPT-5.4 | dual | 4/10 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 5/10 | Fail | Share pagePublic trace |
| GPT-5.4 nano | dual | 4/10 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States that Sarah Rodriguez has to pay child support to William Ito
FailEvidence: TEXT_RESPONSE states, “William Ito should pay Sarah Rodriguez $885 per month.” Assessment: Criterion requires stating Sarah Rodriguez has to pay child support to William Ito; fail because the response states the opposite payer/payee.
States that the monthly child support payment is $195
FailEvidence: TEXT_RESPONSE states “$885 per month.” Assessment: Criterion requires monthly child support payment is $195; fail because the amount stated is $885, not $195.
States that the basic child support obligation is $2,059
FailEvidence: TEXT_RESPONSE states “Basic child support obligation: $1,177 per month.” Assessment: Criterion requires basic child support obligation is $2,059; fail because the response gives $1,177.
States that William Ito has 182 overnights per year
PassEvidence: TEXT_RESPONSE states “I counted William’s schedule as 182 overnights per year.” Assessment: Criterion requires stating William Ito has 182 overnights per year; pass.
States that Sarah Rodriguez has 183 overnights per year
PassEvidence: TEXT_RESPONSE states “Sarah’s as 183 overnights.” Assessment: Criterion requires stating Sarah Rodriguez has 183 overnights per year; pass.
States that William Ito's gross income is based on his salary of $95,000 per year
PassEvidence: TEXT_RESPONSE states “William’s annual gross income at $95,000 from his Chasing Streams employment agreement” and “William’s $95,000/year.” Assessment: Criterion requires stating William Ito's gross income is based on his salary of $95,000 per year; pass.
States that Sarah Rodriguez's income is based on the following: (1) her employment history; (2)the availability of local employers willing to hire her, (3) the prevailing earning levels in the local community, and (4) other relevant job market factors
FailEvidence: TEXT_RESPONSE says Sarah has “no ongoing monthly gross income…after the layoff” and mentions an “expected permanent separation,” but does not list employment history, local employers willing to hire her, prevailing local earning levels, or other job market factors as bases. Assessment: Criterion requires all four listed bases; fail.
States that Sarah Rodriguez's income can be imputed by agreement of the parties
FailEvidence: TEXT_RESPONSE does not state that Sarah Rodriguez’s income can be imputed by agreement of the parties; it only says “we agreed to be bound by Illinois family law” indirectly from the prompt and treats her income as $0. Assessment: Criterion requires this statement; fail.
States that Sarah Rodriguez's gross income is based on both: (1) her severance of $375,000, and (2) the expected period of her unemployment of 3 years
FailEvidence: TEXT_RESPONSE states Sarah has “no ongoing monthly gross income” and “did not treat her future severance as recurring monthly wage income”; it mentions a 3-year job search only as from the email context, not as a basis for gross income. Assessment: Criterion requires stating Sarah’s gross income is based on both severance of $375,000 and expected 3-year unemployment; fail.
States that the gross to net income conversion table is used to deduct the standardized tax amount from each parents' gross income
FailEvidence: TEXT_RESPONSE states William’s gross income “falls in the gross-to-net table row producing $5,798 monthly net income,” but does not state that the table is used to deduct the standardized tax amount from each parent’s gross income. Assessment: Criterion requires that specific explanation; fail.