APEX-Agents · Law
World433_MMF_02
APEX-Agents task World433_MMF_02 in AI Agents for Healthcare and Senior Living Legal Risk. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
Review Dr. Anderson's expert report and analyze whether (1) her opinions as to cause of death are admissible and (2) whether you believe there is a greater than an 80% chance of surviving a summary judgment motion on a crumbling skull rule affirmative defense to the complaint if her report is admitted. The case will be heard in New York. Support your assertions with facts from the case files. Provide your answer in a new documents document that you create.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 3/8 | Fail | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 3/8 | Fail | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 0/8 | Fail | Share pagePublic trace |
| GPT-5.4 | dual | 2/8 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 2/8 | Fail | Share pagePublic trace |
| GPT-5.4 nano | dual | 5/8 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States that Dr Anderson claims that Mr. Buchanan's death was triggered by an unknown event rather than a missed insulin dose
PassEvidence: In the DOCX, Dr. Anderson is described as making an “unsupported suggestion of an unidentified infection or other internal stressor” and stating “the omissions were not the direct or proximate cause.” Assessment: The criterion requires stating she claims death was triggered by an unknown event rather than a missed insulin dose; pass because the document conveys an unidentified stressor/infection theory and contrasts it with medication omissions.
States that Dr Anderson claims that Tom Buchanan’s demise is "generally" triggered by an unknown psychological stressor
FailEvidence: The DOCX refers to an “unidentified infection or other internal stressor” and a “Speculative infection trigger.” Assessment: The criterion requires stating that the demise is “generally” triggered by an unknown psychological stressor. The document does not mention a psychological stressor or use the “generally” framing; fail.
States that Dr Anderson claims that Tom Buchanan’s demise is generally not triggered by a singular medication omission
PassEvidence: The DOCX states Dr. Anderson opines “the omissions were not the direct or proximate cause” and discusses a “conclusory statement that the BGL was already at a fatal threshold independent of the 07:00 omission.” Assessment: The criterion requires stating she claims the demise is generally not triggered by a singular medication omission; pass because the document clearly conveys her view that the missed dose/omission was not the direct cause.
States that Dr Anderson's opinion as to cause of death is insufficient as a matter of law to preclude summary judgment on the crumbling skull rule
FailEvidence: The DOCX concludes that if her opinion is admitted “the defense has a greater than 80% chance of surviving summary judgment - approximately 85%” and that “Dr. Anderson creates a triable medical-causation issue.” Assessment: The criterion requires stating her opinion is insufficient as a matter of law to preclude summary judgment on the crumbling skull rule. The document states the opposite for survival of the defense; fail.
States that Dr Anderson's speculative statements on undiagnosed stressors are not admissible evidence
FailEvidence: The DOCX says “the court may limit or discount opinions that rest on an unidentified infection” and that the infection/internal-stressor theory could be viewed as “speculation,” but its overall conclusion is that her opinions are “likely admissible.” Assessment: The criterion requires stating speculative statements on undiagnosed stressors are not admissible evidence. The document does not clearly say they are inadmissible, only vulnerable/discountable; fail.
States that affidavits devoid of evidentiary facts are insufficient to defeat a motion for summary relief
FailEvidence: The DOCX discusses that experts must raise a triable issue with “competent evidence” and recommends a sworn affirmation, but it does not state that affidavits devoid of evidentiary facts are insufficient to defeat summary judgment. Assessment: The criterion requires that specific legal proposition; fail.
States that Dr Anderson does not specify the undiagnosed event that caused Tom Buchanan's death
PassEvidence: The DOCX repeatedly describes “an unidentified infection or other internal stressor” and says “Without hospital labs, cultures, fever data... a court could view the infection trigger as speculation.” Assessment: The criterion requires stating Dr. Anderson does not specify the undiagnosed event; pass because the report characterizes the alleged event/stressor as unidentified and unsupported.
States that there is not a greater than 80% chance of surviving summary judgment
FailEvidence: The DOCX says “I believe the defense has a greater than 80% chance of surviving summary judgment - approximately 85%,” while separately saying a defense motion for full dismissal has only “35%-45%.” Assessment: The criterion requires stating there is not a greater than 80% chance of surviving summary judgment. The document affirmatively states greater than 80% for surviving the affirmative-defense summary judgment issue; fail.