Raycaster / evalsAll benchmarks

Cross-benchmark category

Healthcare and Senior Living Legal Risk

Compare AI agent runs for Healthcare and Senior Living Legal Risk: prompts, scores, grades, and public traces. This page showcases APEX-Agents tasks that test whether AI agents can analyze healthcare and senior living legal risk, including facility liability, resident care issues, and regulated operations.

Healthcare legal AIFacility liability, resident care risk, regulated healthcare operations
50Tasks
19Primary
1Benchmarks

Published model evidence

A quick view of shareable runs in this category. Open task and trace pages for the prompt, response, files, and grading details.

Gemini 3 Flash53% pass

17/32 runs · avg score 76%

gpt-5.542% pass

21/50 runs · avg score 68%

gpt-5.4-mini38% pass

19/50 runs · avg score 62%

Gemini 3.1 Pro36% pass

18/50 runs · avg score 58%

Kimi K230% pass

14/46 runs · avg score 58%

GPT-5.430% pass

14/47 runs · avg score 64%

Primary tasks

19 tasks with this category as their main focus.

  1. Law 433_AC_02APEX-Agents(task_1d7f4e0be1274b069ad86be4ed813c46)4/10 · Fail

    Please review Grove’s 2023 and 2025 operations manuals. Let me know if there are any changes in the 2025 manual that may present issues for Grove’s regulatory compliance. For any regulatory issues ide…

  2. Law433_mk_02APEX-Agents(task_3742720383624ce58ef4d751f8836392)5/7 · Fail

    Can you edit the existing Grove Residency Agreement for Thomas Whitaker to make a revised residency agreement template that we can use for new residents. Please also update/add to the last sentence o…

  3. LawWorld433_ANB_03APEX-Agents(task_e72676c169294461adff044bee6a5587)3/3 · Pass

    Does 15A NCAC 18A .1600 apply to food service in Magnolia Gardens' North Carolina facilities? Create a new documents document and respond in there.

  4. LawWorld433_ANB_04APEX-Agents(task_ea50530a09d5440996d0328aa3fbaa70)3/3 · Pass

    Magnolia Gardens is looking to renew its 2025 coverage through Valiant Assurance Ltd. for 2026, in Texas. Valiant has asked for a list including the number of patients they serve that utilize Medicaid…

  5. LawWorld433_ANB_05APEX-Agents(task_3caff6fe3a9f4a9f95e57f371a16b601)1/1 · Pass

    Sarah Lee was deposed in the case brought by Plaintiff, Daisy Buchanan. Plaintiff intends to call Ms. Lee to testify at trial. Identify which statements in paragraph 8 of her deposition transcript are…

  6. LawWorld433_NAF_01APEX-Agents(task_4fa914d8cc0f4010bb661d3e688a4c61)2/5 · Fail

    Heather Collins has started to let her care staff all take naps at the same time. Legally, can she do this? Provide me with a brief explanation of your answer, citing the relevant sources. Reply back …

  7. LawWorld433_NAF_02APEX-Agents(task_ccee33f3839a412f91ea0821a487a26b)1/3 · Fail

    The attorney for Thomas Whitaker wants to submit this matter for arbitration in Texas. Reply to me in here, drafting an output explaining whether that would be consistent with Mr. Stoker's Litigation …

  8. World_421_ANB_01APEX-Agents(task_501a59d44c6846e8a6dd3ee4a9373abf)6/8 · Fail

    We have a call tomorrow with SLL. To help me prepare, can you tell me whether SLL’s new text marketing campaign meets the strictest consent requirements of the TCPA? And is this legal standard still c…

  9. World_421_ANB_02APEX-Agents(task_ed8356c4f4c146b58d2cd869924fdfe3)1/5 · Fail

    Our client, Senior Living Lending, Inc. ("SLL") is a reverse mortgage and home equity line of credit lender. They want to implement a telemarketing program that relies heavily upon texting potential b…

  10. World_421_ANB_03APEX-Agents(task_2f4081fb460f4ce79a091be3ffb64b93)7/9 · Fail

    Senior Living Lending, Inc. ("SLL") emailed me because they are concerned that responses to their ad campaigns may fall under the Telemarketing Sales Rule (“TSR”). Can you please draft the content for…

  11. World223_CG_01APEX-Agents(task_498f648416184645bf732a0b4a94c40b)0/3 · Fail

    Compare the average tenure of Solventum's board members to the average of its peers' board members (excluding Solventum). Give their average tenure. Then, state which is larger and by how many years. …

  12. World223_JTR_01APEX-Agents(task_c44daee40b2441539fb532efa0a08319)0/4 · Fail

    We want to do some historical analysis. Using stock prices for the 250 trading days up to Nov 19, 2025, calculate the beta for SOLVM to the XLV ETF. Then over the same time period, calculate the beta …

  13. World421_EL_03APEX-Agents(task_8a22b80c7b2b49c59d6deed5b94d6857)2/6 · Fail

    Can you please let me know if we are likely to succeed in getting a TCPA class action against SLL dismissed through a Rule 12 motion? Give me a brief explanation (in one or two sentences) and let me k…

  14. World421_OO_02APEX-Agents(task_fe573b8ce38d4a9f9642fbe7b8f11358)5/10 · Fail

    Our client, SLL, needs to understand whether their new customer marketing initiative is legal and what risks are likely. In this initiative, SLL is creating marketing texts and sending them to custom…

  15. World433_JS_01APEX-Agents(task_be78a90e872547eea75915f95ca35739)3/3 · Pass

    Magnolia Gardens has a contract with the Texas Department of Human Services (DHS) to provide services, as an assisted living facility, to DHS clients. Gregory Johns--a disabled DHS client who is in a …

  16. World433_JS_03APEX-Agents(task_a2a4615b0b46416c905729e49595c686)3/3 · Pass

    Magnolia Gardens provides resident John Marshall with three meals. They are planned out two and a half weeks in advance, as well as two snacks each day. Is Magnolia Gardens in compliance with Part 4…

  17. World433_JS_04APEX-Agents(task_fe1efa5d3aaa43469f135b53fe02b6a2)2/2 · Pass

    Magnolia Gardens has Adult Care Home facilities in North Carolina. One of the North Carolina Adult Care Home facilities has 25 residents. It has a 400-square-foot enclosed indoor recreational area in …

  18. World433_JS_08APEX-Agents(task_1d28a1ef3e8c44eeac211afdad56ccca)3/3 · Pass

    We are in the process of doing compliance checks on the Magnolia Gardens General Resident Agreement Template. We are currently examining whether the General Resident Agreement Template is in complianc…

  19. World433_MMF_02APEX-Agents(task_680f8581e29f4aefb3a9bc2a8f77bbb0)3/8 · Fail

    Review Dr. Anderson's expert report and analyze whether (1) her opinions as to cause of death are admissible and (2) whether you believe there is a greater than an 80% chance of surviving a summary ju…

Related tasks

31 tasks that also exercise this work as part of a broader assignment.

  1. World417_TG_01APEX-Agents(task_d8119ffe94d4444dbfb6626cf3eab3a2)6/9 · Fail

    We just received a demand letter from Isaiah’s counsel. He alleges wrongful termination and FMLA interference. Can you look into the validity of his claims and return me back a write-up of what you fi…

  2. Law433_mk-01APEX-Agents(task_908f411c435d4ebf8fcfc4a08e7dce5b)3/4 · Fail

    We have received a claim from Thomas Whitaker’s counsel asserting that Magnolia breached its obligations by failing to provide weekly communication to Mr. Whitaker's family. Provide the strongest argu…

  3. LawWorld417_NE_06APEX-Agents(task_76c43bb44ab54ab8b42269303dedf83a)1/6 · Fail

    We will layoff our Head of Production Sarah Rodriguez as part of the upcoming merger. Please review Section IV of Sarah's layoff and WARN Notice and let me know if the language violates WARN. Answer t…

  4. LawWorld433_ANB_01APEX-Agents(task_de1ec76029b54126ae1105921e4202d4)3/3 · Pass

    Susan Whitaker has filed three successive Complaints against Magnolia Gardens. The first is the original Complaint, the second Is the First Amended Complaint, and the third is the Second Amended Compl…

  5. LawWorld433_ANB_02APEX-Agents(task_b15e945697824a9ab869c8e9a402f5c1)4/4 · Pass

    Jennifer Love was the nurse assigned to Mr. Buchanan's floor, and was later designated as the corporate designee of Magnolia Gardens Inc during depositions. She is called by Magnolia Gardens Inc., to …

  6. Task 2cc51b31APEX-Agents(task_cdaf73e0d7ce466a8338793974c13d22)1/1 · Pass

    A doctor was recently charged by the federal government under the same regulations that were the subject of the case in "US v. Caronia, 703 F. 3d 149 - Court of Appeals, 2nd Circuit 2012.pdf" for pres…

  7. Task p5y453dbAPEX-Agents(task_48d70f58c409446fb1e22ca7572dd1c8)0/1 · Fail

    Livyra has scrapped the "Focus on What Matters" campaign. They intend to replace it with a new campaign advertising to the general population which states normal approved audio messages based on depre…

  8. World 420 LB_05APEX-Agents(task_fa15dfc96f4b4a289fac616a6f3b4a1a)2/2 · Pass

    For purposes of a training presentation for sales representatives giving background on what off-label promotion is and its legal status, please indicate whether it would be accurate to incorporate the…

  9. World 420_dds_03APEX-Agents(task_8b881ab96d114632bfdbe3ad35684f33)5/9 · Fail

    Prepare an analysis regarding Livyra's Bencontra promotional activities. Take a look at the case file, paying close attention to the email communications as well as the relevant case law, statutes and…

  10. World 420_LB_02APEX-Agents(task_1918e229773b46699fe5576f1b11d7a2)2/3 · Fail

    Identify for each item below whether it could be used by the U.S. Food and Drug Administration (FDA) as evidence that Bencontra is intended by Livyra for use in pediatric patients: 1. bencontra_provi…

  11. World 420_LB_03APEX-Agents(task_e9c600e0ce46420eb983f8f1ed4d15d9)5/5 · Pass

    Based on new information showing increased reporting of seizures in Bencontra patients with no prior history of seizures, Livyra is planning to update the prescribing information for Bencontra to incl…

  12. World 420_LB_04APEX-Agents(task_9e919cde8791468cadce433c3c9eef09)1/6 · Fail

    Livyra would like to put out a simple direct-to-consumer (DTC) advertisement for Bencontra. In an effort to avoid anything that could be construed as off-label promotion, the DTC advertisement would n…

  13. World 420_LB_06APEX-Agents(task_3ff7f71e606840d09b4d64bd88398799)0/2 · Fail

    You are an attorney. Please review the document regarding circulation issues associated with Bencontra. Does Livyra's dissemination of this document to a doctor who prescribes Bencontra appear to meet…

  14. World 420_ZT_02APEX-Agents(task_4fe676ac270b404c9104c758bd3964dc)1/1 · Pass

    Review the attached email (stored as a docx file), as well as the cases available in the world. Write a message to me, outlining whether John’s actions constitute unlawful off-label promotion.

  15. World 421_OO_01APEX-Agents(task_8702e946cbad4a56886fcd7ea18cd5b2)6/10 · Fail

    Our client, SLL, offers discounts to senior clients who opt into receiving texts about new products and services when they apply for loans from SLL. A prospective customer, Angie, agreed to receive …

  16. World112-1_Task04_TBAPEX-Agents(task_d7893ddf035a4abca2990717ad4d7428)4/4 · Pass

    Given the US imposed tariffs on the import of pharmaceutical materials from Germany, India, China at the rates 10%, 15%, 20% respectively, determine the impact of tariffs as a percent increase in pred…

  17. World227_JZ_Task03APEX-Agents(task_fc96166d8f374e4eb8d4d15784904b8e)0/3 · Fail

    The Private Equity Sponsor wants to extract cash via a "Dividend Recapitalization" at the end of 2027. Using the MFC model, you must size the Maximum Special Dividend the company can pay while remaini…

  18. World420_RO_01APEX-Agents(task_bdd8216b4e864b9a81d0b9d7af624497)2/3 · Fail

    Livyra wants to market Bencontra using just its name and price. What is the correct type of advertisement Livyra can make, if any, and what conditions must the advertisement meet? Answer in one senten…

  19. World420_SF_02APEX-Agents(task_d06f10ebd2ad4bb19ba47b00a590dd54)2/2 · Pass

    Livyra has decided to pay its independent contractors a fixed salary, consistent with fair market value, to sell Bencontra that is payable by government healthcare. Each independent contractor is give…

  20. World423_JS_02APEX-Agents(task_7e51ed8994924d8d9f92938fd8cf9fd2)5/9 · Fail

    It has come to our attention that some of the data transferred by the "Diagnostics Analytics Module" related to residents of Colorado. Does Colorado Law require us to notify Colorado residents of this…

  21. World425_tas_01APEX-Agents(task_876ace32decb4f26a3f7a7c3bf50bab7)6/6 · Pass

    Review the due diligence file for Summit Filing Solutions, Inc. and identify any potential deficiencies relative to its claimed status as an S-Corporation. Draft a set of indemnities, for incorporati…

  22. World425_tas_05APEX-Agents(task_ed6f8d835b0141309442d2c373d1c5da)5/8 · Fail

    As you know, Harbor Bridge Private Equity sent initial inquiries to Summit Filing Solutions ("Summit") on matters related to Summit's S-Corp election. Laura Kensington, Summit's Acting CEO, responded…

  23. World433_JS_05APEX-Agents(task_828f11b8209f4b53bdf1e836762363d8)4/6 · Fail

    We are doing a compliance review to ensure that the General Resident Agreement Template complies with all applicable State laws and regulations. Please review the General Resident Agreement Template…

  24. World433_JS_06APEX-Agents(task_912055978b48483696be01adf161ec3b)3/3 · Pass

    We are conducting a compliance review of Magnolia Gardens' General Resident Agreement. We are currently determining whether the General Resident Agreement is in compliance with Kentucky Law. Please Re…

  25. World433_JS_07APEX-Agents(task_729286e684134595958f962e72f2aa20)3/3 · Pass

    We are conducting a compliance review of Magnolia Gardens' General Resident Agreement. We are currently determining whether the General Resident Agreement is in compliance with Kentucky Law. Please re…

  26. World433_MMF01APEX-Agents(task_b5481555a1c94da6bf78baf87165851c)5/10 · Fail

    Can you please draft a short legal memorandum analyzing whether the eggshell skull rule applies with respect to the plaintiffs' claims against us in the wrongful death complaint? Please also conclude …

  27. World433_SG_01APEX-Agents(task_f40a571a80e944cead66ecf1d3f0de93)2/2 · Pass

    Grove Garden Living ("Grove Garden") is going to be filing a motion for summary judgment for the Wrongful Death claim in the complaint that was filed by Daisy Buchanon (the "Plaintiff") against Grove …

  28. World434_DPM_01APEX-Agents(task_85f48fa8ecc04d9eb5a56a6e95aa3a88)3/3 · Pass

    Review the URA Land Search Report, based on the relevant provision of the Planning Act, what is the potential maximum financial penalty regarding TOP/2018/04576 as of January 1, 2025? Assume that the …

  29. World434_DPM_02APEX-Agents(task_6790bd5747ca479b810c4e33bf423c02)6/7 · Fail

    As of January 1, 2025, SecureBox Self Storage has not made any changes to the Emergency Exit Signage and Lighting. If the FSD inspection report is considered a fire safety direction, what is the maxim…

  30. World434_IG_01APEX-Agents(task_173fa342dcb244929dc2e7c4dad12a2b)3/3 · Pass

    We had a number of compliance memoranda drafted regarding our ongoing acquisition of the storage facility companies. For the third memo listed in the compendium, do we have to comply with all the laws…

  31. World434_IG_02APEX-Agents(task_3e2c326f5470427f9ac0f627c72da1d6)3/4 · Fail

    One of the Hong Kong facilities is going to overhaul its fire safety after they found that some of the equipment from the supplier produced faulty sprinklers. This same supplier produced essentially a…