Raycaster / evalsBack to AI Agents for Employment Law Analysis

APEX-Agents · Law

LawWorld417_AS_03

5/7Fail

APEX-Agents task LawWorld417_AS_03 in AI Agents for Employment Law Analysis. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.

AI Agents for Employment Law AnalysisLaw World 417Dual harnessGrader: rubric
task_01fef67b5576468d854319d4d8a84988
Law World 417
message_in_console
6 models · dual config

Task prompt

What the agent was asked to do

As you know, “Bare Bones Decor” - Ava Kim’s limited series - is near completion. We have been collaborating with Ava on this production for months, with the understanding that Streams would retain the exclusive rights to broadcast it and then package it as an online course, with a 50/50 revenue split with Ava. This was all documented in email exchanges and in meetings with Ava’s counsel, but we have yet to draft an exclusivity contract for the work. Now that the ParaZon acquisition is on the horizon, it’s crucial that we distinguish “Bare Bones Decor” from Ava’s other non-exclusive, transferable content (as per her general talent contract). Can you draft an amendment to her talent contract that outlines Bare Bones Decor’s exclusivity arrangement? Draft it in such a way that it is incorporated into the Asset Purchase Agreement with ParaZon. We do not want to transfer our exclusive rights to Bare Bones and would like to do so without getting ParaZon involved. Just print your assessment to me here.

Published trajectories

Agent runs on this task

Curated dual-harness runs (parsed + original sandbox). Best scored run per model.

ModelHarnessScoreResultLinks
GPT-5.5showcasedual5/7Fail
fireworks models Kimi K2dual4/7Fail
Gemini 3.1 Produal0/7Fail
GPT-5.4dual5/7Fail
GPT-5.4 minidual0/7Fail
GPT-5.4 nanodual6/7Fail

Grading rubric

Criteria and grader verdict (showcase run)

  1. States that the agreement amends the original talent contract between Ava Kim and Streams

    Pass

    Evidence: The draft is titled “AMENDMENT NO. 1 AND PROJECT EXCLUSIVITY ADDENDUM TO TALENT AGREEMENT” and says “Except as expressly modified by this Addendum, the Agreement remains in full force and effect.” Assessment: The criterion asks whether it states the agreement amends the original talent contract between Ava Kim and Streams. Pass; the response clearly frames it as an amendment/addendum to the existing Talent Agreement between Streams and Ava Kim.

  2. States that the scope of the contract is specifically limited to licensure of the video series "Bare Bones Decor."

    Pass

    Evidence: The response says the addendum “constitute[s] that separate Exclusivity Addendum solely with respect to the project currently known as ‘Bare Bones Decor’” and defines the Project as the “‘Bare Bones Decor’ ... limited-series ... audiovisual production.” Assessment: The criterion asks whether scope is specifically limited to licensure of the video series. Pass; the response repeatedly limits the addendum and rights to Bare Bones Decor.

  3. States that the agreement grants Streams an exclusive license to Bare Bones Decor

    Pass

    Evidence: Section 3 is headed “Exclusive grant to Streams” and states “Talent hereby confirms and grants to Company an exclusive, worldwide, perpetual right and license.” It also says “The rights granted in this Section are exclusive to Company.” Assessment: The criterion asks whether it grants Streams an exclusive license to Bare Bones Decor. Pass.

  4. States that the permitted use of the series includes a one-time broadcast on mainstream media

    Fail

    Evidence: The response permits Streams to “broadcast, stream, transmit, exhibit, distribute...” the Project. Assessment: The criterion specifically asks whether permitted use includes “a one-time broadcast on mainstream media.” Fail; the response includes broadcast rights generally, but does not state a one-time broadcast or mainstream media limitation/use.

  5. States that the permitted use of the series includes a direct-to-consumer sale as an online course

    Fail

    Evidence: The response allows Streams to “package, compile, and otherwise prepare the Project as an online course or instructional product” and references “online-course ... revenues.” Assessment: The criterion asks whether permitted use includes a direct-to-consumer sale as an online course. Fail; it mentions online-course packaging/revenues, but does not state direct-to-consumer sale rights.

  6. States that Streams' exclusive license to the series is non-transferable

    Pass

    Evidence: Section 7 is headed “Non-assignment; retained asset; no ParaZon transfer” and states the Bare Bones Rights “shall not be assigned, transferred, delegated, sublicensed, sold... or included in any asset sale... without” specified approvals. Assessment: The criterion asks whether Streams’ exclusive license is non-transferable. Pass; the response clearly provides non-transferability.

  7. States that Streams will list its exclusive rights to the series in a carve-out section of excluded assets in any asset purchase agreement

    Pass

    Evidence: The recommended structure says “APA Schedule 2.2 lists those rights as Excluded Assets,” and the APA language adds “Schedule 2.2 — Excluded Assets” for “Bare Bones Decor / Ava Kim Excluded Rights.” Assessment: The criterion asks whether Streams will list exclusive rights in a carve-out section of excluded assets in any asset purchase agreement. Pass.