APEX-Agents · Law
Task 5vd1ebb0
APEX-Agents task Task 5vd1ebb0 in AI Agents for SEC Disclosure Analysis. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
We've received the attached warranty claims for some of our products. Please review them. Then, edit the existing existing "product purchases" spreadsheet to show the maximum refund amount a customer could receive for each product purchased.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 5/5 | Pass | Share pagePublic trace |
| fireworks models Kimi K2 | dual | 5/5 | Pass | Share pagePublic trace |
| Gemini 3 Flash | dual | 5/5 | Pass | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 5/5 | Pass | Share pagePublic trace |
| GPT-5.4 | dual | 5/5 | Pass | Share pagePublic trace |
| GPT-5.4 mini | dual | 5/5 | Pass | Share pagePublic trace |
| GPT-5.4 nano | dual | 5/5 | Pass | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States Falcon Prevent's Max Refund Amount is $0.00
PassEvidence: TEXT_RESPONSE lists verified result: “Falcon Prevent: $0.00.” Assessment: Criterion asks whether it states Falcon Prevent's Max Refund Amount is $0.00; pass because the response explicitly states that amount.
States Falcon Query's Max Refund Amount is $16,888.89
PassEvidence: TEXT_RESPONSE lists verified result: “Falcon Query: $16,888.89.” Assessment: Criterion asks whether it states Falcon Query's Max Refund Amount is $16,888.89; pass because the response explicitly states that amount.
States Falcon X Elite's Max Refund Amount is $0.00
PassEvidence: TEXT_RESPONSE lists verified result: “Falcon X Elite: $0.00.” Assessment: Criterion asks whether it states Falcon X Elite's Max Refund Amount is $0.00; pass because the response explicitly states that amount.
States Falcon Cloud Workload Protection's Max Refund Amount is $0.00
PassEvidence: TEXT_RESPONSE lists verified result: “Falcon Cloud Workload Protection: $0.00.” Assessment: Criterion asks whether it states Falcon Cloud Workload Protection's Max Refund Amount is $0.00; pass because the response explicitly states that amount.
States Falcon Container Senor's Max Refund Amount is $0.00
PassEvidence: TEXT_RESPONSE lists verified result: “Faclon Container Senor: $0.00.” Assessment: Criterion asks whether it states Falcon Container Senor's Max Refund Amount is $0.00; pass because despite the minor typo “Faclon,” the response clearly identifies the intended product and states $0.00.