APEX-Agents · Management Consulting
Task o88f1452
APEX-Agents task Task o88f1452 in AI Agents for SaaS Due Diligence. Compare dual-harness agent runs across models — rubric criteria, scores, and public traces.
Task prompt
What the agent was asked to do
We need to redo the analysis of the new survey response dataset. Can you re-calculate the standard deviation for Brightpath Software's efficiency dataset? Using both the recalculated average efficiency score for Brightpath Software (by all Brightpath users) and the average efficiency score for Brightpath Software by only Brightpath users who ranked AI capabilities as top priority, please also calculate the fraction of a standard deviation that the two scores differ by. Round all final answers to four decimal places. State the output directly to me here as a reply.
Published trajectories
Agent runs on this task
Curated dual-harness runs (parsed + original sandbox). Best scored run per model.
| Model | Harness | Score | Result | Links |
|---|---|---|---|---|
| GPT-5.5showcase | dual | 0/2 | Fail | Share pagePublic trace |
| Gemini 3.1 Pro | dual | 0/2 | Fail | Share pagePublic trace |
| GPT-5.4 | dual | 0/2 | Fail | Share pagePublic trace |
| GPT-5.4 mini | dual | 0/2 | Fail | Share pagePublic trace |
| GPT-5.4 nano | dual | 0/2 | Fail | Share pagePublic trace |
Grading rubric
Criteria and grader verdict (showcase run)
States the standard deviation for efficiency scores of Brightpath Software is 1.3836
FailEvidence: <TEXT_RESPONSE> states “Brightpath Software efficiency standard deviation: 1.2802.” Assessment: The criterion requires stating the standard deviation is 1.3836; the response gives 1.2802, so this fails.
States the fraction of a standard deviation that the two scores differ is 0.1714
FailEvidence: <TEXT_RESPONSE> states “Difference as fraction of standard deviation: 0.1495 lower for the AI-priority subgroup.” Assessment: The criterion requires stating the fraction of a standard deviation is 0.1714; the response gives 0.1495, so this fails.