Raycaster / evalsSample shared trace

OfficeQA benchmark

Agent Q&A over office documents.

Browse OfficeQA agent evaluation tasks — document Q&A over treasury and office PDFs, with dual-harness runs and public traces.

133Tasks
1Published models
95Primary passes

Published model set

This public benchmark currently includes gpt-5.4. Model counts reflect completed, shareable runs in this catalog.

Related categories

OfficeQA tasks also appear inDocument Q&AandFinancial Statement Analysis.

All tasks

Dual-harness runs on U.S. Treasury bulletin Q&A tasks.

  1. UID0001(officeqa-uid0001)1/1 · PassPublic trace →
  2. UID0003(officeqa-uid0003)1/1 · PassPublic trace →
  3. UID0004(officeqa-uid0004)1/1 · PassPublic trace →
  4. UID0005(officeqa-uid0005)1/1 · PassPublic trace →
  5. UID0007(officeqa-uid0007)1/1 · PassPublic trace →
  6. UID0009(officeqa-uid0009)1/1 · PassPublic trace →
  7. UID0010(officeqa-uid0010)1/1 · PassPublic trace →
  8. UID0012(officeqa-uid0012)1/1 · PassPublic trace →
  9. UID0013(officeqa-uid0013)0/1 · FailPublic trace →
  10. UID0015(officeqa-uid0015)1/1 · PassPublic trace →
  11. UID0017(officeqa-uid0017)1/1 · PassPublic trace →
  12. UID0018(officeqa-uid0018)1/1 · PassPublic trace →
  13. UID0019(officeqa-uid0019)1/1 · PassPublic trace →
  14. UID0022(officeqa-uid0022)1/1 · PassPublic trace →
  15. UID0025(officeqa-uid0025)1/1 · PassPublic trace →
  16. UID0027(officeqa-uid0027)1/1 · PassPublic trace →
  17. UID0028(officeqa-uid0028)1/1 · PassPublic trace →
  18. UID0029(officeqa-uid0029)1/1 · PassPublic trace →
  19. UID0030(officeqa-uid0030)0/1 · FailPublic trace →
  20. UID0031(officeqa-uid0031)1/1 · PassPublic trace →
  21. UID0032(officeqa-uid0032)1/1 · PassPublic trace →
  22. UID0035(officeqa-uid0035)1/1 · PassPublic trace →
  23. UID0036(officeqa-uid0036)0/1 · FailPublic trace →
  24. UID0037(officeqa-uid0037)1/1 · PassPublic trace →
  25. UID0039(officeqa-uid0039)0/1 · FailPublic trace →
  26. UID0042(officeqa-uid0042)1/1 · PassPublic trace →
  27. UID0044(officeqa-uid0044)1/1 · PassPublic trace →
  28. UID0049(officeqa-uid0049)1/1 · PassPublic trace →
  29. UID0050(officeqa-uid0050)1/1 · PassPublic trace →
  30. UID0053(officeqa-uid0053)1/1 · PassPublic trace →
  31. UID0055(officeqa-uid0055)1/1 · PassPublic trace →
  32. UID0056(officeqa-uid0056)1/1 · PassPublic trace →
  33. UID0057(officeqa-uid0057)1/1 · PassPublic trace →
  34. UID0058(officeqa-uid0058)0/1 · FailPublic trace →
  35. UID0059(officeqa-uid0059)1/1 · PassPublic trace →
  36. UID0062(officeqa-uid0062)0/1 · FailPublic trace →
  37. UID0065(officeqa-uid0065)0/1 · FailPublic trace →
  38. UID0068(officeqa-uid0068)1/1 · PassPublic trace →
  39. UID0069(officeqa-uid0069)1/1 · PassPublic trace →
  40. UID0071(officeqa-uid0071)1/1 · PassPublic trace →
  41. UID0073(officeqa-uid0073)1/1 · PassPublic trace →
  42. UID0077(officeqa-uid0077)1/1 · PassPublic trace →
  43. UID0083(officeqa-uid0083)1/1 · PassPublic trace →
  44. UID0084(officeqa-uid0084)1/1 · PassPublic trace →
  45. UID0085(officeqa-uid0085)1/1 · PassPublic trace →
  46. UID0086(officeqa-uid0086)1/1 · PassPublic trace →
  47. UID0093(officeqa-uid0093)0/1 · FailPublic trace →
  48. UID0094(officeqa-uid0094)1/1 · PassPublic trace →
  49. UID0096(officeqa-uid0096)1/1 · PassPublic trace →
  50. UID0097(officeqa-uid0097)1/1 · PassPublic trace →
  51. UID0100(officeqa-uid0100)0/1 · FailPublic trace →
  52. UID0101(officeqa-uid0101)1/1 · PassPublic trace →
  53. UID0102(officeqa-uid0102)0/1 · FailPublic trace →
  54. UID0103(officeqa-uid0103)1/1 · PassPublic trace →
  55. UID0108(officeqa-uid0108)1/1 · PassPublic trace →
  56. UID0109(officeqa-uid0109)1/1 · PassPublic trace →
  57. UID0110(officeqa-uid0110)0/1 · FailPublic trace →
  58. UID0111(officeqa-uid0111)1/1 · PassPublic trace →
  59. UID0112(officeqa-uid0112)1/1 · PassPublic trace →
  60. UID0113(officeqa-uid0113)0/1 · FailPublic trace →
  61. UID0114(officeqa-uid0114)0/1 · FailPublic trace →
  62. UID0117(officeqa-uid0117)1/1 · PassPublic trace →
  63. UID0118(officeqa-uid0118)1/1 · PassPublic trace →
  64. UID0120(officeqa-uid0120)1/1 · PassPublic trace →
  65. UID0121(officeqa-uid0121)1/1 · PassPublic trace →
  66. UID0122(officeqa-uid0122)1/1 · PassPublic trace →
  67. UID0123(officeqa-uid0123)1/1 · PassPublic trace →
  68. UID0124(officeqa-uid0124)1/1 · PassPublic trace →
  69. UID0130(officeqa-uid0130)1/1 · PassPublic trace →
  70. UID0133(officeqa-uid0133)0/1 · FailPublic trace →
  71. UID0134(officeqa-uid0134)1/1 · PassPublic trace →
  72. UID0135(officeqa-uid0135)0/1 · FailPublic trace →
  73. UID0136(officeqa-uid0136)1/1 · PassPublic trace →
  74. UID0140(officeqa-uid0140)0/1 · FailPublic trace →
  75. UID0147(officeqa-uid0147)0/1 · FailPublic trace →
  76. UID0148(officeqa-uid0148)1/1 · PassPublic trace →
  77. UID0149(officeqa-uid0149)1/1 · PassPublic trace →
  78. UID0150(officeqa-uid0150)0/1 · FailPublic trace →
  79. UID0154(officeqa-uid0154)0/1 · FailPublic trace →
  80. UID0155(officeqa-uid0155)0/1 · FailPublic trace →
  81. UID0161(officeqa-uid0161)1/1 · PassPublic trace →
  82. UID0165(officeqa-uid0165)0/1 · FailPublic trace →
  83. UID0168(officeqa-uid0168)1/1 · PassPublic trace →
  84. UID0169(officeqa-uid0169)1/1 · PassPublic trace →
  85. UID0170(officeqa-uid0170)1/1 · PassPublic trace →
  86. UID0172(officeqa-uid0172)0/1 · FailPublic trace →
  87. UID0173(officeqa-uid0173)1/1 · PassPublic trace →
  88. UID0174(officeqa-uid0174)1/1 · PassPublic trace →
  89. UID0175(officeqa-uid0175)0/1 · FailPublic trace →
  90. UID0177(officeqa-uid0177)0/1 · FailPublic trace →
  91. UID0179(officeqa-uid0179)1/1 · PassPublic trace →
  92. UID0180(officeqa-uid0180)1/1 · PassPublic trace →
  93. UID0182(officeqa-uid0182)1/1 · PassPublic trace →
  94. UID0183(officeqa-uid0183)1/1 · PassPublic trace →
  95. UID0184(officeqa-uid0184)1/1 · PassPublic trace →
  96. UID0187(officeqa-uid0187)1/1 · PassPublic trace →
  97. UID0188(officeqa-uid0188)0/1 · FailPublic trace →
  98. UID0190(officeqa-uid0190)1/1 · PassPublic trace →
  99. UID0193(officeqa-uid0193)0/1 · FailPublic trace →
  100. UID0194(officeqa-uid0194)1/1 · PassPublic trace →
  101. UID0195(officeqa-uid0195)1/1 · PassPublic trace →
  102. UID0196(officeqa-uid0196)1/1 · PassPublic trace →
  103. UID0201(officeqa-uid0201)0/1 · FailPublic trace →
  104. UID0203(officeqa-uid0203)0/1 · FailPublic trace →
  105. UID0204(officeqa-uid0204)1/1 · PassPublic trace →
  106. UID0205(officeqa-uid0205)1/1 · PassPublic trace →
  107. UID0207(officeqa-uid0207)1/1 · PassPublic trace →
  108. UID0211(officeqa-uid0211)1/1 · PassPublic trace →
  109. UID0212(officeqa-uid0212)1/1 · PassPublic trace →
  110. UID0213(officeqa-uid0213)1/1 · PassPublic trace →
  111. UID0214(officeqa-uid0214)0/1 · FailPublic trace →
  112. UID0215(officeqa-uid0215)1/1 · PassPublic trace →
  113. UID0216(officeqa-uid0216)0/1 · FailPublic trace →
  114. UID0218(officeqa-uid0218)1/1 · PassPublic trace →
  115. UID0219(officeqa-uid0219)1/1 · PassPublic trace →
  116. UID0220(officeqa-uid0220)0/1 · FailPublic trace →
  117. UID0221(officeqa-uid0221)1/1 · PassPublic trace →
  118. UID0222(officeqa-uid0222)1/1 · PassPublic trace →
  119. UID0223(officeqa-uid0223)0/1 · FailPublic trace →
  120. UID0224(officeqa-uid0224)1/1 · PassPublic trace →
  121. UID0225(officeqa-uid0225)1/1 · PassPublic trace →
  122. UID0226(officeqa-uid0226)1/1 · PassPublic trace →
  123. UID0227(officeqa-uid0227)0/1 · FailPublic trace →
  124. UID0228(officeqa-uid0228)1/1 · PassPublic trace →
  125. UID0230(officeqa-uid0230)0/1 · FailPublic trace →
  126. UID0233(officeqa-uid0233)1/1 · PassPublic trace →
  127. UID0237(officeqa-uid0237)1/1 · PassPublic trace →
  128. UID0238(officeqa-uid0238)0/1 · FailPublic trace →
  129. UID0240(officeqa-uid0240)0/1 · FailPublic trace →
  130. UID0243(officeqa-uid0243)1/1 · PassPublic trace →
  131. UID0244(officeqa-uid0244)1/1 · PassPublic trace →
  132. UID0245(officeqa-uid0245)0/1 · FailPublic trace →
  133. UID0246(officeqa-uid0246)0/1 · FailPublic trace →