Raycaster / evalsAll benchmarks

Cross-benchmark category

Digital Transformation

Compare AI agent runs for Digital Transformation: prompts, scores, grades, and public traces. This page showcases APEX-Agents tasks that test whether AI agents can analyze digital transformation opportunities, forecasting accuracy, plant downtime, and modernization roadmaps.

Operations consulting AIForecasting accuracy, plant downtime, modernization roadmaps
72Tasks
4Primary
1Benchmarks

Published model evidence

A quick view of shareable runs in this category. Open task and trace pages for the prompt, response, files, and grading details.

Gemini 3 Flash48% pass

12/25 runs · avg score 68%

gpt-5.4-mini36% pass

26/72 runs · avg score 56%

GPT-5.436% pass

25/70 runs · avg score 55%

gpt-5.4-nano32% pass

23/72 runs · avg score 52%

Gemini 3.1 Pro32% pass

23/72 runs · avg score 51%

gpt-5.532% pass

23/72 runs · avg score 49%

Primary tasks

4 tasks with this category as their main focus.

  1. Task_World130_CamilleMoingeon_4APEX-Agents(task_2516425094194379af7c5b6d5180a608)0/5 · Fail

    Can you look at the Frito-Lay case study and apply their downtime reduction to HarFeast Good Group's number in the baseline file? I want to estimate what the improvement would look like for us (rounde…

  2. World130_Al-Zhoheir_Hajim_1APEX-Agents(task_3d4d03776d704758bb1bb9931e301b1c)0/9 · Fail

    1. Give me the total labor cost for each plant location. 2. Give me the efficiency gains for each plant location. West North Central division plant locations only have a 10% annual efficiency gain f…

  3. World130_Al-Zhoheir_Hajim_04APEX-Agents(task_632105b2a93445ac8c4e16d6b17b836a)0/5 · Fail

    Calculate the Adjusted Cost of Instability for each site, defined as Abnormal scrap cost/(Actual Scrap %−Normal Scrap %) = adjusted cost of instability. The target scrap rate of HarFeast is the minimu…

  4. world130_HO_05APEX-Agents(task_dba0d3217ada47eb93028eed8e14d69d)5/5 · Pass

    Use the v1 version of the survey responses to identify the number of respondents who received any kind of training on digital tools. Of those respondents, return the percentage of respondents for each…

Related tasks

68 tasks that also exercise this work as part of a broader assignment.

  1. Change in Competitor SG&A Spend by Category_Task02_SCAPEX-Agents(task_449e96fd1efb4a7f9ecbb65e82fbd6c1)0/7 · Fail

    Analyze the SG&A files to compare the CAGR trends of competitors versus Impact’s US business across 2020–22 and 2022–24. For each SG&A Category, determine the percentage point change in CAGR between t…

  2. Law 433_AC_02APEX-Agents(task_1d7f4e0be1274b069ad86be4ed813c46)4/10 · Fail

    Please review Grove’s 2023 and 2025 operations manuals. Let me know if there are any changes in the 2025 manual that may present issues for Grove’s regulatory compliance. For any regulatory issues ide…

  3. Law_World_423_DM_03APEX-Agents(task_39431bef81164cd2843e9369b8f7fdc5)2/3 · Fail

    Northstar is evaluating a situation where Bluequill had utilized the EU personal data received by its analytics module from Northstar, and utilized it for the purposes of sending out marketing emails …

  4. Law_World_434_sg_01APEX-Agents(task_981f4f72a55c4e7598b6f23f3e576e54)0/10 · Fail

    HarborView requires all customer documents to be sent to their offices in the Cayman Islands headquarters, including information of Hong Kong customers. We’ve also already sent them documents as part …

  5. Law433_mk_02APEX-Agents(task_3742720383624ce58ef4d751f8836392)5/7 · Fail

    Can you edit the existing Grove Residency Agreement for Thomas Whitaker to make a revised residency agreement template that we can use for new residents. Please also update/add to the last sentence o…

  6. Marketing Case Studies_Task03_SCAPEX-Agents(task_f525769ab6a748e6855e03c95e4b4bd7)6/8 · Fail

    Using the research found on pharma marketing and articles around competitor shifts in SG&A spend, identify key trends that Impact can apply to SG&A spend that could reduce overall costs. In particular…

  7. Task Seed 2_Adjust TargetAPEX-Agents(task_75bca3bb30dc4d9db71d253063c96e97)5/5 · Pass

    Can you take a fresh pass at our cost savings targets? Start by resetting the Manufacturing and Supply Chain savings goals based on the best-in-class cost as a percentage of 2024 revenue benchmarks. …

  8. Task_World130_CamilleMoingeon_2APEX-Agents(task_1137ecf56e014b749a7af76a40057560)8/8 · Pass

    I'm trying to get a sense of which HarFeast employees are most ready for the digital training rollout. Can you pull the workforce survey data and identify all employees who are above their role type's…

  9. Task_World130_CamilleMoingeon_3APEX-Agents(task_9363f85adf864ccaa8dbff72d38225cb)2/6 · Fail

    Using HarFeast's baseline diagnostic dataset, assess the impact of predictive maintenance on HarFeast's scrap rate. We will pilot predictive maintenance only on equipment a) whose scheduled hours per …

  10. TaskWorld130_CamilleMoingeon_5APEX-Agents(task_a0d3895b47a04ae0bba65caa932f6fac)2/2 · Pass

    Using HarFeast's equipment data by location and quality losses dataset, we will consider all canned vegetables assets with a scrap rate > 5% and with unplanned downtime hours above the plant median fo…

  11. World 112-1 | Task 2 - Depreciation Reduction for plants (JS)APEX-Agents(task_6b27cc3ab9da428eaa9daa9f5100882b)6/7 · Fail

    Calculate 2024 manufacturing overhead (MOH) costs in $ for each Impact plant. Use the midpoint value of the benchmark ranges for the specific product type manufactured at the plant from the attached f…

  12. World 127_HLV_Task 03APEX-Agents(task_626333c69ffb46d0ad041a2dd6916fdf)4/6 · Fail

    We want to understand the most important investment areas for pureplay EV respondents. The previous file contained incomplete information, so please use the newly attached updated file. Identify the…

  13. World 130_JB_Task xnxu78f8APEX-Agents(task_8ad37c5248434c99a23c0b6d6524e9d2)0/8 · Fail

    To implement the required roadmap for our recommendations, we need to identify what roles and plants are most and least willing to go through a digital transformation. We will start with a small tra…

  14. World 131_MK_Task 1APEX-Agents(task_ec97a505112645f9b266df1954f2738d)0/2 · Fail

    Assuming Eurogrid goes through with the labor reallocation efforts described in the operational efficiency analysis, calculate both the % of total staff that is a manager and the average span of contr…

  15. World 131_MK_Task 2APEX-Agents(task_4b3c2dfc4d164a25831e8787397766c3)4/4 · Pass

    Identify the region with the highest average asset-level Total Score (defined as the sum of Criticality Score, Renewable Impact, and Risk Score), and the country within that region that has the highes…

  16. World 131_MK_Task 3APEX-Agents(task_ee6e8c8f7075427ba5381c453a14c71c)0/5 · Fail

    Please use EuroGrid's headcount per department and the attached benchmarks to calculate the estimated total cost of each of the departments' headcount. Round all final amounts to full USD. Provide yo…

  17. World 131_MK_Task 4APEX-Agents(task_1f6c5b8814fa40bdb25ae11bbd48b6ea)0/4 · Fail

    Please calculate how much Germany's and Netherlands' renewables pipelines (will be only 95%) will cover out of their total yearly loads in 2027 and 2028 in % terms. You can use their average historica…

  18. World_131_JR_1APEX-Agents(task_1dba146a5da645b3a27ea3212c0bffc4)4/6 · Fail

    Can you please evaluate the outage causes that affect each country the most, in terms of total outage duration? Categorize hazards relating to flooding or storms as the "Weather - Storm" cause and tho…

  19. World112-1_Task02_NAAPEX-Agents(task_d1185507e9d24d74aaff30450c2698ed)2/2 · Pass

    Calculate the difference in dollars between Impact's actual 2024 US COGS and the hypothetical US COGS if they reached the straight average of COGS as a percentage of revenue given in Exhibit 2 of the …

  20. World112-1_Task02_TBAPEX-Agents(task_87dd4d824ff84835b0ed505d2499c15a)0/3 · Fail

    I want to inform client discussions around what action to take regarding sourcing of materials and the speculated tariffs. Can you summarize averages of the following 3 data points pertaining to tarif…

  21. World112-1_Task03_NAAPEX-Agents(task_0c527e11cc1c436696561a15eead68b2)10/10 · Pass

    Analyze the attached report and the continuous manufacturing report to state the FDA-encouraged manufacturing style that differs from traditional methods which is mentioned in both reports. Using the …

  22. World112-1_Task04_TBAPEX-Agents(task_d7893ddf035a4abca2990717ad4d7428)4/4 · Pass

    Given the US imposed tariffs on the import of pharmaceutical materials from Germany, India, China at the rates 10%, 15%, 20% respectively, determine the impact of tariffs as a percent increase in pred…

  23. World112-1_Task05_NAAPEX-Agents(task_b63eb63a9b964203bb7033ed3682f06e)2/3 · Fail

    We need to perform a payback period analysis for the investment to upgrade Lorexa's equipment to adopt continuous manufacturing. The methodology and results of the previous cost-benefit analysis of up…

  24. World112-1_Task05_SCAPEX-Agents(task_7c7edc1b3dea41ec89f00eb4581df3fa)3/3 · Pass

    Review industry reports to determine the range of industry-wide reductions in force (RIFs). Using Impact’s revised SG&A breakdown, calculate the cost savings Impact would achieve by reducing its sales…

  25. World112-1_TK_02APEX-Agents(task_112defba78604abcb27f4afb573d8d05)0/1 · Fail

    Can you do some benchmarking analysis of Impact with its 6 peers for 2024 US operations? Let me know the difference between the lowest average monthly revenue per batch passed and the highest average …

  26. World112-1_TK_03APEX-Agents(task_45f1d761bb464e18993968273a9a9040)1/1 · Pass

    Can you please do some what-if analysis of Impact with its 6 peers for 2024 US operations and let me know the following: If the Impact site with the lowest average monthly revenue per head operated a…

  27. World112-1_TK_04APEX-Agents(task_854fefdbea5740a3a238c5930c2721e7)2/2 · Pass

    Can you please calculate the z score of US 2024 Average Monthly Revenue per Head, for all of Impact’s sites? Use a distribution of US 2024 Average Monthly Revenue per Head per site for all the sites i…

  28. World125_Task01_NAAPEX-Agents(task_dbf1b00ea3cc4a59af744f370f7a1acf)4/4 · Pass

    Using the 2024 annual report, identify whether Impact Therapeutics sells primarily branded drugs. A company sells primarily branded drugs if the majority of drugs have launch dates within the last 7 y…

  29. World125_Task01_TBAPEX-Agents(task_2a441de423284de492c369432970eef5)3/3 · Pass

    Using the latest materials cost reduction analysis, assume Impact reduces its 2025 materials cost with current supplier relationships by 20%. Benchmark Impact's resulting 2025 materials cost as a perc…

  30. World130_Al-Zhoheir_Hajim_2APEX-Agents(task_908a5e7cd4fe4994a47cb2c8c0fbf53f)0/8 · Fail

    1. What is the digital lever that Sarah Jenkins, David Chen, and Mike Russo agree will deliver the fastest and biggest boost to HarFeast's Gross Margin? 2. Assuming HarFeast adopts the chosen digi…

  31. World130_Al-Zhoheir_Hajim_3APEX-Agents(task_88469a82ab624b0084d2381fbaeb38d1)2/6 · Fail

    Can you calculate the total labor variance in hours (favorable should be positive) and dollars for the Illinois and Wisconsin plants? A positive variance should mean that Total Actual Hours are less t…

  32. World130_Al-Zhoheir_Hajim_5APEX-Agents(task_5df3da6f8f5c459484670772c82f6941)3/10 · Fail

    Identify the top five technology investments from the Aptean report with the largest positive difference in percentage revenue growth between users and non-users. Include only investments that the rep…

  33. world130_HO_02APEX-Agents(task_cfca811300e14fe8ae12e0f97f6c6c78)9/10 · Fail

    Analyze the operational efficiency at HarFeast and assess how many inefficient employee hours each plant is recording on average. Which plants have the most efficient operations and the least efficien…

  34. world130_HO_04APEX-Agents(task_619f045f82f340a3871847bf1e7e2b40)0/9 · Fail

    I want to quantify the average annual productivity loss at a cost level for each employee in each primary role based on the sum of average hours spent doing manual entry, searching data, and fixing er…

  35. world130_HO_08APEX-Agents(task_1e19b26b91804fe386ef911e347df0f8)0/2 · Fail

    The client sent us employee wage data (attached), so we need to update our assumptions in the financial analysis section of the survey analysis report to display the updated annual productivity loss f…

  36. World131_acd_task09APEX-Agents(task_c0476484bd64414f87f46f1868cde2f1)0/3 · Fail

    EuroGrid wants to understand whether the root cause of its asset failures can be explained by age, load, and/or frequency of weather events. Identify the 3 manufacturers with the highest total failur…

  37. World131_acd_task10APEX-Agents(task_5b0050f41fc9495fa8c34b69f1184418)9/9 · Pass

    Take a look at our workforce distribution in the country where we've spent the most on OPEX from 2020-24. Knowing we need 2 line technicians per transmission line, 1 substation technician for each sub…

  38. World131_acd_task11APEX-Agents(task_f32cf6dbffac45eea7454b4f3a62eaf9)0/2 · Fail

    Identify the Phase 1 Assets from the 10 Year Roadmap, assuming that SAIFI / SAIDI hotspots can be defined as assets having SAIFI > 1.0 and SAIDI > 60. Ignore the key criteria for substations and the n…

  39. World131_acd_task12APEX-Agents(task_f9f2907c268c44f686496a4934f1fc15)2/3 · Fail

    Looking only at projects in the Connection Queue that have a status of "Approved" or "Connected", calculate the percentage of the Total Forecasted Demand for the years 2026, 2027, and 2028 that could …

  40. World131_DV_02APEX-Agents(task_d55fe268d7f64a74aacfce4fc374ea96)6/6 · Pass

    Can you calculate the annual EU implied revenue for each Eastern European TSO? Use the midpoint of their implied market share ranges and 40 billion euros as the total market size. Using the implied r…

  41. World131_DV_03APEX-Agents(task_a179d38b095f46eba5eff7baf8f7fd87)0/1 · Fail

    Calculate the NPV from the 12-year cash flow on renewable enablement benefits, considering the following assumptions: - The steady-state annual benefits from renewable enablement mentioned in the busi…

  42. World131_DV_05APEX-Agents(task_9d99126a403e41838e1473d33884ed2f)1/2 · Fail

    Can you state the total simple average of the average implementation cost values, across the various technologies? Use the attached implementation cost deck. Also, state how many technologies have a t…

  43. World131_DV_06APEX-Agents(task_806c8178532944a78b78d565e4bc0313)5/5 · Pass

    What is the net aggregate annual benefit (i.e., total annual savings minus total annual opex) of all of the use cases in the digital use case sizing analysis? According to the new transmission technol…

  44. World131_IB_05APEX-Agents(task_699ea5ca3b0243a5852ad33e33043f12)5/6 · Fail

    Find out the ratio between Curtailment_GWh and Redispatch_GWh and for the lowest average ratio of Country-Region pair, report the average Avoided Curtailment (MWh) and causes of curtailment. Represen…

  45. World131_IB_Task 12APEX-Agents(task_0ea0001d6cb34cc6abf9bf6dc4e8e30b)0/2 · Fail

    Using the Digital Twin Input and Additional bus datasets, identify the Bus IDs associated with renewable energy generation. For each of these Bus IDs, calculate the average (in GW) of their three high…

  46. World131_MD_01APEX-Agents(task_55a201bd877a42eeb0e1868fb6a84230)4/4 · Pass

    Tell me whether or not the asset type that has the highest average adjusted failure probability per outage is also responsible for the highest average Value of Lost Load (VOLL) per asset. VOLL is defi…

  47. World131_MD_03APEX-Agents(task_7acc98cded8b49de972ee79b0461107e)3/3 · Pass

    Investigate whether EuroGrid should consider increasing staffing. Determine if the number of working people per impacted asset is correlated with the expected economic impact of unforeseen downtime in…

  48. World221_TR_07APEX-Agents(task_9fad1b1520eb46778e34e950c41be109)1/4 · Fail

    What if BBDC change target and achieve a partial merger with FIDUS Investment Corporation (FDUS) instead of TriplePoint Venture Growth BDC Corp. (TPVG)? Use the FDUS SEC filings as of Q3 2025 and Q4 …

  49. World223_SMN_04APEX-Agents(task_973f357750e2416c90f45be924d87d7f)2/4 · Fail

    Using the 3Q 2025 3M 10Q and the accretion dilution model, calculate the following deliverables. Present all values in millions ($mm), number of shares in millions (mm), and round to whole numbers. Fo…

  50. World419_AH_02APEX-Agents(task_a28445db856341ecb5c47d86e488b3d4)0/1 · Fail

    The containment and remediation efforts for the crude oil spill were successful, and our total liability was $1.5 million, which we claimed under our insurance. 6 months after the spill, we were con…

  51. World421_OO_02APEX-Agents(task_fe573b8ce38d4a9f9642fbe7b8f11358)5/10 · Fail

    Our client, SLL, needs to understand whether their new customer marketing initiative is legal and what risks are likely. In this initiative, SLL is creating marketing texts and sending them to custom…

  52. World431_amk_01APEX-Agents(task_c7fc2009b5b849b4b769f0eafe64f461)0/2 · Fail

    MGR Real Estate Inc. (the "Lessor") and "AI Automation Group, LLC" (the "Lessee") entered into the final lease agreement on December 5, 2025 for 2020 Main Street, Irvine, CA (the "Premises"). On De…

  53. World431_amk_02APEX-Agents(task_76585a8b9eb3423895795c8715f924c7)2/5 · Fail

    MGR Real Estate Inc. (the "Lessor") and "AI Automation Group, LLC" (the "Lessee") entered into the final lease agreement on December 5, 2025 (the “Lease”) for 2020 Main Street, Irvine, CA (the "Premis…

  54. World431_AVK_01APEX-Agents(task_611b910ce2514004993effe836ef3d74)2/6 · Fail

    On December 10, 2025 prior to closing on the lease with MGR Real Estate, Inc. ("MGR"), Grace Joblin passed away. The draft consent was not signed by the members of AI Automation Group, LLC ("AIAG") au…

  55. World431_AVK_02APEX-Agents(task_829dd434f17b415c951bd7b7f543d35e)1/1 · Pass

    In addition to the 15,000 sq ft rented to AI Automation Group, LLC (“AIAG”), MGR Real Estate, Inc. ("MGR") rents a total of 157,235 sq ft. of space to the other tenants in the building. Using the FINA…

  56. World431_AVK_03APEX-Agents(task_ccd303088b344ea8b81c13dc04309142)0/2 · Fail

    Review the attached documents to determine the amounts to be distributed from AI Automation Group, LLC ("AIAG") to Shohei Yamamoto and Janet Swift. Get me your reply right back here.

  57. World431_DM_01APEX-Agents(task_286b3c8f3efc44299eaa62dd70054bad)3/3 · Pass

    MGR Real Estate Inc. ("MGR") and "AI Automation Group, LLC" ("AIAG") entered into the final lease agreement on December 12, 2025 for 2020 Main Street, Irvine, CA (the "Building"). On December 15, 2025…

  58. World431_DM_02APEX-Agents(task_f31aa2ecb08b4843ab62cba393599947)0/3 · Fail

    MGR Real Estate Inc. ("MGR") and AI Automation Group, LLC ("AIAG") entered into the final lease agreement on December 12, 2025 for 2020 Main Street, Irvine, CA (the "Building"). A couple years later, …

  59. World431_DM_03APEX-Agents(task_65f3283e76254376bb94f65cc9f14897)4/5 · Fail

    Given the information below, is AIAG correct? Print your reply to me here. Give me a Yes/No, and a 1-2 sentence explanation. MGR Real Estate Inc. ("MGR") and "AI Automation Group, LLC" ("AIAG") enter…

  60. World431_DM_04APEX-Agents(task_cba85c18cb7c456da8497c1018c75b16)0/3 · Fail

    MGR Real Estate Inc. ("MGR") and "AI Automation Group, LLC" ("AIAG") entered into a letter of intent (“LOI”) on December 2, 2025 for 2020 Main Street, Irvine, CA (the "Building"). The letter included …

  61. World431_DM_05APEX-Agents(task_18263dbcc46a4fabaf9b7b6f243aa748)3/3 · Pass

    MGR Real Estate Inc. ("MGR") and "AI Automation Group, LLC" ("AIAG") entered into the final lease agreement on December 12, 2025 for 2020 Main Street, Irvine, CA (the "Building"). The Parties mutually…

  62. World431_DM_06APEX-Agents(task_70328d2e54c04d4cb829435037f3c548)0/4 · Fail

    MGR Real Estate Inc. ("MGR") and "AI Automation Group, LLC" ("AIAG") entered into the final lease agreement (the "Lease") on December 12, 2025 for 2020 Main Street, Irvine, CA (the "Building"). On Dec…

  63. World431_JS_01APEX-Agents(task_7a420a91d97b4bad8c88e994f8df7768)2/2 · Pass

    2020 Main Street, Irvine (the “Premises”) needs a new HVAC System in order to accommodate INTELLIGENCE INC.'s server rooms. INTELLIGENCE INC. is a tenant at the Premises. The HVAC system will replac…

  64. World431_JS_02APEX-Agents(task_0d1c4041b5e945bc867fa627e302b503)2/2 · Pass

    MGR Real Estate Inc. (The "Lessor") and "AI Automation Group, LLC" (the "Lessee") entered into the final lease agreement on December 5, 2025 for 2020 Main Street, Irvine, CA (the "Premises"). On Decem…

  65. World431_JS_03APEX-Agents(task_d7d932117f2141eba5b8b142fa862a6b)0/2 · Fail

    AI AUTOMATION GROUP, LLC (The "Lessee") is leasing a portion of 2020 MAIN STREET, IRVINE, CA (The "Premises") from MGR REAL ESTATE INC. (The "Lessor"). As part of its business, the Lessee makes intell…

  66. World431_JS_04APEX-Agents(task_78b12b856287427d86e75dadc6ffa465)0/5 · Fail

    MGR REAL ESTATE INC. (The "Lessor") is leasing 2020 MAIN STREET, IRVINE, CA (The "Premises") to AI AUTOMATION GROUP, LLC (The "Lessee"). The Lessee asked their lawyers to identify provisions that are …

  67. World431_JS_05APEX-Agents(task_35d3f7c310b7451fa7400ebc3a0ff372)0/2 · Fail

    MGR REAL ESTATE INC. (The "Lessor") has leased a portion of 2020 MAIN STREET, IRVINE, CA (The "Premises") to AI AUTOMATION GROUP, LLC (The "Lessee"). The lease started on December 5, 2025. The Lesse…

  68. World434_AH_04APEX-Agents(task_5ee798d029884a06bb17787179206c7a)1/4 · Fail

    We had an intern prepare a list of what needed to be done after closing and they made a mess of it. We've already established the holding company, reviewed insurance policies, and completed a complia…