Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More
In the race to harness the transformative power of generative AI, companies are betting big – but are they flying blind? As billions pour into gen AI initiatives, a stark reality emerges: enthusiasm outpaces understanding. A recent KPMG survey reveals a staggering 78% of C-suite leaders are confident in gen AI’s ROI. However, confidence alone is hardly an investment thesis. Most companies are still struggling with what gen AI can even do, much less being able to quantify it.
“There’s a profound disconnect between gen AI’s potential and our ability to measure it,” warns Matt Wallace, CTO of Kamiwaza, a startup building generative AI platforms for enterprises. “We’re seeing companies achieve incredible results, but struggling to quantify them. It’s like we’ve invented teleportation, but we’re still measuring its value in miles per gallon.”
This disconnect is not merely an academic concern. It’s a critical challenge for leaders tasked with justifying large gen AI investments to their boards. Yet, the unique nature of this technology can often defy conventional measurement approaches.
Why measuring gen AI’s impact is so challenging
Unlike traditional IT investments with predictable returns, gen AI’s impact often unfolds over months or years. This delayed realization of benefits can make it difficult to justify AI investments in the short term, even when the long-term potential is significant.
At the heart of the problem lies a glaring absence of standardization. “It’s like we’re trying to measure distance in a world where everyone uses different units,” explains Wallace. “One company’s “productivity boost”’ might be another’s “cost savings”. This lack of universally accepted metrics for measuring AI ROI makes it difficult to benchmark performance or draw meaningful comparisons across industries or even within organizations.
Compounding this issue is the complexity of attribution. In today’s interconnected business environments, isolating the impact of AI from other factors – market fluctuations, concurrent tech upgrades, or even changes in workforce dynamics – is akin to untangling a Gordian knot. “When you implement gen AI, you’re not just adding a tool, you’re often transforming entire processes,” explains Wallace.
Further, some of the most significant benefits of gen AI resist traditional quantification. Improved decision-making, enhanced customer experiences, and accelerated innovation don’t always translate neatly into dollars and cents. These indirect and intangible benefits, while potentially transformative, are notoriously difficult to capture in conventional ROI calculations.
The pressure to demonstrate ROI on gen AI investments continues to mount. As Wallace puts it, “We’re not just measuring returns anymore. We’re redefining what ‘return’ means in the age of AI.” This shift is forcing technical leaders to rethink not just how they measure AI’s impact, but how they conceptualize value creation in the digital age.
The question then becomes not just how to measure ROI, but how to develop a new framework for understanding and quantifying the multifaceted impact of AI on business operations, innovation, and competitive positioning. The answer to this question may well redefine not just how we value AI, but how we understand business value itself in the age of artificial intelligence.
Summary table: Challenges in measuring gen AI ROI
| Challenge | Description | Impact on Measurement |
| Lack of standardized metrics | No universally accepted metrics exist for measuring gen AI ROI, making comparisons across industries and organizations difficult. | Limits cross-industry benchmarking and internal consistency. |
| Complexity of attribution | Difficult to isolate gen AI’s contribution from other influencing factors such as market conditions or other technological changes. | Introduces ambiguity in identifying gen AI’s true impact. |
| Indirect and intangible benefits | Many gen AI benefits, like improved decision-making or enhanced customer experience, are hard to quantify directly in financial terms. | Complicates the creation of financial justifications for gen AI. |
| Time lag in realizing benefits | Full benefits of gen AI might take time to materialize, requiring long-term evaluation periods. | Delays meaningful ROI assessments. |
| Data quality and availability issues | Accurate ROI analysis requires comprehensive and high-quality data, which many organizations struggle to gather and maintain. | Undermines reliability of ROI measurements. |
| Rapidly evolving technology | Gen AI advances rapidly, making benchmarks and measurement approaches outdated quickly. | Increases the need for continuous recalibration. |
| Varying implementation scales | ROI can differ significantly between pilot tests and full implementations, making it difficult to extrapolate results. | Creates inconsistencies when projecting future returns. |
| Integration complexities | Gen AI implementations often require significant changes to processes and systems, making it challenging to isolate the specific impact of gen AI. | Obscures direct cause-and-effect analysis. |
Key performance indicators for gen AI ROI
To better navigate these challenges, organizations need a blend of quantitative and qualitative metrics that reflect both the direct and indirect impact of gen AI initiatives. “Traditional KPIs won’t cut it,” says Wallace. “You have to look beyond the obvious numbers.”
Among the essential KPIs for gen AI are productivity gains, cost savings and time reductions—metrics that provide tangible evidence to satisfy boardrooms. Yet, focusing only on these metrics can obscure the real value gen AI creates. For example, reduced error rates may not show immediate financial returns, but they prevent future losses, while higher customer satisfaction signals long-term brand loyalty.
The true value of gen AI goes beyond numbers, and companies must balance financial metrics with qualitative assessments. Improved decision-making, accelerated innovation and enhanced customer experiences often play a crucial role in determining the success of gen AI initiatives—yet these benefits don’t easily fit into traditional ROI models.
Some companies are also tracking a more nuanced metric: Return on Data. This measures how effectively gen AI converts existing data into actionable insights. “Companies sit on massive amounts of data,” Wallace notes. “The ability to turn that data into value is often where gen AI makes the biggest impact.”
A balanced scorecard approach helps address this gap by giving equal weight to both financial and non-financial metrics. In cases where direct measurement isn’t possible, companies can develop proxy metrics—for instance, using employee engagement as an indicator of improved processes. The key is alignment: every metric, whether quantitative or qualitative, must tie back to the company’s strategic objectives.
“This isn’t just about tracking dollars,” Wallace adds. “It’s about understanding how gen AI drives value in ways that matter to the business.” Regular feedback from stakeholders ensures that ROI frameworks reflect the realities of day-to-day operations. As gen AI initiatives mature, organizations must remain flexible, fine-tuning their assessments over time. “Gen AI isn’t static,” Wallace notes. “Neither should the way we measure its value.”
Industry-specific approaches to gen AI ROI
Not all industries leverage gen AI in the same way, and this variation means that ROI measurement strategies must be tailored accordingly. Insights from the KPMG survey highlight key differences across sectors:
- Healthcare and Life Sciences: 57% of respondents reported document assessment tools as a critical value driver.
- Financial Services: 30% identified customer service chatbots as one of the most impactful applications.
- Industrial Markets: 64% highlighted inventory management as a primary use case.
- Technology, Media, and Telecommunications: 43% saw workflow automation as a key driver of value.
- Consumer and Retail: 19% emphasized the importance of customer-facing chatbots in their AI strategy.
These findings underscore the importance of building ROI frameworks that align with the specific use cases and strategic goals of each industry. “You can’t force-fit gen AI into existing measurement models,” Wallace warns. “It’s about meeting the use case where it lives, not where you want it to be.”
Example: How Drip Capital measured gen AI ROI
Drip Capital, a fintech startup specializing in cross-border trade finance, provides a concrete example of how businesses can apply a structured approach to measuring the ROI of gen AI initiatives.
The company’s use of large language models (LLMs) has led to a 70% productivity increase by automating document processing and enhancing risk assessment. Rather than building proprietary models, Drip Capital focused on optimizing existing AI tools through prompt engineering and a hybrid human-in-the-loop system to address challenges like hallucinations.
Their journey aligns closely with key elements of the 12-step framework, offering insights into the practicalities of quantifying AI’s impact.
To assess the success of their gen AI implementation, Drip Capital uses both quantitative metrics and qualitative assessments:
1. Productivity Gains
How They Can Measure It:
- Baseline comparison: Number of trade documents processed per day before gen AI deployment vs. after.
- Efficiency ratio: Total documents processed per employee to validate scalability.
Example Calculation:
- Before gen AI: 300 documents/day with 10 employees
- After gen AI: 500 documents/day with the same staff
- Productivity Increase: (500 – 300) / 300 = 67%
They also monitor operational capacity increases, ensuring no additional staffing is required to handle larger volumes.
2. Cost Savings
How They Can Measure It:
- Labor cost savings: Reduced need for manual document handling.
- Transaction approval efficiency: Faster processing reduces delays, improving cash flow.
- Infrastructure costs: Monitoring whether AI implementation reduces reliance on outsourced services or third-party vendors.
Example Calculation:
- Manual labor costs saved: $50,000 annually from reduced staff hours
- Faster approvals: Transactions approved 1 day faster, reducing working capital requirements
- Overall Savings: $50,000 (labor) + $10,000 (interest from faster payments) = $60,000/year
3. Error Reduction Rate
How They Can Measure It:
- Error rate comparison: Number of errors per 1,000 processed documents before and after gen AI.
- Key field accuracy: Focus on high-risk data points, such as payment terms or credit amounts, where mistakes can be costly.
Example Calculation:
- Before gen AI: 15 errors per 1,000 documents
- After gen AI: 3 errors per 1,000 documents
- Error Reduction Rate: (15 – 3) / 15 = 80%
This metric ensures accuracy improvements while validating the effectiveness of their human-in-the-loop verification layer.
4. Time Savings
How They Can Measure It:
- Baseline comparison: Time required to process one trade transaction before and after AI.
- Throughput improvement: Total documents processed per hour, ensuring faster service delivery.
Example Calculation:
- Before gen AI: 3 days to process a transaction
- After gen AI: 6 hours to process the same transaction
- Time Saved: (3 days – 6 hours) / 3 days = 92% reduction
This metric reflects both increased throughput and improved customer satisfaction.
5. Risk Assessment Impact
How They Measure It:
- Predictive accuracy: Compare AI-driven credit risk predictions with historical performance data.
- Faster decision-making: Measure the time saved in generating risk reports and liquidity projections.
Example Calculation:
- Before gen AI: Risk analysis took 3 business days
- After gen AI: Completed in 6 hours
- Time Savings: (3 days – 6 hours) / 3 days = 92% reduction
They also track the number of accurately flagged high-risk accounts as a key measure of gen AI’s predictive power.
6. Customer Satisfaction Scores
How They Measure It:
- Net Promoter Score (NPS): Track improvements in customer loyalty and satisfaction post-gen AI implementation.
- Survey results: Gather feedback from clients regarding faster approvals and accuracy.
Example Calculation:
- Pre-AI NPS: 50
- Post-AI NPS: 70
- NPS Improvement: (70 – 50) / 50 = 40% increase
Higher scores directly correlate with gen AI-driven improvements in service delivery.
7. Return on Data
How They Measure It:
- Data utilization rate: Percentage of available historical data used effectively in AI models.
- Insight-to-decision rate: Measure how often AI-generated insights lead to actionable business decisions.
Example Calculation:
- Before gen AI: 60% of historical data leveraged for insights
- After gen AI: 90% utilization through advanced AI prompts
- Return on Data Increase: (90% – 60%) / 60% = 50% improvement
This metric ensures that Drip Capital maximizes the value of its accumulated data assets through AI optimization.
A comprehensive 12-step framework for measuring gen AI ROI
Through our conversations with industry experts across multiple sectors—technology, healthcare, finance, retail and manufacturing—we identified patterns in what works, what doesn’t and the blind spots most organizations encounter. Drawing from these insights, we’ve created a 12-step framework to help organizations evaluate their gen AI initiatives holistically.
The idea is to provide IT leaders with a roadmap for measuring, optimizing, and communicating the impact of gen AI initiatives. Rather than relying on outdated ROI models, this framework offers a more nuanced approach, balancing immediate financial metrics with strategic, qualitative benefits.
This 12-step approach balances quantitative metrics like cost savings and revenue generation with qualitative benefits such as improved customer experience and enhanced decision-making. It guides organizations through every phase of the process, from aligning gen AI investments with strategic goals to scaling successful pilots across the enterprise.
This framework ensures that companies capture both financial and non-financial outcomes while maintaining flexibility to adjust as the technology and business landscape evolve:
1. Strategic alignment and objective setting
The success of any gen AI initiative depends on its alignment with broader business objectives. Before diving into implementation, organizations must ensure that the use cases they pursue are linked to strategic priorities, such as revenue growth, operational efficiency, or customer satisfaction. This alignment prevents AI investments from becoming siloed projects disconnected from the core business mission.
Key Actions:
- Identify specific business goals that the gen AI initiative will support.
- Define KPIs and success metrics aligned with strategic objectives.
- Engage executives and key stakeholders to ensure buy-in and clarity.
2. Baseline assessment
Establishing a clear performance baseline is essential to measure progress accurately. This involves collecting data on current processes, outcomes, and key metrics before deploying gen AI solutions. The baseline serves as a reference point for assessing post-implementation impact.
Key Actions:
- Gather quantitative and qualitative data on existing processes.
- Identify bottlenecks, inefficiencies, or gaps that gen AI aims to address.
- Document current performance metrics for future comparison.
3. Use case identification and prioritization
Not all AI initiatives deliver the same value, so it’s critical to identify and prioritize high-impact use cases. Decision-makers should focus on projects with a clear path to ROI, strong strategic alignment, and measurable outcomes.
Key Actions:
- Conduct feasibility assessments for potential use cases.
- Prioritize based on potential impact, ease of implementation, and alignment with long-term goals.
- Build a roadmap for phased implementation to manage complexity.
4. Cost modeling
Effective gen AI deployment requires a detailed cost model that goes beyond upfront investments. Organizations need to capture ongoing operational expenses, including infrastructure, maintenance, and staffing.
Key Actions:
- Estimate costs across all phases of implementation.
- Account for hidden expenses such as training, data management, and change management.
- Develop financial models that include both one-time and recurring costs.
5. Benefit projection
Forecasting potential benefits provides a roadmap for expected outcomes. In addition to financial returns, organizations should project intangible benefits like improved employee satisfaction, decision-making, or customer engagement.
Key Actions:
- Identify both tangible and intangible benefits of gen AI solutions.
- Model scenarios for best, worst, and likely outcomes.
- Develop a timeline for when benefits are expected to materialize.
6. Risk assessment and mitigation
Every gen AI project carries risks, from technical challenges to ethical considerations. Identifying these risks early and developing mitigation strategies ensures smoother implementation.
Key Actions:
- Identify risks such as data privacy concerns, talent shortages, and potential bias.
- Develop mitigation plans, including contingency strategies.
- Assign ownership for monitoring risks throughout the project lifecycle.
7. ROI calculation
Standard ROI formulas may not capture the complexity of gen AI’s impact. Organizations should tailor their ROI models to include direct, indirect, and strategic returns, balancing immediate financial gains with long-term value creation.
Key Actions:
- Use multi-layered ROI models that capture both hard and soft benefits.
- Incorporate time lags in realizing gen AI’s impact into financial projections.
- Adjust models based on pilot results or early outcomes.
8. Qualitative impact assessment
Many of gen AI’s most valuable contributions—such as improved customer experience or enhanced innovation—resist traditional quantification. Organizations need qualitative assessments to capture these impacts effectively.
Key Actions:
- Develop proxy metrics for qualitative benefits where possible.
- Conduct surveys or interviews with employees and customers to gauge satisfaction.
- Use narrative reporting to communicate intangible outcomes.
9. Implementation and monitoring
Implementation must include a robust monitoring system to track progress against benchmarks. Real-time data collection allows organizations to course-correct as needed and ensures that benefits materialize as planned.
Key Actions:
- Set up dashboards for tracking KPIs and other key metrics.
- Monitor progress regularly to identify potential issues early.
- Establish a feedback loop between technical teams and business units.
10. Continuous improvement and optimization
Gen AI initiatives require constant fine-tuning to maximize impact. Regular evaluation and iteration allow organizations to identify opportunities for improvement and adapt to changing needs.
Key Actions:
- Schedule periodic reviews to assess performance and outcomes.
- Identify areas where gen AI models or processes can be optimized.
- Incorporate feedback from users and stakeholders to refine solutions.
11. Scalability and enterprise-wide impact assessment
Once a gen AI solution proves successful in a limited context, organizations must evaluate its potential for broader deployment. Assessing scalability ensures that AI investments deliver value across the enterprise.
Key Actions:
- Identify opportunities to scale successful pilots across departments or regions.
- Assess infrastructure and resource needs for full-scale deployment.
- Track the cumulative impact of gen AI solutions at the enterprise level.
12. Stakeholder Communication and Reporting
Clear communication with stakeholders is essential to maintain alignment and support. Regular reports that capture both financial and non-financial outcomes keep stakeholders informed and engaged.
Key Actions:
- Develop concise, meaningful reports tailored to different audiences (executives, boards, investors).
- Highlight both quantitative results and qualitative achievements.
- Use reporting as an opportunity to align future goals with evolving gen AI capabilities.
Summary Table: 12-Step framework for measuring gen AI ROI
| Step | Description |
| Strategic Alignment and Objective Setting | Ensure gen AI initiatives align with business goals. |
| Baseline Assessment | Establish performance baselines for comparison. |
| Use Case Identification and Prioritization | Focus on high-impact, strategic use cases. |
| Cost Modeling | Capture upfront and ongoing costs comprehensively. |
| Benefit Projection | Forecast both financial and non-financial benefits. |
| Risk Assessment and Mitigation | Identify and mitigate risks throughout the project lifecycle. |
| ROI Calculation | Tailor ROI models to include direct, indirect, and strategic returns. |
| Qualitative Impact Assessment | Capture intangible benefits using qualitative metrics. |
| Implementation and Monitoring | Track progress with real-time data and course-correct as needed. |
| Continuous Improvement and Optimization | Regularly review and refine gen AI processes. |
| Scalability and Enterprise-Wide Impact Assessment | Assess scalability and broader enterprise impact. |
| Stakeholder Communication and Reporting | Communicate outcomes clearly to stakeholders. |
Practical Strategies for Achieving ROI early with gen AI
From our conversations with experts across industries, a clear theme emerged: achieving measurable ROI with gen AI requires more than enthusiasm—it demands a deliberate, strategic approach. Many companies dive into ambitious AI projects, only to encounter challenges in translating initial excitement into meaningful outcomes. The key to success isn’t launching large, complex systems right away but focusing on manageable, high-impact use cases that demonstrate value early.
Below are a few practical takeaways from these expert discussions, designed to help organizations move from gen AI exploration to execution and ROI measurement. These strategies serve as a bridge from planning to sustained value creation, laying the groundwork for effective implementation and continuous ROI growth.
1. Start with focused use cases
Begin with smaller, high-impact use cases: Start with something that offers immediate value without being overwhelming. The trick is to target processes that are both measurable and impactful. This approach avoids the complexity of large-scale rollouts and ensures early wins.
2. Select the right infrastructure
Many companies struggle with infrastructure decisions. Prototype with cloud tools first, then refine as you go. The key is to remain flexible—hybrid or on-prem setups might make sense later, depending on your data compliance needs.
3. Set realistic expectations on returns
Don’t expect miracles out of the gate. The first phase is experimental, and that’s okay. Plan for iterative learning cycles, where teams refine prompts and processes over time to maximize ROI.
4. Maintain human oversight
Keep people in the loop, especially in areas like finance or legal, the AI’s output needs verification. Combining automation with human expertise ensures both efficiency and reliability.
5. Leverage existing data
Organizations sitting on years of data can turn it into a goldmine by refining AI prompts and validating outcomes. Well-curated datasets lead to better, more consistent returns.
Redefining business value in the age of gen AI
In the race to harness the transformative power of gen AI, enthusiasm alone won’t generate returns. As companies confront the complexities of measuring impact, they must move beyond traditional metrics to embrace a more nuanced understanding of value—one that accounts for both tangible and intangible outcomes. The path to success lies not in grand, sweeping implementations but in focused, high-impact initiatives that align with business objectives and evolve over time.
The challenges are clear: a lack of standardization, complexities in attribution, and benefits that often resist easy quantification. Yet, as the experiences of companies like Drip Capital show, a pragmatic, iterative approach—anchored by clear objectives, human oversight, and data-driven insights—can unlock gen AI’s potential. Organizations that treat ROI as a continuous process, refining their strategies and metrics as they go, will be best positioned to turn AI investments into measurable impact.
The true value of gen AI goes beyond cost savings and efficiency gains—it lies in its ability to transform processes, spark innovation, and empower better decision-making. In this evolving landscape, those who succeed will be the ones who reimagine ROI, balancing measurable financial outcomes with strategic, long-term contributions.
VB Daily
Stay in the know! Get the latest news in your inbox daily
Thanks for subscribing. Check out more VB newsletters here.
An error occured.
