Gartner Generative AI: Shifting Gears to GenAI at the 2024 Data & Analytics Summit

This will be the Year of Early GenAI Failures 

According to Gartner analysts – and backed up by the executives I met at the summit – 2024 will see organizations shifting from explorations to PoC’s and pilot executions.  

They’re creating multi-disciplinary teams – composed of business, IT, data, finance and legal – to plan, manage, and run the GenAI pilot projects. This year’s hot GenAI use cases are customer support and chatbots, with the goal of cutting costs and improving customer experience and retention.

Despite these expectations, Gartner predicts that 30% of all GenAI projects will fail – due to poor data quality, inadequate risk controls, escalating costs, and/or unclear business value. 

Source: 10 Best Practices for Scaling Generative AI Across the Enterprise, Arun Chandrasekaran 

Enterprise Data Must be Part of the Equation 

In the various session I attended, there was a lot of talk about AI data – specifically around making your data GenAI-ready, especially in terms of addressing common challenges like hallucinations, data leakage, lack of source attribution, and outdated or biased data.  

Many organizations prefer to extend GenAI with Retrieval-Augmented Generation (RAG) frameworks because it allows them to ground LLMs with enterprise data in a simple and cheaper way. But before beginning a PoC or pilot project, you need to make sure you have the data for your selected use cases, and that the data is GenAI-ready by the following criteria:

1. Accuracy 

   Ensuring data accuracy is of the utmost importance because it directly impacts a GenAI app’s ability to generate reliable and valid responses. 

2. Enrichment 

   Augmenting the relevant dataset with additional context or metadata improves RAG GenAI understanding and creativity. 

3. Fairness 

   Assessing and mitigating data biases assure that the GenAI’s outputs are balanced, without perpetuating existing inconsistencies. 

4. Security 

   Protecting data integrity and confidentiality is crucial to preventing unauthorized access and manipulation of RAG AI frameworks.  

5. Governance 

   Establishing clear governance protocols for data management helps maintain quality and compliance standards throughout the GenAI lifecycle.  

 Source: What Everyone in D&A Needs to Know About (Generative) AI: The Foundations, Joe Antelmi 

Among the 10 best practices Gartner recommended, the 1 that grabbed my attention most was, “GenAI models deliver the most value when combined with organizational data.”  

Why? Because if GenAI is a game-changer for customer-centric operations, then customer data MUST be part of the equation. It’s what turns a generic LLM into one that knows your business, and more importantly, your customers. This concept resonates well with the enterprises we’re talking to.

Get the condensed version of the Gartner RAG report absolutely free. 

Structured Data is the Missing Link 

So, if everyone agrees that enterprise data is your differentiator, and that customer data makes customer-centric GenAI use cases (like RAG chatbots) effective, then why aren’t we hearing about more successes here?

After pondering this question with Gartner analysts, as well as with my fellow colleagues at the summit, I concluded that perhaps customer data is NOT yet ready for GenAI. And here’s why:

• Conversational customer interactions require quick responses, based on current information. Does your data infrastructure allow you to access the most up-to-date information about a customer in real time, concurrently?

• Customer data is typically spread across multiple enterprise systems (CRM, billing, invoices, etc.) Can your data infrastructure provide a complete view of a customer that can be injected to the LLM to provide accurate, personalized, and relevant responses to customer queries?

• Security and governance are critical for safe LLM usage of customer data. Without the right controls in place, can you guarantee that a particular customer can access his/her data only?

• Translating data to contextual, meaningful LLM prompts is challenging. Does your organization have expertise required in both data engineering and prompt engineering? 

For me, the 2024 Orlando summit was about shifting gears. I see organizations moving too quickly to GenAI pilot execution, only to realize that enterprise data, specifically structured customer data, is critical to GenAI success. 

Retrieval-Augmented Generation is Today’s Best Fit 

One of the GenAI frameworks being bantered about at the event was Retrieval-Augmented Generation (RAG). In fact, I addressed RAG optimization at my speaking session. RAG is a generative AI framework that augments an LLM with fresh, trusted data retrieved from authoritative internal sources, to generate more informed and reliable responses.

While lots of vendors and analysts refer to RAG in terms of augmenting an LLM with unstructured data, we believe that – for RAG to be truly effective – it must also augment the LLM with real-time data from your enterprise systems, on demand.  

We’ve been working on a GenAI Data Fusion product, that unifies and manages multi-source enterprise data so that it can be queried by, and injected into, the LLM as a contextual prompt – in real time. Think about it as RAG for structured data.

K2view GenAI Data Fusion powers your RAG conversational AI apps to generate accurate recommendations, information, and content for many customer-centric use cases, including:

• Accelerating issue resolution

• Creating hyper-personalized marketing campaigns

• Generating personalized cross-/up-sell recommendations for call center agents

• Detecting fraud by identifying suspicious activity in a user account 

 Discover the K2view GenAI Data Fusion RAG tool.