GenAI Framework for Building and Scaling Enterprise Solutions
- By Mathews Thomas, Utpal Mangla and Dinesh Verma, IBM
- February 05, 2024
Artificial intelligence has been around since the 1950s, but even to skeptics, the recent advances with GenAI significantly move the needle forward. There has been massive early adoption, and Goldman Sachs estimates that Generative AI could raise global GDP by 7% within 10 years. However, while the focus has been on GenAI, it is essential to remember that any GenAI strategy needs the right data and that it can govern this data effectively. And all of this needs to be integrated effectively with the GenAI tool. An effective GenAI strategy will include the following key components.
GenAI Engine: The engine that enables you to train, validate, tune and deploy generative AI, foundation models and machine learning capabilities with ease and build AI applications in a fraction of the time with a fraction of the data. Critical use cases built on these engines enable sophisticated Q&A, summarization of content, classification of content and generation of content for a specific purpose.
Data Engine: The GenAI engine will need large amounts of data. Enterprises must scale analytics and AI with a fit-for-purpose data store supported by querying with open data formats to access and share data. It is essential to connect to data in minutes, quickly get trusted insights and reduce your costs. There will be existing data engines in most enterprises. They need to be extended to support data from multiple sources, including data warehouses and data lakes, quickly and cost-effectively. Use cases include deploying AI/ML at scale, applying real-time analytics/BI and streamlining data engineering.
Governance Engine: The GenAI engine needs data to drive responsible, ethical decisions across the business. This includes directing, managing, and monitoring your organization’s AI activities, strengthening your ability to mitigate risk, managing regulatory requirements, and addressing ethical concerns. The key use cases included lifecycle governance, risk management and regulatory compliance.
These three engines need to work together to support AI-enabled applications. While the data engine provides access to enterprise data, the Gen AI engine enables new AI-enabled applications to use its models, and the governance engine controls the operation of the AI engine to prevent hallucinations, reduce model bias and ensure compliance with applicable regulations. See below for a high-level framework for the interaction between them.
Use Case: Let’s consider a specific use case to illustrate the use of the above:
Use case overview: A telco wants to improve customer experience when their subscribers interact with the telco using the chat or call center channels. The following needs to happen for the enterprise to achieve this:
- Its existing chat and call center systems need to be integrated with GenAI to improve the customer experience through the channels. Improving the experience includes being able to access the relevant data, including past customer interactions with these channels, customer sentiment, access to various back-end systems such as billing and recommendations on how to respond to the customer.
- Various systems spread across data lakes, data warehouses and other sources can store the required data.
- The telco needs to ensure all the data is governed correctly, such as knowing the data lineage of the data use, ensuring the data meets regulatory requirements and monitoring the AI for drift and biases, including corrections when necessary.
Solution:
- The data engine is used to gather the right data from different sources, including data lakes, data warehouses and other sources. The data will then be transformed and consumed by the GenAI engine.
- The governance engine keeps track of data lineage, including where the data is sourced from, ensures the models operate within specifications and confirms that the models and data adhere to compliance rules by providing the right tools, including access to relevant dashboards and KPIs.
- The GenAI engine uses the data made available from the data engine, which may require additional tuning and training to the engine to ensure that the chat system and call center agent use the best data in their interactions.
- The call center application and chat assistant integrate with the GenAI APIs to enable subscribers to interact with the telco.
Below is a description of the framework's functions used in this application. Please note that only the functions discussed in this write-up are provided as examples, and many more will be needed for a full-scale deployment. Various data sources are available to the GenAI engine using the data engine. The Governance engine governs the data to ensure governance requirements are met. Different applications are integrated with the GenAI engines to provide the GenAI functions, and the application may also access additional data directly through the data engine.
As seen above, implementing the use case requires the GenAI system to interact with multiple systems. You must consider the GenAI engine, data engine and governance engine as part of your GenAI strategy. These technologies need to be built in an open environment using the best AI and cloud technologies running in a hybrid cloud environment with access to the innovation of the open community.
The views and opinions expressed in this article are those of the author and do not necessarily reflect those of CDOTrends. Image credit: iStockphoto/Maxger
Mathews Thomas, Utpal Mangla and Dinesh Verma, IBM
Mathews Thomas is a distinguished engineer at IBM. He has held various research, software development, consulting, technical sales, and marketing positions. He currently focuses on Telco and Media & Entertainment and has prior experience in Retail, Industrial and E&U. His technical focus areas include GenAI/AI/Analytics, analytics, hybrid cloud, blockchain, 5G and edge computing.
Utpal Mangla is a general manager responsible for Telco Industry & EDGE Clouds in IBM. Before that, he was the vice president, senior partner and global leader of TME Industry’s Centre of Competency. He led the 'Innovation Practice' focusing on AI, 5G EDGE, Hybrid Cloud and Blockchain technologies for clients worldwide. Under Utpal's leadership, IBM recently achieved the mission of scaling to make "Watson AI Impact 1.5 Billion Consumers” and creating “Industry Blockchain platforms”. Utpal is a Master inventor and is at the forefront of making Hybrid Cloud and 5G/EDGE real for enterprises globally
Dinesh Verma is an experienced researcher, business leader, innovator and software developer at IBM. He is an IEEE Fellow, IBM Fellow, AAIA Fellow and Fellow of the U.K. Royal Academy of Engineering. He has authored 11 books, 200+ technical papers and 200+ U.S. patents and led multiple multi-national, multi-organizational research programs for over 15 years. He contributed to several IBM products and service offerings with documented business impact exceeding USD4B+. At IBM, he has served in various roles, including CTO, strategist, chief scientist, and senior manager.