Training Wheels Off: How GenAI is Pushing Storage to the Limit
- By Winston Thomas
- March 26, 2024
Remember when everyone said ChatGPT was the AI breakthrough of 2023? Well, brace yourself. The true power of Generative AI (GenAI) is about to be unleashed, riding on a wave of new use cases in 2024—and your data infrastructure better be ready for the ride.
But prepping your data infrastructure should not be skin deep. With massive datasets and hungry AI models on the horizon, we need a new mindset about how we store data and a reexamination of our organically grown and inherited storage subsystems.
These are some of the critical insights industry heavyweights like Nick Eshkenazi, chief digital officer at Astellas Pharma, Milind, AI scientist at Mercedes-Benz AG, and Matthew Oostveen, chief technology officer for Asia Pacific & Japan at Pure Storage, recently highlighted in the Pure Leadership Series panel, “Will GenAI Break Your Data Infrastructure?” Below are four key observations they shared during the lively discussion.
Observation 1: A mindset reset is sorely needed
Stochastic GenAI just flipped the script on how we think about storage. Forget those tidy transactional databases—GenAI demands scale and flexibility only object storage can provide and focuses on data quality.
However, stopping at data pipelines and databases will not cut it. To put this emerging technology at the center of our IT strategies, we need storage that can easily expand and scale as quickly. The reason: we’re not just ingesting data but also creating new ones.
All these issues make data outages a dealbreaker. Whether you’re training a massive model or deploying it in production in the inference stage, systems can’t afford to blink—that translates to missed insights or, worse, lost revenue.
Beyond resiliency, GenAI demands a lightning-quick storage system. Imagine AI models churning through massive datasets in realtime, needing instant access to critical information. We’re talking about high-performance flash and NVMe technology meticulously tuned for AI’s unique demands.
Don’t overlook the human aspect of GenAI storage, i.e., keeping your rather expensive but hard-to-find data scientists busy. Fast and resilient storage can help them do their work efficiently, while a slow one will see them twiddling their thumbs.
Observation 2: When GenAI goes live, your storage can’t choke
Okay, you’ve trained your GenAI model. Now, the real fun begins—unleashing it on the world in the inference stage.
However, the heavy lifting is far from over. You need to find out whether your inference game is ready. After all, we’re discussing exploding datasets that could choke your system and leave expensive GPUs idle.
Picture this: your shiny new model, trained on carefully curated data, is now bombarded with massive real-world datasets full of noise and surprises. Those GPUs you invested heavily in will be useless if your storage solution can’t keep up.
It’s a high-stakes race against time—slow data delivery means lost productivity and wasted potential. Your storage system needs to be optimized for speed, reliability, and the unique demands of AI inference.
Observation 3: Cloud vs. on-prem? Well, GenAI just upped the stakes
The cloud promised limitless AI possibilities, but GenAI is rewriting the playbook. Forget the one-size-fits-all public cloud pitch. Sure, it’s easy, but those skyrocketing bills? Ouch.
Savvy organizations are crunching the numbers, sometimes even building their own mini “AI clouds” with shared GPU clusters for ultimate cost control. Nothing beats the raw speed of your data center paired with cutting-edge storage. GenAI models can be sensitive to storage latency.
Then, there’s data sovereignty. Some industries can’t afford to have their most valuable assets floating in the cloud—it’s where hybrid models come in. It’s all about finding that sweet spot between cloud flexibility and the iron-clad control of in-house infrastructure.
Of course, the public cloud still has its allure. Easy setup and those specialized AI toolsets are tricky to replicate on your own. The cloud vs. on-prem debate rages on, but with GenAI in the mix, it just got a whole lot more complex—and a whole lot more interesting.
Observation 4: Your AI investment needs to include storage
Think of this nightmare scenario: Your shiny new GenAI model spits wildly inconsistent results despite the pristine code. Your GPUs, a significant investment, are barely breaking a sweat. Yet, those AI project costs are spiraling out of control.
Before you blame the algorithm, a silent kink lurks in your infrastructure—outdated storage systems. Look for these telltale signs of trouble: Erratic data access and inconsistent quality. Your storage could be choking your model.
Besides, idle GPUs can be another bleeding wound on your balance sheet—if your storage can’t keep up, all that processing power is wasted.
It’s precisely why storage can’t be another money pit or an afterthought. You need storage solutions that strike the perfect balance between power and cost-efficiency, and you need them now.
The bottom line: Ignoring your storage layer in the GenAI era is a recipe for costly failures.
Tear up the playbook, GenAI demands a storage rethink
As the GenAI revolution picks up, the old rules no longer apply. The IT and AI teams must step up to stay ahead of the curve.
Think of it like an F1 pitstop: We must swap out those clunky legacy storage systems for ones built for GenAI’s raw power.
We also need new benchmarks—forget the old IOPS standards. We need metrics that measure how well storage handles GenAI’s massive, unpredictable workloads. And we need real-time monitoring tools designed for the unique ways AI gobbles up data and stresses infrastructure.
Whatever the future might hold, it’s clear that the GenAI train has left the station—it’s time to upgrade our tracks. It’s a brave new world of experimentation, constant iteration, and tossing aside outdated assumptions. Those who adapt their storage will thrive; the rest will be left in the digital dust.
This article is part of a CDOTrend eGuide on AI and Storage. To follow the various insights, trends and latest solutions, download the eGuide here.
Image credit: iStockphoto/ALLVISIONN
Winston Thomas
Winston Thomas is the editor-in-chief of CDOTrends. He likes to piece together the weird and wondering tech puzzle for readers and identify groundbreaking business models led by tech while waiting for the singularity.