The Bill Nobody Modeled: What Happens After the AI Pilot Works

Every AI pilot has a Day One. The demo works, the board nods, and the budget expands. But far fewer survive Day Two. That’s because the system runs against real data and at real volume, every day. This is when the bill stops looking like anything anyone approved.

The numbers back it up. According to the Deloitte-HKU AI Adoption Index 2026, only 23% of companies across Hong Kong and mainland China say AI delivered real financial results. Just 4% say they have hit their transformation goals. Globally, it’s worse: MIT’s NANDA project found that about 95% of companies running AI pilots see no measurable return at all.

The real bill does not come from buying the AI; it comes from running it.

This is not new. Client-server computing hit the same wall in the 1990s. So did service-oriented architecture in the 2000s, and the first wave of cloud computing after that. While the demo always looked great, the real cost showed up later, once the system had to run every day, at scale, for real people.

The diagnosis nobody agrees on

Deloitte and HKU blame company culture. Teams that do not talk to each other, staff who resist change, and managers who do not understand what AI can and cannot do. That’s fair. But ask the people who build and run these systems, and you get a different answer: the infrastructure underneath was never built to carry this much weight.

“Day 2 operations and scalability break it,” says Albert Law, director and head of solution architecture for Hong Kong, Macau and Greater Bay Area at Red Hat. “Without a unified, scalable platform like Red Hat OpenShift AI, businesses encounter inconsistent environments, shadow IT, and compliance gaps. We witness AI project pilots fail at scale because they lack integrated MLOps.” What’s left, he says, are “model silos,” which are pockets of AI experimentation that are impossible to govern or scale once they leave the sandbox.

Where the money actually goes

One widely cited estimate, from cloud cost firm Cast AI and reported by VentureBeat this year, says the average enterprise uses only 5% of its GPU capacity. Gartner expects new AI infrastructure spending to hit USD401 billion in 2026. If you do the math, it shows that most of that spend buys silicon that sits idle.

The fix, according to Law, starts one layer down. “The biggest drain on enterprise budgets is idle time and fragmented resource allocation,” he says. “GPUs are often provisioned but massively underutilized due to poor orchestration.”

His approach uses open-source software like vLLM, running on an enterprise Kubernetes platform. This lets workloads share the same chips instead of every team hoarding its own. Red Hat also offers compressed versions of open models. These, Law says, “maintain ~99% accuracy while significantly reducing the compute footprint.” The upshot: companies stop paying for more compute than the job needs.

For CFOs concerned about runaway token spend, Law advises them to reframe the actual challenge: “Your escalating AI costs aren’t driven by the models, but by bespoke infrastructure creating massive operational debt. Every isolated pilot multiplies maintenance and security costs.”

Here, a standard platform turns “unpredictable R&D expenses into a governable, scalable factory.” It moves a company from what Law calls a “token consumer” to an “AI provider.”

The part that should worry the CISO

Then there are the autonomous agents. Once an AI system can act on its own and call its own APIs, a bad prompt can become a live wire into systems nobody expected to be exposed to the internet.

“Agentic systems operate autonomously and at high velocity, easily overwhelming legacy core systems if connected directly,” Law says. “More than overloading, agents need guardrails because they autonomously reason, call APIs, and execute code, exposing systems to severe jailbreaks and unauthorized data access.”

Law’s solution reads like an airport security line, but built for machines. A gateway filters what an agent can see and do. Each agent gets its own identity and permissions, instead of one shared password. Containers isolate what it can touch, and a human must approve anything sensitive.

Sovereignty, defined

In Hong Kong, that budget question leads straight into a second one: who really controls the technology?

This is no longer a theoretical debate. In March 2026, the Hong Kong Monetary Authority, the Securities and Futures Commission, the Insurance Authority and the Mandatory Provident Fund Schemes Authority, together with Cyberport, launched the GenA.I. Sandbox++, giving financial firms supervisory guidance and free GPU computing power to test generative AI use cases safely under regulatory oversight.

Three months later, in June 2026, the SFC warned firms to harden their systems against a new class of threat: attackers using AI itself to discover vulnerabilities, automate attacks and break in faster than defenders can respond. By August 2026, regulators had selected the first cohort of 36 AI pilots from banks, insurers and fintechs, with technical trials set to begin later in the year on Cyberport’s platform.

Regulators are no longer asking whether financial firms will use AI. They are asking who controls it, and how fast a firm can prove that on paper.

This is when you often hear about “sovereignty,” a word that gets used in boardrooms to mean almost anything. Law breaks it down into four parts:

• Data sovereignty: who controls where training and inference data sits.
• Model sovereignty: whether you can audit and change the model, not just rent access to it.
• Infrastructure sovereignty: whether it runs anywhere, air-gapped included.
• Operational sovereignty: who can touch the system, and whether the bill is predictable.

Put these four together, Law says, and “it means owning the entire execution environment: data location, model selection, hardware/infrastructure choices and daily operation.”

“Your escalating AI costs aren't driven by the models, but by bespoke infrastructure creating massive operational debt.” — Albert Law @ Red Hat 

Here is a simple test. If your AI model provider raised its price, or pulled the product, next quarter, how fast could your company switch to something else? “Some enterprises admit it would take quarters,” Law says. He points to vendor lock-in: companies get stuck inside proprietary APIs or “AI in-a-box” hardware. On a platform that is not tied to one type of infrastructure, Law says, the truthful answer becomes “immediately.” Swapping one model for another does not mean rebuilding everything around it.

There’s a harder problem for companies operating across borders. The same AI model, fed different regional data, behaves differently depending on which side of the border it runs. Law says this is not a bug that needs fixing by forcing both sides to act the same. “The model isn’t broken; it is dynamically adapting to its distinct environment,” he says. The harder task is explaining the difference, not eliminating it.

The question nobody’s asking

That question is the bill nobody modeled, the one that shows up after the pilot works: what a company owes, every month, for infrastructure it doesn’t control. Law’s version of it, for any board still asking the wrong questions about AI: “Do we actually own our AI ecosystem, or are we just renting it?”

Given how few companies can answer yes, that’s where the next AI budget meeting should start — not with the model, but with what happens after it works.

Red Hat and CDOTrends are hosting exactly that conversation on Sept. 15, 2026. A selected group of Hong Kong senior tech leaders will sit down to discuss “The Model Isn’t Your Next Decision. The Platform Is.” They will look at three concerns: where the AI budget leaks once inference replaces training as the highest cost, what sovereignty means once ownership is separated from geography, and what a single open source platform, spanning servers to agents, saves once vendor lock-in is off the table.

Day Two comes for every AI pilot eventually. The conversation at the table on Sept. 15 is about being ready for it.  

Image credit: iStockphoto/Deagreez