Most companies can build an AI demo in a week. Very few can turn it into scalable AI that runs safely for millions of customers. On 22 October 2026, a session at MIT Technology Review’s EmTech Future conference argued that the difference has little to do with the model. It comes from five layers underneath it.
The speaker was Jing Wang, who leads Vanguard’s Center for AI and Insights, and her premise was blunt: “A scalable AI system requires a strong foundation.” Here is what that foundation is made of, why regulated firms find it hardest, and what the numbers say about everyone who skips it.
Key takeaways
- Vanguard’s session names five layers that must work together: data, engineering, platforms, operating models and governance.
- MIT’s NANDA initiative found that about 95% of enterprise generative AI pilots in its 2025 study produced no measurable profit-and-loss impact.
- Gartner predicts that more than 40% of agentic AI projects could be cancelled by the end of 2027 because of costs, unclear value or weak risk controls.
- In finance, the rules are still catching up. US bank regulators replaced their 15-year-old model risk guidance on 17 April 2026, but left generative AI for later.
What did the EmTech session actually say?
The session, published by MIT Technology Review as a Vanguard-presented talk, makes one central claim. Data, engineering, platforms, operating models and governance have to “converge to work as one”. A strong model sitting on weak plumbing does not scale.
Wang is well placed to say it. Her team runs AI, machine learning, generative AI and behavioural science work at one of the world’s largest asset managers. She joined Vanguard in 2009 after working as a principal at Boston Consulting Group. The talk focuses on two things: building that foundation in a heavily regulated business, and turning newer capabilities, including agentic AI, into “lasting value for investors”. Agentic AI means systems that plan and carry out multi-step tasks on their own instead of just answering a prompt.
The 5 layers of scalable AI, explained
Think of an AI model as the engine in a car. The five layers are the chassis, the fuel, the road, the driver and the traffic laws. A Formula 1 engine bolted to a shopping trolley still goes nowhere.
| Layer | What it covers | What breaks without it |
|---|---|---|
| Data | Clean, documented, permissioned data the model can use | Confident answers built on stale or wrong records |
| Engineering | Testing, monitoring, cost control, rollback | A demo that falls over under real traffic |
| Platforms | Shared tools every team builds on | Ten teams building ten incompatible chatbots |
| Operating model | Who owns the AI, who funds it, who changes the workflow | Pilots that nobody is responsible for scaling |
| Governance | Risk review, audit trails, human sign-off | A compliance team that blocks launch, or a regulator who finds the gap first |
The point of “work as one” is that these layers depend on each other. Governance needs engineering logs to audit anything. Platforms only help if the operating model makes teams use them.
Why do most AI pilots never scale?
The best-known data point comes from MIT’s NANDA initiative. Its report, The GenAI Divide: State of AI in Business 2025, found that about 95% of enterprise generative AI pilots delivered no measurable profit-and-loss impact. Fortune reported that the study drew on 150 leader interviews, a survey of 350 employees and an analysis of 300 public deployments.
Lead author Aditya Challapally blamed a “learning gap” in tools and organisations rather than model quality. Two findings stand out. Tools bought from specialist vendors succeeded about 67% of the time, while internal builds succeeded about a third as often. Also, more than half of generative AI budgets went to sales and marketing, yet the biggest returns came from back-office automation.
That is a foundation problem, not a model problem. It matches what we heard in a survey of 520 leaders on where AI lock-in really comes from.
Why is agentic AI harder to scale?
An agent doesn’t just suggest an answer. It takes actions, such as sending emails, moving data or triggering trades. Every weak layer gets amplified.
In June 2025, Gartner predicted that more than 40% of agentic AI projects could be cancelled by the end of 2027, citing “escalating costs, unclear business value or inadequate risk controls”, as W.Media reported. Analyst Anushree Verma said that “many use cases positioned as agentic today don’t require agentic implementations.” Gartner also estimated that only about 130 of the thousands of vendors selling “agentic” products offer the real thing.
Gartner still expects agents to spread. It forecasts that by 2028, 33% of enterprise software applications will include agentic AI, up from less than 1% in 2024. If you’re building one, our 7-level maturity model for production-grade agents is a useful self-check.
What makes regulated firms like Vanguard different?
A retailer whose chatbot gets a return policy wrong has a bad day. An investment firm whose AI gives a misleading answer about a retirement account has a regulatory problem. That changes the order of work. In finance, governance has to be designed in from the start, because it can’t be added once the system is live.
The rules themselves are moving. On 17 April 2026, the Federal Reserve, OCC and FDIC issued updated model risk management guidance. The Fed’s SR 26-2 letter replaces SR 11-7, the 2011 rulebook banks have used to validate their models. According to Baker Tilly’s analysis, the new guidance does not address generative AI or advanced AI tools, and separate guidance or rulemaking is expected to cover them.
So regulated firms are building AI foundations before the final rules exist. In my view, that is the strongest argument for Vanguard’s approach. If your data lineage, audit logs and human checkpoints are already solid, a new rule means adjusting what you have instead of starting over.
What happens next
- AI-specific bank guidance. US regulators have signalled that generative AI risks will get their own treatment. Watch for a consultation or request for comment.
- The Gartner test. The end of 2027 is the date for Gartner’s cancellation forecast. Agent projects that started without a clear return will face budget reviews well before then.
- Back-office first. If the MIT findings hold, expect more spending to shift from flashy customer-facing tools to quieter internal automation, where the returns show up.
FAQ
What does scalable AI mean?
It means an AI system that keeps working reliably, affordably and safely as usage grows from a small pilot to an entire business. Scalability depends on data, engineering, platforms, ownership and governance as much as on the model.
Who spoke at the EmTech “Foundations of Scalable AI” session?
Jing Wang, principal and head of Vanguard’s Center for AI and Insights. MIT Technology Review published the session on 22 October 2026.
Does new US bank guidance cover generative AI?
Not directly. The April 2026 interagency model risk guidance (SR 26-2) does not address generative AI, according to Baker Tilly’s analysis. Separate AI guidance or rulemaking is expected.
Next step: choose one AI pilot your team runs and score it against the five layers in the table above. The weakest layer is the one to fix first, and our 7-point agent evaluation checklist can help.
Leave a comment