AI inference is changing shape. What used to be short, stateless requests is becoming long-lived, multi-turn reasoning across users and agents. As that shift accelerates, the bottleneck is no longer raw GPU compute. It is memory, context, and how efficiently inference history can be stored and reused without slowing everything down.
That is the problem
VAST Data is addressing with its updated inference architecture, which runs the VAST AI Operating System natively on NVIDIA BlueField-4 DPUs and Spectrum-X Ethernet. The focus is simple: make large, shared inference context predictable and usable at scale.
Why GPU-first assumptions break down
Many enterprises still assume GPU or CPU memory can hold inference history. That assumption fails quickly as models run longer and serve more users.
Aaron Chaisson, VP Product and Solutions at VAST Data, explained to ChannelE2E, “The assumption that GPU/CPU memory capacity can handle inference history breaks immediately. Once context grows beyond hundreds of thousands of tokens, it simply doesn't fit, requiring external storage to house large kvcaches required for multi-turn AI and large user/agent concurrency.”
The access pattern also changes. “Traditional storage approaches assume infrequent, bursty access patterns,” Chaisson says. “But agentic workflows create sustained, concurrent access where every GPU is constantly reading and writing context.”
At that point, the limiting factor shifts. “Infrastructure teams will find that storage and/or network bottlenecks and data path inefficiencies become the dominant performance constraint, not raw GPU compute,” he adds. This is why VAST is working closely with NVIDIA to enable line-rate data transfers from SSD to GPU memory.
Self-Contained GPU hosts
VAST’s design removes the classic separation between compute and storage by embedding data services directly into the GPU server using BlueField-4 DPUs.
For day-to-day operations, this reduces moving parts. “Infrastructure teams eliminate an entire fleet of x86 storage servers, reducing the number of systems to provision, monitor, and maintain,” Chaisson says.
Each GPU host becomes self-contained. “Each GPU host becomes self-sufficient with its own dedicated file server running on the BlueField-4 DPU, eliminating complex storage cluster coordination and troubleshooting cross-server dependencies,” he explains.
That also improves consistency. “Performance becomes more predictable because VAST uniquely guarantees no contention for shared storage server resources,” Chaisson notes. With VAST software running on each DPU, every host can access SSDs in parallel without waiting on other systems.
Sharing context without losing control
As inference context becomes shared across agents and tenants, speed alone is not enough. Teams also need isolation, policy, and auditability.
According to Chaisson, those controls are enforced directly in the data path. “The VAST AI OS on BlueField-4 DPUs enforces placement, access control, and validation directly in the data path at line rate,” he says.
Optional services add governance where required. “VAST's optional data services enable audit trails, versioning, data retention limits and compliance controls for regulated industries without reintroducing the overhead that CME is designed to eliminate.”
What this means for MSPs and service providers
For MSPs and channel partners, the question is whether this can realistically be delivered as a managed service.
“This can realistically be packaged as managed AI infrastructure or ‘AI factory’ services,” Chaisson says. “The main limiter for MSPs is usually economics and deployment scale, not operational complexity.”
Removing a separate storage tier makes deployments easier to repeat. “Providers avoid standing up and operating a separate x86 storage tier, which makes deployments more repeatable and appliance-like,” he explains. That means fewer systems to manage and fewer cross-tier issues to troubleshoot.
Chaisson sees the best fit at the high end of the market. “Where it fits best is with GPU clouds, sovereign AI providers, and large enterprises delivering managed on-prem or colo AI factories,” especially where long-lived context and high concurrency make data movement the gating factor. In those environments, predictable performance, isolation, and governance directly support SLAs and managed service differentiation.
As AI becomes more agentic, inference behaves less like a bursty compute job and more like a shared memory system. VAST’s architecture reflects that shift by treating context as core infrastructure, not an afterthought. For teams planning multi-turn, multi-agent workloads, the takeaway is clear. Managing context efficiently and predictably will matter as much as GPU choice when it comes to performance, cost control, and production readiness.
