Part of my job is keeping an H200 cluster alive for research projects and workloads heavy enough to need it. Eight of those GPUs wired into one NVLink domain can hold a 405 billion parameter model in memory without flinching, moving 4.8 terabytes of data every second through HBM3e like it's nothing. Standing next to that rack still does something to me, the fans pulling cold air through a machine that costs somewhere around a quarter million dollars for a single fully configured node. It is built for a scale that most companies, including most of the ones I've worked with outside that job, will never need and could never justify.
On my own desk sits something almost comically smaller. A Gigabyte AI TOP Atom, roughly the size of a hardback book, holding 128 gigabytes of unified memory across a Grace CPU and a Blackwell GPU, the same GB10 silicon underneath NVIDIA's own DGX Spark. I use it for the inference testing and data analysis I don't want touching a cloud endpoint, the kind of work where the whole point is that nothing leaves the room. It has been sitting a few feet from my keyboard for months now, quietly doing exactly what I bought it for, and somewhere in that quiet is where this idea started.
Because the gap between that rack and this box isn't really a gap in capability. It's a gap in scale, and scale is something you can rebuild from smaller pieces if the interconnect will let you. For the price of one H200 node, a business could buy four DGX Sparks and still have money left for the cables to connect them. Four boxes, stacked in something closer to a closet than a data center, adding up to 512 gigabytes of memory to work with. That is enough room to hold models that not long ago needed a server rack of their own, and it's the architecture I keep sketching out when I think about what a small business actually needs from AI infrastructure it owns outright.
NVIDIA only ever promises you half of that dream out of the box. Two Sparks will talk to each other over a direct cable through the ConnectX-7 network card built into each one, moving 200 gigabits a second between them, enough to run a 405 billion parameter model quantized down to fit. That pairing is real and documented. Getting from two boxes to four is not. There is no button for it, no mode you switch on, just two more Sparks and a networking decision nobody at NVIDIA is going to make for you.
The honest answer is a small managed switch, and this is where the plan lives or dies on a part most people would treat as an afterthought. A 25 gigabit switch sounds like plenty until you actually load a model across more than two nodes, at which point the tokens slow to something you could count out loud. That's the moment NVLink stops being a spec sheet line item and starts being the reason data centers don't run their GPUs over ethernet. A 100 gigabit switch closes most of that gap, though never all of it. Some distance between a dedicated fabric and a network cable is just going to remain, no matter how much bandwidth gets thrown at it.
The software layer that would stitch four boxes into one thinking machine is vLLM, running on top of Ray to split the model across nodes with pipeline parallelism. None of this is plug and play. Every layer of the model has to hop across ethernet during prefill, while the machine is still building its understanding of what's been asked, and that hop costs time you don't pay when everything lives on one card. But it holds together once it's running, and from what I've seen in similar deployments at work, it holds up under more than one request at a time without becoming fragile. Quantizing everything down to 4-bit before it touches the cluster isn't optional either, because the memory a growing conversation consumes will eat the room you thought you had faster than expected.
What would actually fit inside those 512 gigabytes is the part that makes this worth proposing at all. Llama 3.1 405B fits at 4-bit with room left for that growing conversation memory. DeepSeek's 671 billion parameter models fit too, if the quantization goes down to 3 or 4 bits and you accept a tight fit rather than a comfortable one. Qwen 72B and Llama 3.3 70B need none of those compromises, running at full 16-bit precision with space to spare for long conversations. Mixtral 8x22B runs easily enough to leave room beside it for a second, smaller model handling embeddings or routing requests to the right place. That's the same class of open weight model the frontier labs are talking about, running on hardware a small business would own outright.
None of this is a machine for training anything, and I wouldn't pitch it as one. The memory bandwidth isn't there for it and neither is the interconnect. This is an architecture built to answer questions, not to learn new ones, and pretending otherwise to a client would be the kind of mistake that costs more than the hardware did.
There's a ceiling here worth stating plainly. If a business needs to answer hundreds of people at once without any of them noticing a delay, four Sparks will let that business down, and the honest recommendation at that point is to keep renting time on someone else's H200s. But most small businesses I can picture aren't running that scale. They're running the handful of questions a small team actually asks in a day, and for them the deciding factor usually isn't speed at all, it's the fact that a customer's data never has to leave a closet the business controls. That's the same reason my Atom sits on my desk instead of a browser tab pointed at someone else's API, just scaled up to a team instead of one person.
The electricity bill makes the same argument in smaller print. Each Spark pulls somewhere around 200 to 240 watts under load. All four running at once would use less power than the cooling system alone on an H200 rack, the part of that machine that never even touches the model. I think about that comparison more than I probably should, the quiet box that already sits on my desk and the roaring rack I help keep running at work, both chasing the same answer from very different distances and very different price tags. For a small business trying to own its AI instead of renting it, the four-Spark closet is the version of that answer actually built to fit.