I don’t even think lanes really matter when you’re not training.

For people like us who don’t share the gpu, it doesn’t make much sense outside of rare cases.

Multiple agents talking to each other. Quickly parsing a knowledge base. Sampling methods like: tree of thought, the simple old beam search, or using multiple prompts.

I don’t want to spend that amount of money, but I definitely want play on one for a few months.

but in a nut shell I think Microsoft will eventually push out a local LLM to machines to cut down on a lot of resources and cost

Not their valuable LLMs. You don’t want to give those weights away.

H100 in the DGX is not the H100 PCI-e, but about 30% faster. When in doubt, just go DGX