With the proof of concept done and users able to get over 180gb/s on a PC with AMD’s 3d vcache, it sure would be nice if we could figure a way to use that bandwidth for CPU based inferencing. I think it only worked on Windows but if that is the case we should be able to come up with a way to do it under Linux too.
180GB/s isn’t really all that fast.
Maybe, but it’s a lot faster than what we can do right now and its only the start.