With a dedicated 3090 (another card for OS) a 34b 5bpw just fits and runs very fast. Like 10-20t/s. The quality is good for my application, but I’m not coding.

CPUs don’t run LLMs.

Even if you get a 3090 with 24gb of vram, you’re going to load the biggest model you can and realize it is useless for most tasks. Less than that and I don’t even know what you would use it for.

For some people “uncensored” means it hasn’t been lobotomized, but for others it means it can write porn.

We are very close to the point where people can credibly insist that a real video is a deepfake. But so far I’ve only seen realistic deepfakes where the face is swapped.

When a deepfake of someone actually doing bad things is possible, we will see people claiming the video is fake.

When this is trvial to do, I’m not sure what happens to the concept of security cameras. The recordings may need to be hashed and “notorized” on a blockchain to be legally valid.