Alan Siefert on Nostr: With 64GB RAM I’d guess you could locally run the 70B version just fine with a 4000 ...

With 64GB RAM I’d guess you could locally run the 70B version just fine with a 4000 series Nvidia GPU. Not sure about the 405B version. Supposedly the 5000 series GPUs are releasing beginning of 2025. I wonder if they will have LLMs in mind in their design.