someone on Nostr: I think 1 GPU like RTX 3090 or MI60 could be enough for 8B if you want to do lora or ...

I think 1 GPU like RTX 3090 or MI60 could be enough for 8B if you want to do lora or qlora with maybe 64GB ram or less. If you want to do faster, more GPU. More GPU could also allow better learning.

All I did was 4bit qlora of 70B and it seems to work for my purposes, which can be summarized as "aligning a model with human values". I haven't tried full file tuning, freeze fine tuning or 8bit qlora. I bought new cards and will try 8bit qlora soon.