I’ve cancelled my subscription yesterday, conveniently it was about to renew at the end of the week, so I have a few days to migrate from Proton, here’s what I’m thinking of, but I’d love to hear what other people are switching to
I’ve cancelled my subscription yesterday, conveniently it was about to renew at the end of the week, so I have a few days to migrate from Proton, here’s what I’m thinking of, but I’d love to hear what other people are switching to
What does “lower-end” mean in this case? 8GB VRAM could be anything from a 2014 Radeon R9 290X to a 2026 RTX 5060.
Radeon RX 7600, but yes, you’re right, it’s relative. It’s good enough for any gaming I do, which is not an awful lot. I think it’s considered mid-range for its generation, I guess I was just thinking of the GPUs I see people mention they have which seem much beefier than mine lol.
I’m no AI expert but when I looked into it, it seemed that VRAM was the biggest factor being mentioned in discussions of GPUs, hence why I mentioned it.
I could be wrong, but I think there are differences not just in FLOPS but perhaps more importantly in what kinds of functions are supported (with newer GPUs becoming gradually more general-purpose), so I’m not sure if the GPU from 2014 is capable of running AI models at all (slowly or otherwise).
I actually kinda want to find out, because my old GPU is a Vega 56 (which is also 8GB) and I’m not sure if it would be useful or not.
I’m sure you’re probably right. I haven’t looked into how things like clock speed/FLOPS/etc affect LLM performance.
The main thing is you need ROCm/CUDA/Vulkan support. If you do then just try running some models that fit into your VRAM and see how they perform? On my card, it takes maybe a minute or two to generate an answer, sometimes several minutes if the answer is long. I use small models that are <8GB in size, so not the best open-weight ones, but I don’t use LLMs for anything particularly complex anyway so the dumb ones do the trick for me.