I’ve cancelled my subscription yesterday, conveniently it was about to renew at the end of the week, so I have a few days to migrate from Proton, here’s what I’m thinking of, but I’d love to hear what other people are switching to
I’ve cancelled my subscription yesterday, conveniently it was about to renew at the end of the week, so I have a few days to migrate from Proton, here’s what I’m thinking of, but I’d love to hear what other people are switching to
I’m sure you’re probably right. I haven’t looked into how things like clock speed/FLOPS/etc affect LLM performance.
The main thing is you need ROCm/CUDA/Vulkan support. If you do then just try running some models that fit into your VRAM and see how they perform? On my card, it takes maybe a minute or two to generate an answer, sometimes several minutes if the answer is long. I use small models that are <8GB in size, so not the best open-weight ones, but I don’t use LLMs for anything particularly complex anyway so the dumb ones do the trick for me.