Is more VRAM always better?
Hello everyone.
Im not interested in training big LLM models but I do want to use simpler models for tasks like reading CSV data, analyzing simple data etc.
Im on a tight budget and need some advice regards running LLM locally.
Is an RTX 3060 with 12GB VRAM better than a newer model with only 8GB?
Does VRAM size matter more, or is speed just as important?
From what I understand, more VRAM helps run models with less quantization, but for quantized models, speed is more important. Am I right?
I couldn't find a clear answer online, so any help would be appreciated. Thanks!