With the release of Kimi 3, which looks to be roughly on par with Fable/Sol, I was thinking about how long until I’ll be able to run a model with similar ability at home fully on my 32 GB GPU.
The equivalency here is based on the Artificial Analysis Intelligence Index, and assumes there is negligible quality loss running an FP8/FP4 quant, which seems to be the consensus these days. We also don’t really know how big some of those closed-source models are; those are just best guesses.
It’s crazy that only 8 months after GPT-5 we have a model that you can run at home with better performance on the benchmarks. Benchmarks aren’t the whole story, and models can be overfit to them, so sometimes your experience with the model doesn’t really line up with what they’re saying. But from my own experience using it for coding tasks, Qwen3.6 27B feels like using Claude from late 2025.
So when will I be able to run a Fable-class model at home? If we keep going at the current pace, it could be around a year from now. However, I think the size of the model may be a requirement for that general intelligence with our current model architectures. I think there will likely be specialised smaller models that achieve Fable-level benchmarks in one area, like coding or medicine (Qwen3.6 is focused more on coding and agent ability). It seems like we waste a lot of space in these models on things that aren’t really relevant to the main task. If I’m just using it for coding, does it need to know about medicine? I think this is why mixture-of-experts models are popular right now, but you still need all the weights resident on the GPU even if only a small subset are used.
It seems like we still have a long way to go on efficient model architectures. How much energy does it take to run the best mathematician in the world? A couple of meals a day? How much does it cost to run Kimi 3 for a day? A lot more than that. Maybe they’re not comparable?