I just wish we could invest the time/money/resources into compressing AI and making it smaller and more efficient. I’d so much rather have a somewhat capable AI that can be run locally and offline, to outsource menial tasks to like alphabetizing spreadsheets and so basic image modification, than to have to upgrade my hardware constantly or use cloud based SaaS and/or have newer models that are more accurate in their predictions.
Of course that assumes a lot of things, like the intent to help people and not make money. Maybe someone in the Linux-sphere will make something.
There are efforts there. The new Deepseek 4 compresses a lot of its knowledge using something they call engrams. But it’s unfortunately still too big for a consumer GPU.
Gemma 4 is small enough to run on your cellphone.
If your GPU has at least 8GB there are a lot of options for self hosting your own local models
I just wish we could invest the time/money/resources into compressing AI and making it smaller and more efficient. I’d so much rather have a somewhat capable AI that can be run locally and offline, to outsource menial tasks to like alphabetizing spreadsheets and so basic image modification, than to have to upgrade my hardware constantly or use cloud based SaaS and/or have newer models that are more accurate in their predictions.
Of course that assumes a lot of things, like the intent to help people and not make money. Maybe someone in the Linux-sphere will make something.
There are efforts there. The new Deepseek 4 compresses a lot of its knowledge using something they call engrams. But it’s unfortunately still too big for a consumer GPU.
Gemma 4 is small enough to run on your cellphone.
If your GPU has at least 8GB there are a lot of options for self hosting your own local models