In my humble opinion the point of self hosting an LLM is so that the data doesn’t leave your LAN.
In my humble opinion the point of self hosting an LLM is so that the data doesn’t leave your LAN.
That’s going to be a lot of North Koreans fleeing to Ukraine
It’s as complex as you make it, is linux native, is scriptable, doesn’t use YAML, is native to the OS, and is free as in beer. Just like SNMP. however they’ll also get logs at a central server they can drill into if needed.
Which I believe fulfills the requirements of OPs post.
Sidenote, self-hosting is absolutely overkill just as a theory and process. I often read responses to suggestions as this or that is overkill, or complicated, or non-trivial effort.
The self hosting community is a broad spectrum of users , from those with home labs on an old dying laptop to those with a full rack setup. People have different needs and interests. Some are learning infra and devops for work or to get into a new job. Some are privacy minded. Some are trying to get the most bang for their buck. Some just want to pay for a cloud hosted solution. Some just want an automated home. Some run a home business.
Edit: to the point of your valid and helpful SNMP post, most syslog servers also will ingest and report on SNMP traffic as well. The container I linked does exactly that. If they find they want to automate processes in the future they can also trigger on the syslog stream as well. But that complexity is only there if they want it. Otherwise it’s just a stream they can parse and trigger an alert, just like SNMP. So OP could have an extensible solution if they want to expand. Also Grafana/Prometheus will take in syslog natively with a couple standard YAML configs if they choose that they want to look at that solution again in the future.
/Rant
Edit: found better resources
https://linuxhandbook.com/syslog-guide/
https://github.com/linuxserver/docker-syslog-ng
That should be a good place to start. Syslog will do what you want.
Agreed this is essentially a style guide.
Proxmox also can run on ZFS, has support to run containers, and can also manage backups.
If you’re always an immigrant and outsider, it sounds like borders are porous enough where you are that you’ll be alright.
Are you a Russian immigrant in Serbia, Hungary or Cyprus?
Third Navy at it again
We’re not handling the LLM generative bullshit bots now, anywhere. There’s a thing called the dead Internet theory. Essentially most of the traffic on the Internet now is bots.
https://en.m.wikipedia.org/wiki/Dead_Internet_theory