Tools like restic and Borg and so critical that you will regret not having had them sooner.
100000%
I just experienced this when a little mini PC I bought <2y ago cooked its nvme and died this month. Guess who has been meaning to set up backups on that guy for months?
Unfortunately, that nvme is D. E. D. And even more unfortunately, that had a few critical systems for the local network (like my network controller/DHCP). Thankfully it was mostly docker containers so the services came up pretty easy, but I lost my DBs so configs and data need to be replicated :(
The first task on the new box was figuring out and automating borg so this doesn’t happen again. I also set up backups via my new proxmox server, so my VMs won’t have that problem too.
Now to do the whole ‘actually testing the backups’ thing.
Goddamn, I was sure I ate the onion on this one.
This is probably the best possible outcome.