How we monitor our services
@sir very nice read! It seems like it's low tech enough to be highly reliable (and I mean low-tech as in not too flashy). I keep wanting to play around with Prometheus and replicating some of the things you described might both be a good starting point and useful to keep an eye on the humble homelab.
The only thing I missed (and that may have been me missing it) is how you make sure you can rely on your Prometheus alerting. Is there a fallback of sorts in case that machine goes down?
@doenietzomoeilijk I set up a weekly test alarm, and a calendar reminder which reminds me that it has just gone off
The social network of the future: No ads, no corporate surveillance, ethical design, and decentralization! Own your data with Mastodon!