Remember that guy who was fucking with and deleting my comments asking him to remove the commons clause from his license?
Sway 1.0-beta.2 is released!
I met a group of Japanese girls at a cafe a few minutes ago and sat down with them for a chat while I waited for my drink. Was able to chat effortlessly with zero confusion on either side. I'm pretty sure that's the first time that's ever happened. Words can't describe how happy I feel! Only took 2 and a half years of study 🎉
We are proud to have played our small part in InSight's #MarsLanding. Congratulations NASA!
Also, this exists now: https://status.sr.ht
Long post with details of recent outages for my services Show more
So, to clarify all the goings-on with my infrastructure
I had the primary database server for my private fleet suffer a RAID failure over the weekend. This server was also acting as a temporary build slave for sr.ht while I fixed issues running builds on the compute boxen. The outage affected several important sr.ht services, my mastodon instance, fosspay, packages.knightos.org, and a handful of other things.
I had to pull the server out of service, provision a temporary database server, and restore from backups (the only service to lose data was mastodon, and less than a day's worth). I also had to reprioritize "fix the issue with the primary builds.sr.ht slaves" and transfer build service to one of them - and in the coming days I'll spin up all 3 of the normal slaves. I also had to rebuild the base images, since those weren't backed up, and so far I've rebuilt everything but Debian, which I'll be doing tomorrow evening.
The malfunctioning server is still undergoing maintenance. I'll be reprovisioning it and reintroducing it to the fleet within a week or two, but I'm not in a rush since things are no longer on fire.
My current failure mitigation strategy is designed to preserve data integrity, but does not ensure high availability. I figured that preserving data integrity is important enough to do early, but HA is less of a priority. The implementation of this was proved correct this weekend, as no important data was lost. That being said, I'm prioritizing HA now, and will be deploying a psql failover, haproxy in front of all sr.ht services, a third DNS server, and an off-site backup box (on-site backups are already operational).
Sorry for the inconvenience!