Things I like less in life: Servers locking up for hard to explain reasons. Somewhere between a weird Ceph object storage daemon, a coredump handler that freezes and general k8s madness lies the explanation why some nodes end up with a constant iowait, no responses to ps/htop and general madness unless rebooted. I can’t begin to explain how annoyed I am with this looming threat of this happening to a cluster that’s not only hosting internal tooling but real production services.