Host recovering from XFS failures Wednesday 30th June 2021 16:16:00

After a power outage at the data center the host is encountering XFS errors and is in process of recovering/repairing. Check out our blog post (in german) for more background information.

The host is up and running again :tada:

We are still syncing backups, so you might want to check if /var/www/virtual/username already contains what you expect.

We managed to fix a lot of issues with SE labels and PHP. Hopefully most things work for you now.

Just to clarify: the host is up, but still not fully functioning again. We are working on it, but it will take some more time (and reboots).

The host is back up again.

We still have some repair task and testing to do though. Meaning there could be more reboots and performance issues.

Syncing completed, but the host won't boot due to SELinux errors. We to avoid mistakes made due to sleep deprivation, we'll continue the process tomorrow.

backup is still transferring to the new VM

The host is re-created with a fresh deployment. Backups are currently copyed over (this will take some time).

we had to give up the system recovery and will set up a new VM with data backup on monday, please note the update in our blogpost.

we are still transferring data to the new block device and doing some tests

we are mounting a new block device to replace the VM, the syncing is in progress now