

I would recommend Unraid. Not sure what people think of it round here as surprised no one has mentioned it. My homelab was a mix of machines for VMs, Docker and NAS, and I consolidated it last year with Unraid and couldn’t be happier. I run Plex, Immich, Wordpress, Home Assistant and a load of other containers, alongside a Windows and Ubuntu VM on a cheap eBay HP Z workstation. If on a NUC with only a single drive, V7 of Unraid will now work without an array, so a single drive basically. It’ll give you a GUI for Docker and everything.
I think the thing with self hosting is that it’s a hobby, and when it goes wrong, it’s part of the hobby to figure it out. But in terms of business, then it becomes a risk. By all means try and use FOSS to improve solutions. I use a self hosted dropbox / file delivery to clients as it can saturate my 1Gbps fibre which is faster than most cloud file shares, but only because if it goes wrong one day, it’s a 2 min job to use a cloud solution instead (temporarily) and email clients with the alternative solution. But I would never build something up that only ever worked via one system.
Don’t just have data backups, have service backups. And in that regard, you may decide it’s just easier to do as others have said and use enterprise solutions from the start.
If using a self hosted Office suite, have all files duped into a single Google Drive account for example. That way you’re only paying for one Google account and have an emergency backup solution in place. EDIT: I’ve just recently degoogled and use Infomaniak in Europe for my office suite backup as its free for the 1st user. Experimenting with other non-Google/Microsoft solutions might be part of your journey.
You may decide the savings aren’t worth the effort in what you’re trying to achieve. EDIT: but I want to add that this is all part of the fun of what we do: thinking outside the box!