That TP-link is a dumb switch. Unless you’re telling me that someone is going to find an opening in the firmware and hack their way into the ARP table or something (in which case the threat model here just became state actors and I don’t think the OP is safe with this equipment), I don’t think it affects much, if anything.

Now, if I’m mistaken and that is actually a managed switch; god help them with network security.

I didn’t know libvirt supported HA

I suppose RoCE/iWARP were what I was asking for

I read a bit more and I’d like to add:

RoCE/iWARP is the technology with which one would be able to route DMA over the network. The bandwidth of the network is the bottleneck but we’ll ignore that for now.

SR-IOV is a way to share virtual functions of PCIe devices on the same host.

Regardless of whether one uses IB or iWARP, they can also route data to and from a PCIe device attached to a host to another host over the network. I still have to research the specifics but I’m now positive that it can be done.

Thanks

Well, I’m not a systems engineer, so I probably don’t understand the scale of something like this.

With that said, is it really hard to slap TCP/IP on top of SR-IOV? That is literally what I wanted to know, and I thought RDMA could do that. Can it not?

Is there a PCIEoE?

I’m fairly sure there’s a way to provide compatible PCIe devices over IP on a network, or “some network” (if you’re bypassing the IP stack, perhaps). I just don’t know what it’s called, and I’m getting more confused by whether RDMA support can do this or not. Essentially, I want to leverage what SR-IOV allows me to do (create virtual functions of eligible PCIe devices) and pass them over IP or some other network tech to VMs/CTs on a different physical host.

It seems I have gaps in my understanding. I had assumed that SR-IOV allowed me to “break” PCIe devices (with firmware that supports it) into virtual functions (“slices”), to then be passed through to VMs/used by containers like physical devices.

You’re right, in that I didn’t really see a mention of TCP/IP in the blogs I’ve read about RDMA. I understand what it is but unless I can access host memory by bypassing the kernel on other machines on the network, this isn’t something I need to consider.

I think virtual functions for compatible PCIe devices is chugging along well in the Linux kernel: check videos about the Nvidia P4 sliced into virtual functions and passed through to different VMs using KVM. It’s either that or I’m completely missing the point somewhere.

So it is RDMA.

Indeed, I have come across RoCE, and support seems to be quite active on Debian. I was looking at QLogic hardware for this, and whilst I know that firmware for such stuff is really difficult to find, I’m fine with just FOSS support on Debian

I don’t think they assume an “advanced” knowledge of Docker. They do require an investment in time and a willingness to play around though. I’d like for you to point me to said guide if you have it around, I’ll attempt to explain if there’s something you’d like more clarity (although TBH you don’t even need humans to explain things to you these days. It’s such a convenience).

Unfortunately for you OP, you’re going to have to become at least decent at networking. The good part is that it will happen naturally as you learn, break and re-do your homelab.

Incidentally, I’m interested in any guides you might have regarding CPU performance metrics and cache. If you can recollect where you got them from.

So, essentially you want a File server and a media server, yes?

I think the parts to something like this would be:

1. DNS, so you don’t have to remember IP addresses. Most people use Pi-Hole/Adguard home, but the idea is the same.
2. A platform to run your services. A lot of people run containers bare-metal, but a lot of people also run their containers on top of VMs using a hypervisor. Some people just run VMs. For a beginner, containers will be a little more of a learning curve compared to plain VMs but it will pay off very soon. I tend to suggest Podman since it’s binary compatible with Docker and I like it more philosophically.
3. Some way to acquire media. This is where the payoff really starts showing up since you’re running about 10 containers or so already. Imagine running 10 VMs on restricted hardware. If you’re going to torrent, I’d suggest looking up Sonarr and going from there.
4. Actual media and file servers. Since you like GUIs, try Jellyfin and (this is not a favourable recommendation but here we go) Nextcloud.

Have fun!

Could you let me know about the problems you’re facing? I’ll try to help if I have ever heard/used the service before

At which point such an user might already be looking at TrueNAS/DIY setups TBH

Is it your main firewall?

Do you do in-place upgrades, and you do have HA for your firewall?

Intel SGX requires for me to set a CPU flag in the .conf file. For now, it’s a shell script and I can do it with Ansible, but I’d like to not have to do such half-baked measures

Are you saying that you haven’t heard of media company affiliates prowling in such groups from invites given to them?

You’re not paying for the content, you’re paying for the server upkeep. I assume you don’t donate to people who rip your favourite albums/films either or to people who seed a lot.

You could just physical if you wanted to not pay a few dollars to piracy services, but those will be more expensive

Can’t cloud-init only really run scripts in the guest and not on the host?

I was under the impression that cloud-init could only really be used to run commands inside the guest? Well, I could technically use Ansible and edit the file every time I provision something - this was just an example of however much the community tries, there might be something missing in the provider because proxmox doesn’t take this on directly.

I should have worded that better. In using MAC, AppArmor effectively reduces access to files that would be essential for the VM to run. That is the sense in which I mentioned “security enclave” but I can see now that that isn’t quite correct.

Either way, that is my philosophical reasoning for complaining this much. Ansible is pretty decent and has decent Proxmox integration, but Terraform is, in my opinion, superior when it comes to deploying infrastructure. That might be a bias from my side, of course. For now, I’m also going through the OpenStack documentation to see if the things I want to achieve can be done there, because they have an official Ansible project alongside their version of Cloudformation - Heat.

Thanks

Could you tell me which cloud providers are using Incus?

It’s a bit hard to search info about it with the name. But it’s a fantastic project