Alex on Nostr: The thing that tripped me up was persistent volumes. If you use local volumes your ...
The thing that tripped me up was persistent volumes. If you use local volumes your pods get pinned to the host. I tried Longhorn https://longhorn.io/ but it kept corrupting my data.
I also had frequent problems with nodes losing connection and then not being able to get them to rejoin the cluster.
Not to mention you're running a ton of code in Go for these operators and I was digging into their source code a lot more frequently than I would have liked.
Most problems are issues with running a multi-node system. So if I had to do it again I'd try doing it on one giant server. But there are simpler solutions.
FWIW, the client-server interaction is pretty nice, eg: https://gitlab.com/tribes-host/tribes/-/blob/develop/lib/tribes_deploy/k8s_resource.ex It's the internals of K8s itself I really struggled with.
I also had frequent problems with nodes losing connection and then not being able to get them to rejoin the cluster.
Not to mention you're running a ton of code in Go for these operators and I was digging into their source code a lot more frequently than I would have liked.
Most problems are issues with running a multi-node system. So if I had to do it again I'd try doing it on one giant server. But there are simpler solutions.
FWIW, the client-server interaction is pretty nice, eg: https://gitlab.com/tribes-host/tribes/-/blob/develop/lib/tribes_deploy/k8s_resource.ex It's the internals of K8s itself I really struggled with.