I've been playing around with self hosting for file sharing, backups, and a handful of other ideas I might one day get round to. I like the idea of a mesh VPN and being able to, for example, connect a travelling laptop to a 'host' laptop nearby, though my only public ip is a VPS in another country.
Of all the options I found, I liked the look of Nebula most. Fiddly in some places, but it's working nicely for me, and I appreciate some of the simplicity of design.
I'm wondering if people here have much experience of it, though? My biggest concern is over its future. With,
The Defined Networking site focusing on making money off it, and
The Android app doesn't allow full configuration (including the firewall, so I can't host a website from a phone) but - I heard - does if you use Defined Networking's paid service for configuration,
makes me worry they might be essentially trying to deprecate viable FOSS Nebula in favour of a paid or controlled service.
lots of services behind my router that connect to that VPN
router DNS to resolve my domains to my internal services when on my LAN
This gets me like 95% of the benefit of something like Nebula or Tailscale. When connecting to my internal services, I get LAN speeds if I'm on my LAN and WAN speeds if not. I initially started with Tailscale, but realized that I really didn't care about most of what it provided.
Not saying you should do it or that it is better overall, but ignoring those is not fair.
Personally i would never go for Tailscale since i give away the access control to my kingdom to a company.
Exactly what i want to get away from through selfhosting.
Exactly. I tried Tailscale to get things off the ground, but it didn't do precisely what I wanted, so I abandoned it and built exactly what I needed, which for me was a VPN at the gateway that tunneled SSL traffic via HAProxy to my internal network.
If Nebula solves your problems, great! I find I don't need its features, and prefer to keep things relatively simple, which for me is a WireGuard VPN and a handful of containers to run my things. My setup is basically HAProxy -> Wireguard VPN -> Caddy (TLS termination; docker container) -> Docker container on internal network. HAProxy routes to the appropriate machine, and Caddy renews TLS certs and routes to the appropriate container. I could probably accomplish the same w/ Nebula, but I understand my setup a bit more than Nebula.
What's an edge vps? Is that some sort of distributed cdn-style vps? Or just a VPS at the 'edge' of your network?
Biggest points for me of having a mesh, not a central Wireguard hub, are,
I have a VPS in one country, a 'host' laptop in a friend's house in another and a third laptop. I want the two laptops to connect directly to each other not bouncing all packets off the vps.
For backups, ssh, etc, I'd like to be able to just call the VPN IP, whether two machines are on the same LAN or not. Nebula/etc makes that work; a centralised VPN would sometimes be sending packets pointlessly out on WAN and back.
The latter, a VPS at the "edge" of my network. It doesn't run any services itself other than HAProxy, which just routes connections to services inside my network.
That use case makes a ton of sense.
I only have my VPS and internal devices, so using DNS names makes it trivial to always get the best route since the only options are within my LAN (simple router config) or over WAN. If it was any more complex, I'd probably do the same as you.
Is Headscale easier than Nebula? I thought it looked like it might become much more work.
Nebula was mostly easy, but had a few hurdles I needed to learn.
Setting up systemd. I think I had to look that up and write a startup thing for it. I might have copied one from Syncthing or something! I don't remember right now.
firewalls confused me a couple of times
and I had to get the hang of the certificate system of course
I have mixed feelings about trying Defined Networking's managed config, but I imagine that would get round the learning curve of the config.
I have been using Tailscale, connected it to my domain, I use Authentik for my OIDC/SSO Sign in and tied it that way for the MFA OIDC Login Tailscale let's you use. All I needed to do is setup a webfinger for it and once it verified my domain, I was able to give them my OIDC settings for them. Tailscale so far for me in the last year or so has been quite simple to use. Plus, being able to log into my admin console and any devices I enroll through Authentik's front end, has given me peace of mind knowing it's quite secure. (All of this on a Proxmox server BTW).
One may argue about self hosting Wireguard and I agree, it's quite easy to do if you use something like wg-easy which makes it simple to add phones to your network. My concern with it though was having to poke a hole into my firewall for the WG traffic to hit the server, once I got into Tailscale, it's made it easier and I don't have any open ports on the router now. I think this is primarily why the Jupiter Broadcasting guys push it so much on their podcasts, not to mention one of the hosts on his podcast is an employee for Tailscale as well, so that probably helps a bit.
As for funding for both Nebula, or Tailscale, they do cater to enterprise customers so you have the assurance that they do have to answer to them if they revoke a service or ruin it. :)
For Tailscale, it's just a matter of them allowing you to add 100 devices for free and it's simple command to install it on any client via the cli including Apple TV for example. For phones, I have Tailscale on my phone connected 24/7 to my exit node which is my Proxmox server which acts as one, and as a backup, my Raspberry Pi which acts as one as well. So, even if I'm on the road or away from home, I'm always on my home network (unless blocked by overzealous sysadmins on their public WiFi networks). There's not much to manage via the phone, but I like to think it's 'set and forget' really, once you have it all configured, it just runs in the background and they do not decrypt your traffic much less care what goes through it.
What made you choose Nebula over Tailscale? I'm running it through a self-hosted Headscale server and it's working well so far. I haven't looked into Nebula too much.
the core bits of nebula are all open source. With tailscale, there is headscale, but that is made by a tailscale employee and it feels ripe for a rug pull whenever tailscale feels like it. with nebula, the lighthouse and user clients are open, so there is far less chance of that.
I see. That is a valid concern. Though it feels unfair to say that headscale is 'made by a tailscale employee'. From what I understand, one of the main contributors of headscale was hired by tailscale, though he is not the only maintainer and does not own the repo from what I can tell. Still, Tailscale could decide to cede all support of headscale and that would likely hurt the project a lot. In the same way however nebula could decide to switch to proprietary licenses and discontinue their open source offerings.
Biggest thing was actually the sign up options. What if I don't want my machines calling to Google or Microsoft to get access to Tailscale? I need to look up the other OIDC providers but don't know much about that yet.
Then the fact of Nebula being fully open source and fully on my machines. (Though that's a little undercut by the Android problem being solved only by their managed service).
Headscale gave me an impression of being more complicated to set up and maintain. Haven't tried it yet, that was just my feel when I chose which one to try.
More recently, I saw Nebula's interesting post on performance benchmarks. At high throughout Tailscale can be better for CPU but heavier on memory. Hopefully at my sort of very low throughout it's small on memory but if I'm squeezing a client into a cheap vps alongside nextcloud and other things, memory use is more concerning to me than CPU... I wonder how much memory Tailscale uses when not doing much.
Can Wireguard to NAT traversal? Let's say I have a publicly facing server A and then two devices B and C behind two separate nats. Can B reach C directly via hole punching by A?
I don't know a lot about Tinc, but it looked to me like both Nebula (directly inspired by Tinc) and Tailscale solve problems Tinc has, and improve on its excellent but older design.
I think nebula is really cool and am heavily considering it in production.
Having a paid-for service that makes things easier is a good way to keep money going into the project, I think. And it feels a lot safer in terms of rug pull than tailscale/headscale. The android apps not being in fdroid and have some other limitations sucks... but I feel like those are easier to solve than some other issues that could be there.
If you want tailscale, but not tailscale, check out netbird. You can self host the auth server and it isn't some side project, the whole auth server is open.
I agree having a paid service, or some viable finance model, is a good sign for longevity ...that said Nebula is what Slack use themselves so publicly or privately it's going to be kept developed!
Just the fact the Android client is only properly configurable if you use their managed config service, made me worry a bit. Even though Tailscale you're signing up for more eggs in their basket (unless you use Headscale), it felt like at least you start out on that basis, you aren't pushed into it unexpectedly.
I do like that both projects talk politely about each other. That feels like a good sign for both!