Made a quick test of mesh VPN clients. Test was performed between host and a VM, both running Kubuntu 23.04. VM ran on KVM with a virtio network adapter.
Test machine is oldish laptop with an i5-2540M, so VPN performance was probably CPU bound. Still, tests help to understand how different mesh VPNs compare against each other.
Tailscale surprisingly was the fastest, even faster than plain Wireguard, despite being userspace. But it also consumed more memory (245 MB after the iperf3 test!) and CPU.
Netbird’s CPU usage is so low I almost doubt if that’s fair comparison, most usage might be by kernel, since it uses kernel Wireguard. I don’t know how to measure that better. Memory usage is moderate. For some reason it wasn’t as fast as plain Wireguard.
Zerotier has the lowest memory usage, but is the slowest. Although this would probably only matter on LAN. Hope upcoming v2 closes the gap.
| Tailscale | Netbird | Zerotier | Wireguard | Raw | |
|---|---|---|---|---|---|
| Version | 1.48.2 | 0.23.3 | 1.12.2 | ||
| Idle, PSS, MB | 66 | 36 | 12 | ||
| iperf3, PSS, MB | 245 | 36 | 12 | ||
| Idle, CPU time, s / real minute | 0.505 | 0.120 | 0.297 | ||
| iperf3, CPU time, s / real minute | 115.23 | 0.14 | 78.72 | ||
| iperf3, Mb/s | 860 | 630 | 360 | 730 | 9600 |
Same table as an image with best / worst results highlighted: https://ibb.co/Gn2RRjb
Nice test! I’m a Tailscale user and I liked it being faster than others. I don’t care about memory usage but curious why there is a big gap 🤔 Like its using 20x more memory than Zerotier.
Tailscale is written in Go with lots of dependencies. It also has a lot more features, to the point some would call that too much 😅 Zerotier is pretty lean and in C. That would explain those 55 MB idle memory usage difference. But those 245 MB after iperf3 test though… I can’t explain, but it’s consistent and repeatable.
Is it possible the others are using the Wireguard kernel module? In that case, a lot of the memory usage will be in kernel/system memory, and just looking at the app’s memory usage won’t be the full story.
Netbird uses kernel Wireguard module, right. Is there a way to measure kernel memory / CPU usage attributed to Wireguard? Zerotier, which has the lowest memory usage, does not use Wireguard at all, they have their own custom protocol and it’s userspace AFAIK.
Not that I’m aware of, unfortunately.
I might just compare
freeoutput before / during / after iperf3 test. Will do that later today.