Description
Running performance tests with macOS 15.1, 15.1.1, and 15.2 show 35% regression compared with 14.7.1.
host to vm
vz
% iperf3-darwin -c 192.168.105.12 -l 1m
Connecting to host 192.168.105.12, port 5201
[ 5] local 192.168.105.1 port 51210 connected to 192.168.105.12 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd RTT
[ 5] 0.00-1.00 sec 271 MBytes 2.27 Gbits/sec 0 8.00 MBytes 14ms
[ 5] 1.00-2.00 sec 265 MBytes 2.22 Gbits/sec 0 8.00 MBytes 15ms
[ 5] 2.00-3.00 sec 268 MBytes 2.25 Gbits/sec 0 8.00 MBytes 16ms
[ 5] 3.00-4.00 sec 271 MBytes 2.28 Gbits/sec 0 8.00 MBytes 13ms
[ 5] 4.00-5.00 sec 272 MBytes 2.28 Gbits/sec 0 8.00 MBytes 14ms
[ 5] 5.00-6.00 sec 270 MBytes 2.26 Gbits/sec 0 8.00 MBytes 15ms
[ 5] 6.00-7.00 sec 271 MBytes 2.27 Gbits/sec 0 8.00 MBytes 14ms
[ 5] 7.00-8.00 sec 271 MBytes 2.27 Gbits/sec 0 8.00 MBytes 15ms
[ 5] 8.00-9.00 sec 272 MBytes 2.28 Gbits/sec 0 8.00 MBytes 14ms
[ 5] 9.00-10.00 sec 268 MBytes 2.25 Gbits/sec 0 8.00 MBytes 14ms
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 2.63 GBytes 2.26 Gbits/sec 0 sender
[ 5] 0.00-10.23 sec 2.63 GBytes 2.21 Gbits/sec receiver
Similar run on macOS 14.7, copied from #58:
% caffeinate -d iperf3-darwin -c 192.168.105.58 -l 1m -t 10
Connecting to host 192.168.105.58, port 5201
[ 5] local 192.168.105.1 port 60990 connected to 192.168.105.58 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd RTT
[ 5] 0.00-1.00 sec 460 MBytes 3.86 Gbits/sec 0 8.00 MBytes 9ms
[ 5] 1.00-2.00 sec 421 MBytes 3.53 Gbits/sec 0 8.00 MBytes 9ms
[ 5] 2.00-3.00 sec 435 MBytes 3.65 Gbits/sec 0 8.00 MBytes 10ms
[ 5] 3.00-4.00 sec 411 MBytes 3.45 Gbits/sec 0 8.00 MBytes 14ms
[ 5] 4.00-5.00 sec 317 MBytes 2.66 Gbits/sec 0 8.00 MBytes 9ms
[ 5] 5.00-6.00 sec 430 MBytes 3.61 Gbits/sec 0 8.00 MBytes 9ms
[ 5] 6.00-7.00 sec 423 MBytes 3.55 Gbits/sec 0 8.00 MBytes 9ms
[ 5] 7.00-8.00 sec 433 MBytes 3.63 Gbits/sec 0 8.00 MBytes 10ms
[ 5] 8.00-9.00 sec 437 MBytes 3.67 Gbits/sec 0 8.00 MBytes 9ms
[ 5] 9.00-10.00 sec 430 MBytes 3.61 Gbits/sec 0 8.00 MBytes 9ms
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 4.10 GBytes 3.52 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 4.10 GBytes 3.52 Gbits/sec receiver
qemu
For reference, same test with qemu instead of vz on macOS 15.2:
% iperf3-darwin -c 192.168.105.13 -l 1m
Connecting to host 192.168.105.13, port 5201
[ 5] local 192.168.105.1 port 51254 connected to 192.168.105.13 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd RTT
[ 5] 0.00-1.00 sec 205 MBytes 1.72 Gbits/sec 0 6.36 MBytes 15ms
[ 5] 1.00-2.00 sec 206 MBytes 1.73 Gbits/sec 0 6.36 MBytes 15ms
[ 5] 2.00-3.00 sec 209 MBytes 1.75 Gbits/sec 0 6.36 MBytes 16ms
[ 5] 3.00-4.00 sec 211 MBytes 1.77 Gbits/sec 0 6.36 MBytes 15ms
[ 5] 4.00-5.00 sec 204 MBytes 1.71 Gbits/sec 0 6.36 MBytes 15ms
[ 5] 5.00-6.00 sec 196 MBytes 1.64 Gbits/sec 0 6.36 MBytes 15ms
[ 5] 6.00-7.00 sec 203 MBytes 1.70 Gbits/sec 0 6.36 MBytes 15ms
[ 5] 7.00-8.00 sec 203 MBytes 1.70 Gbits/sec 0 6.36 MBytes 13ms
[ 5] 8.00-9.00 sec 207 MBytes 1.74 Gbits/sec 0 6.36 MBytes 19ms
[ 5] 9.00-10.00 sec 205 MBytes 1.72 Gbits/sec 0 6.36 MBytes 15ms
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 2.00 GBytes 1.72 Gbits/sec 0 sender
[ 5] 0.00-10.01 sec 2.00 GBytes 1.72 Gbits/sec receiver
I don't have qemu results from macOS 14.7.
vm to vm
macOS 15.2:
$ iperf3 -c 192.168.105.12 -l 1m
Connecting to host 192.168.105.12, port 5201
[ 5] local 192.168.105.14 port 45308 connected to 192.168.105.12 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 141 MBytes 1.18 Gbits/sec 0 3.91 MBytes
[ 5] 1.00-2.00 sec 141 MBytes 1.18 Gbits/sec 0 3.91 MBytes
[ 5] 2.00-3.00 sec 135 MBytes 1.13 Gbits/sec 0 3.91 MBytes
[ 5] 3.00-4.00 sec 135 MBytes 1.13 Gbits/sec 0 3.91 MBytes
[ 5] 4.00-5.00 sec 142 MBytes 1.19 Gbits/sec 0 3.91 MBytes
[ 5] 5.00-6.00 sec 134 MBytes 1.12 Gbits/sec 0 3.91 MBytes
[ 5] 6.00-7.00 sec 127 MBytes 1.07 Gbits/sec 0 3.91 MBytes
[ 5] 7.00-8.00 sec 139 MBytes 1.16 Gbits/sec 0 3.91 MBytes
[ 5] 8.00-9.00 sec 137 MBytes 1.15 Gbits/sec 0 3.91 MBytes
[ 5] 9.00-10.00 sec 139 MBytes 1.17 Gbits/sec 0 3.91 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 1.34 GBytes 1.15 Gbits/sec 0 sender
[ 5] 0.00-10.02 sec 1.34 GBytes 1.15 Gbits/sec receiver
I don't have results from 14.7, but I remember if was around 2 Gbits/s.
More info
Tested with both launchd service and by running socket_vment manually in a shell, so eliminate ProcessType regression in launched.
Tested on both managed mac with corporate VPN and content filters, and unmanned mac, both show the same regression.
Testing forwarding packets from vmnet using vfkit does not show any regression, so the issue is not in vmnet and likely not in socket_vment. It may be a performance regression in Go on macOS 15, affecting forwarding packets in lima.
I eliminated issues with not using the mac address assigned by vment, by testing vfkit with vment using both assigned mac addreess and unrelated mac address. Both show the same performance with iperf3.
I think we need a benchmark tool sending and receiving qemu packets to socket_vment unix socket. This will make it easy to tell if a performance regression is in socket_vment, lima, or qemu.