home | hardware | software | misc | about | rss

OpenBSD 6.9 Router Benchmarks

21-May-2021

OpenBSD Routers

I've been using OpenBSD for my home router/gateway for years, and have run it on a number of different types of hardware. I recently got Verizon's gigabit service which is advertised at up to 940 Mb/s download and 880 Mb/s upload speeds, so I decided to benchmark some of the routers I have laying around. The contestants are:

All of these routers are running OpenBSD 6.9 and are set up exactly as described in the Building a Router FAQ page. There are lots of tweaks people do for performance, but I did not test any of them. What you see in the guide is what you get. CPU frequency was set at the maximum via apmd, where applicable.

Testing was done from a T14 running Ubuntu 20.04 with 3 runs at Speedtest.net using the Vultr speed test server. This is not perfect as it only tests web traffic, so smaller or larger packet sizes than what you find in web traffic are not tested, nor is UDP tested either. Since this is going to an external server outside my network, it also sends that traffic over other carriers, so some of this is at the mercy of the route. But this is not meant to be scientific, and probably represents how most people will use their router anyway.

Results are below, (averaged over the 3 tests):

The EdgeRouter Lite has a small ASIC onboard for packet forwarding which is used by Ubiquiti's EdgeOS. OpenBSD does not use this at all, so all of this was done by the two 500 MHz cores. One interesting thing is that the EdgeRouter 4 was signifcantly faster than the APU4, beating it by about 100 Mb/s. Despite the fact that both have four 1 GHz cores, the Octeon apparently moves packets around better than the AMD Jaguar. The fastest in the list is unsurprisingly the Xeon powered Supermicro E300, and it likely can go faster, but the connection may have been the limitation here.

I am currently using the Supermicro as my router, though my favorite device is the EdgeRouter 4 here for the best balance of speed and low power consumption. The E300's fans have a very whiny pitch to them that has made me relegate this machine to the basement. On the plus side, it has IPMI, and it is fast enough to be usable for other purposes.

Update: There was some criticism of using Speedtest.net as the test method, which I expected and I agree with it. I ran the following tests with iperf3 to see if there would be a difference. I did not test the Supermicro in this manner. iperf3 was running on the Supermicro in server mode, and the T14 laptop was connecting through the tested router to the iperf3 server on the Supermicro for all of these tests. These were with iperf3's default options.

I also tried with and without the scrub and antispoof lines in pf.conf to see if that would make any difference. See below for the results:

EdgeRouter Lite with scrub/antispoof


Accepted connection from 192.168.1.99, port 51479
[  5] local 192.168.1.1 port 5201 connected to 192.168.1.99 port 50181
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  12.9 MBytes   108 Mbits/sec                  
[  5]   1.00-2.00   sec  14.8 MBytes   124 Mbits/sec                  
[  5]   2.00-3.00   sec  15.1 MBytes   127 Mbits/sec                  
[  5]   3.00-4.00   sec  15.5 MBytes   130 Mbits/sec                  
[  5]   4.00-5.00   sec  15.6 MBytes   131 Mbits/sec                  
[  5]   5.00-6.00   sec  15.1 MBytes   126 Mbits/sec                  
[  5]   6.00-7.00   sec  15.4 MBytes   129 Mbits/sec                  
[  5]   7.00-8.00   sec  14.7 MBytes   123 Mbits/sec                  
[  5]   8.00-9.00   sec  14.9 MBytes   125 Mbits/sec                  
[  5]   8.00-9.00   sec  14.9 MBytes   125 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-9.00   sec   142 MBytes   133 Mbits/sec                  receiver

EdgeRouter Lite without scrub/antispoof


Accepted connection from 192.168.1.99, port 62412
[  5] local 192.168.1.1 port 5201 connected to 192.168.1.99 port 51770
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  10.5 MBytes  87.8 Mbits/sec                  
[  5]   1.00-2.00   sec  15.6 MBytes   131 Mbits/sec                  
[  5]   2.00-3.00   sec  16.5 MBytes   139 Mbits/sec                  
[  5]   3.00-4.00   sec  16.6 MBytes   139 Mbits/sec                  
[  5]   4.00-5.00   sec  16.6 MBytes   139 Mbits/sec                  
[  5]   5.00-6.00   sec  17.3 MBytes   145 Mbits/sec                  
[  5]   6.00-7.00   sec  16.0 MBytes   134 Mbits/sec                  
[  5]   7.00-8.00   sec  18.2 MBytes   153 Mbits/sec                  
[  5]   8.00-9.00   sec  17.9 MBytes   150 Mbits/sec                  
[  5]   9.00-10.00  sec  17.0 MBytes   143 Mbits/sec                  
[  5]  10.00-10.01  sec   177 KBytes   132 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.01  sec   162 MBytes   136 Mbits/sec                  receiver

APU4 with scrub/antispoof


Accepted connection from 192.168.1.103, port 62063
[  5] local 192.168.1.1 port 5201 connected to 192.168.1.103 port 58644
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  28.4 MBytes   238 Mbits/sec                  
[  5]   1.00-2.00   sec  41.9 MBytes   351 Mbits/sec                  
[  5]   2.00-3.00   sec  47.4 MBytes   397 Mbits/sec                  
[  5]   3.00-4.00   sec  50.2 MBytes   421 Mbits/sec                  
[  5]   4.00-5.00   sec  51.6 MBytes   433 Mbits/sec                  
[  5]   5.00-6.00   sec  52.4 MBytes   439 Mbits/sec                  
[  5]   6.00-7.00   sec  53.3 MBytes   447 Mbits/sec                  
[  5]   7.00-8.00   sec  54.0 MBytes   453 Mbits/sec                  
[  5]   8.00-9.00   sec  54.3 MBytes   456 Mbits/sec                  
[  5]   9.00-10.00  sec  54.6 MBytes   458 Mbits/sec                  
[  5]  10.00-10.00  sec  64.2 KBytes   451 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.00  sec   488 MBytes   409 Mbits/sec                  receiver

APU4 without scrub/antispoof


Accepted connection from 192.168.1.103, port 60797
[  5] local 192.168.1.1 port 5201 connected to 192.168.1.103 port 64446
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  35.2 MBytes   295 Mbits/sec                  
[  5]   1.00-2.00   sec  45.1 MBytes   379 Mbits/sec                  
[  5]   2.00-3.00   sec  49.9 MBytes   419 Mbits/sec                  
[  5]   3.00-4.00   sec  51.6 MBytes   433 Mbits/sec                  
[  5]   4.00-5.00   sec  53.6 MBytes   450 Mbits/sec                  
[  5]   5.00-6.00   sec  52.4 MBytes   440 Mbits/sec                  
[  5]   6.00-7.00   sec  55.9 MBytes   469 Mbits/sec                  
[  5]   7.00-8.00   sec  56.6 MBytes   475 Mbits/sec                  
[  5]   8.00-9.00   sec  57.0 MBytes   478 Mbits/sec                  
[  5]   9.00-10.00  sec  57.3 MBytes   481 Mbits/sec                  
[  5]  10.00-10.00  sec   245 KBytes   543 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.00  sec   515 MBytes   432 Mbits/sec                  receiver

EdgeRouter 4 with scrub/antispoof


Accepted connection from 192.168.1.97, port 57379
[  5] local 192.168.1.1 port 5201 connected to 192.168.1.97 port 54255
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  26.4 MBytes   222 Mbits/sec                  
[  5]   1.00-2.00   sec  42.8 MBytes   359 Mbits/sec                  
[  5]   2.00-3.00   sec  51.7 MBytes   434 Mbits/sec                  
[  5]   3.00-4.00   sec  56.2 MBytes   472 Mbits/sec                  
[  5]   4.00-5.00   sec  60.2 MBytes   505 Mbits/sec                  
[  5]   5.00-6.00   sec  62.5 MBytes   524 Mbits/sec                  
[  5]   6.00-7.00   sec  64.4 MBytes   540 Mbits/sec                  
[  5]   7.00-8.00   sec  65.5 MBytes   549 Mbits/sec                  
[  5]   8.00-9.00   sec  66.8 MBytes   560 Mbits/sec                  
[  5]   9.00-10.00  sec  66.4 MBytes   557 Mbits/sec                  
[  5]  10.00-10.00  sec   156 KBytes   515 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.00  sec   563 MBytes   472 Mbits/sec                  receiver

EdgeRouter 4 without scrub/antispoof


Accepted connection from 192.168.1.97, port 60739
[  5] local 192.168.1.1 port 5201 connected to 192.168.1.97 port 60797
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  25.0 MBytes   209 Mbits/sec                  
[  5]   1.00-2.00   sec  42.7 MBytes   358 Mbits/sec                  
[  5]   2.00-3.00   sec  54.5 MBytes   457 Mbits/sec                  
[  5]   3.00-4.00   sec  60.4 MBytes   507 Mbits/sec                  
[  5]   4.00-5.00   sec  64.7 MBytes   543 Mbits/sec                  
[  5]   5.00-6.00   sec  68.1 MBytes   571 Mbits/sec                  
[  5]   6.00-7.00   sec  69.3 MBytes   581 Mbits/sec                  
[  5]   7.00-8.00   sec  71.1 MBytes   596 Mbits/sec                  
[  5]   8.00-9.00   sec  72.2 MBytes   606 Mbits/sec                  
[  5]   9.00-10.00  sec  72.8 MBytes   611 Mbits/sec                  
[  5]  10.00-10.00  sec   158 KBytes   552 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.00  sec   601 MBytes   504 Mbits/sec                  receiver

I also measured the power consumption of each of these from the wall at idle and under load:

There was little discernible difference in power consumption between these, and they could be within the margin of error of the device I used to measure.

Update 11-June-2021: The APU4 was on an old BIOS when tested, which did not support boosting CPU core clock speed under load. This is a feature added in v4.9.0.2, and will boost to 1.4 GHz. I updated the BIOS to v4.13.06 and re-ran the tests on the APU4, which can be seen below (tested for 60 seconds with 6 second interval):


Connecting to host 192.168.1.1, port 5201
[  5] local 172.16.1.3 port 4631 connected to 192.168.1.1 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-6.00   sec   349 MBytes   488 Mbits/sec                  
[  5]   6.00-12.00  sec   422 MBytes   590 Mbits/sec                  
[  5]  12.00-18.00  sec   425 MBytes   594 Mbits/sec                  
[  5]  18.00-24.00  sec   425 MBytes   595 Mbits/sec                  
[  5]  24.00-30.00  sec   425 MBytes   595 Mbits/sec                  
[  5]  30.00-36.00  sec   425 MBytes   594 Mbits/sec                  
[  5]  36.00-42.00  sec   425 MBytes   595 Mbits/sec                  
[  5]  42.00-48.01  sec   427 MBytes   597 Mbits/sec                  
[  5]  48.01-54.01  sec   424 MBytes   592 Mbits/sec                  
[  5]  54.01-60.01  sec   425 MBytes   594 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-60.01  sec  4.08 GBytes   583 Mbits/sec                  sender
[  5]   0.00-60.17  sec  4.08 GBytes   582 Mbits/sec                  receiver

Without scrub and antispoof:


Connecting to host 192.168.1.1, port 5201
[  5] local 172.16.1.3 port 4336 connected to 192.168.1.1 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-6.00   sec   387 MBytes   541 Mbits/sec                  
[  5]   6.00-12.00  sec   476 MBytes   665 Mbits/sec                  
[  5]  12.00-18.00  sec   482 MBytes   675 Mbits/sec                  
[  5]  18.00-24.00  sec   484 MBytes   676 Mbits/sec                  
[  5]  24.00-30.00  sec   482 MBytes   675 Mbits/sec                  
[  5]  30.00-36.00  sec   483 MBytes   676 Mbits/sec                  
[  5]  36.00-42.00  sec   481 MBytes   672 Mbits/sec                  
[  5]  42.00-48.00  sec   480 MBytes   671 Mbits/sec                  
[  5]  48.00-54.00  sec   480 MBytes   670 Mbits/sec                  
[  5]  54.00-60.00  sec   469 MBytes   655 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-60.00  sec  4.59 GBytes   658 Mbits/sec                  sender
[  5]   0.00-60.14  sec  4.59 GBytes   656 Mbits/sec                  receiver