Congestion Control algorithms in UEK5 preview - try out BBR

One of the new features in UEK5 is a new TCP congestion control management algorithm called BBR (bottleneck bandwidth and round-trip propagation time). You can find very good papers here and here.

Linux supports a large variety of congestion control algorithms,  bic, cubic, westwood, hybla, vegas,  h-tcp, veno, etc..

Wikipedia has some good information on them : https://en.wikipedia.org/wiki/TCP_congestion_control

Here is a good overview of the important ones, including BBR : https://blog.apnic.net/2017/05/09/bbr-new-kid-tcp-block/

The default algorithm used, for quite some time now, is cubic (and this will remain the default also in UEK5). But we now also include support for BBR. BBR was added in the mainline Linux kernel version 4.9. UEK5 picked it up because we based the UEK5 tree on mainline 4.14. Remember we have our kernels on github for easy access and reading. We don't do tar files, you get the whole thing with changelog - standard upstream kernel git with backports, fixes, etc...

We have seen very promising performance improvements using bbr when downloading or uploading large files over the WAN. So for cloud computing usage and moving data from on-premises to cloud or the other way around, this might (in some situations) provide a bit of a performance boost. I've measured 10% in some tests. Your mileage may vary. It certainly should help when you have packet loss.

One advantage is that you don't need to have both source and target systems run this kernel. So to test out BBR you can run OL7 on either side and install uek5 on it (see here) and just enable it on that system. Try ssh or netperf or wget of a large(ish) file.

All you have to do is:

- use an Oracle Linux 7 install on one of the 2 servers.

- install the UEK5 preview kernel and boot into that one

- use sysctl (as root) to modify the settings / enable BBR. You can do this online. No reboot required.

You should also set the queue discipline to fq instead of pfifo_fast(default).

# sysctl -w net.ipv4.tcp_congestion_control=bbr # sysctl -w net.core.default_qdisc=fq

if you want to go back to the defaults:

# sysctl -w net.ipv4.tcp_congestion_control=cubic # sysctl -w net.core.default_qdisc=pfifo_fast

(feel free to experiment with switching pfifo_fast vs fq as well).

If need be, this can be set on an individual socket level in Linux. If you have a specific application (like a webserver or a data transfer program), use setsockopt(). Something like:

sock = socket(AF_INET, SOCK_STREAM, 0); sockfd = accept(sock, ...); strcpy(optval, "bbr"); optlen = strlen(optval); if (setsockopt(sockfd, IPPROTO_TCP, TCP_CONGESTION, optval, optlen) < 0) error("setsockopt(TCP_CONGESTION) failed");

or you should be able to do the same in Python starting in Python 3.6+.

sock.setsockopt(socket.IPPROTO_IP, socket.TCP_CONGESTION,...)

Have fun playing with it. Let me know if/when you see advantages as well.