This post is my efforts to pull together info around the cake queue discipline (qdisc) available in more recent Linux versions.

diffserv8

This is one of cake’s priority queue parameters, but one that is strangely lacking in documentation around what DSCP values fall into what “tins”. Using the other parameters, cake prints nice headings when running tc show, but not for diffserv8.

Like all things, the diffserv8 classifications can be found in the source code, but here’s a table:

Tin numberDSCP nameSource description
0CS1Background traffic
1AF1x, TOS2High throughput
2CS0Best effort
3AF4x, AF3x, CS3Video streaming
4AF2x, TOS4Low latency transactions
5CS2, TOS1Interactive shell
6EF, VA, CS5, CS4Minimum latency
7CS6, CS7Network control
cake tin numbers to DSCP in diffserv8 mode

cake and tunnels/VPNs

How does this all behave on outer and inner interfaces? For example, you have an eth1 interface that leads to the Internet, but you run a wg0 Wireguard tunnel over that too?

Well, apparently since Linux kernel 5.7, magic happens around hashing of flows in packet headers, before being encrypted. Linux patch here. To quote tohojo:

The packets are sorted into flows by hashing on the packet header. Wireguard will make sure this happens prior to encryption, and that the result (the hash) is kept with the packet even after it is encrypted.

Toke Høiland-Jørgensen

This functionality appears to also need another patch to work, which should be in the same kernel version.

A discussion around why Wireguard won’t propagate the diffserv marking (due to data leaking) can be found here.

That said, either I’m clueless, or this doesn’t appear to be in Debian kernel linux-image-5.10.0-8-amd64, because after I tried some tests, I didn’t see tc show counters on the outer interface go up.

The “average user” isn’t going to magically see traffic pre-encryption end up in tins other than the default on the outer qdisc.

In a Github issue, I asked for more feedback. Essentially, without a userspace BPF program run to set things up in the kernel, the “average user” isn’t going to magically see traffic pre-encryption end up in tins other than the default on the outer qdisc. You will only see diffserv-to-tins on the inner one that you might configure (and don’t need to if your setup meets the above conditions, though you still could.)

Note I mentioned the default tin; inner traffic will end up in the outer default tin (due to the hashing mentioned above, a la “best effort” mode), so you still get the benefits of per-flow queueing.

This behaviour also only works when running the qdisc in non “host fairness” mode. This eessentially means you need to use flows for the flow isolation parameter. To quote Toke again:

The critical parameter is that you select ‘flows’ as the flow mode. ‘flowblind’ turns off multi-queue entirely, and all of ‘srchost’, ‘dsthost’, ‘hosts’, ‘dual-srchost’, ‘dual-dsthost’ and ‘triple-isolate’ will do host-based hashing which is not compatible with preserving the hash from inside wireguard.

TOKE HØILAND-JØRGENSEN

Testing the tins

A useful feature of iputils-ping is that you can specify the TOS (not DSCP, TOS) value to use, which makes testing easy. Look here for a great table mapping TOS, DSCP etc all together.

For example, use ping -Q 192 to aim for diffserv8 tin 7 (CS6).

More to come

I’ll be adding more to this, if anything more comes to light, or I think might be useful in my cake journey.