Contents
This post is my efforts to pull together info around the cake queue discipline (qdisc) available in more recent Linux versions.
diffserv8
This is one of cake’s priority queue parameters, but one that is strangely lacking in documentation around what DSCP values fall into what “tins”. Using the other parameters, cake prints nice headings when running tc show, but not for diffserv8.
Like all things, the diffserv8 classifications can be found in the source code, but here’s a table:
| Tin number | DSCP name | Source description |
|---|---|---|
| 0 | CS1 | Background traffic |
| 1 | AF1x, TOS2 | High throughput |
| 2 | CS0 | Best effort |
| 3 | AF4x, AF3x, CS3 | Video streaming |
| 4 | AF2x, TOS4 | Low latency transactions |
| 5 | CS2, TOS1 | Interactive shell |
| 6 | EF, VA, CS5, CS4 | Minimum latency |
| 7 | CS6, CS7 | Network control |
cake and tunnels/VPNs
How does this all behave on outer and inner interfaces? For example, you have an eth1 interface that leads to the Internet, but you run a wg0 Wireguard tunnel over that too?
Well, apparently since Linux kernel 5.7, magic happens around hashing of flows in packet headers, before being encrypted. Linux patch here. To quote tohojo:
The packets are sorted into flows by hashing on the packet header. Wireguard will make sure this happens prior to encryption, and that the result (the hash) is kept with the packet even after it is encrypted.
Toke Høiland-Jørgensen
This functionality appears to also need another patch to work, which should be in the same kernel version.
A discussion around why Wireguard won’t propagate the diffserv marking (due to data leaking) can be found here.
That said, either I’m clueless, or this doesn’t appear to be in Debian kernel linux-image-5.10.0-8-amd64, because after I tried some tests, I didn’t see tc show counters on the outer interface go up.
The “average user” isn’t going to magically see traffic pre-encryption end up in tins other than the default on the outer qdisc.
In a Github issue, I asked for more feedback. Essentially, without a userspace BPF program run to set things up in the kernel, the “average user” isn’t going to magically see traffic pre-encryption end up in tins other than the default on the outer qdisc. You will only see diffserv-to-tins on the inner one that you might configure (and don’t need to if your setup meets the above conditions, though you still could.)
Note I mentioned the default tin; inner traffic will end up in the outer default tin (due to the hashing mentioned above, a la “best effort” mode), so you still get the benefits of per-flow queueing.
This behaviour also only works when running the qdisc in non “host fairness” mode. This eessentially means you need to use flows for the flow isolation parameter. To quote Toke again:
The critical parameter is that you select ‘flows’ as the flow mode. ‘flowblind’ turns off multi-queue entirely, and all of ‘srchost’, ‘dsthost’, ‘hosts’, ‘dual-srchost’, ‘dual-dsthost’ and ‘triple-isolate’ will do host-based hashing which is not compatible with preserving the hash from inside wireguard.
TOKE HØILAND-JØRGENSEN
Testing the tins
A useful feature of iputils-ping is that you can specify the TOS (not DSCP, TOS) value to use, which makes testing easy. Look here for a great table mapping TOS, DSCP etc all together.
For example, use ping -Q 192 to aim for diffserv8 tin 7 (CS6).
More to come
I’ll be adding more to this, if anything more comes to light, or I think might be useful in my cake journey.
Dave Taht
I really recommend against the diffserv8 (precedence) setting. It’s been obsolete since 2003. diffserv4 is saner in our videoconferencing world and is a closer match to wifi.The default of diffserv3 is pretty good, but a less close match.
Yes, the wireguard devs regarded passing the tos field through as a data leak.
And it’s the fair queuing and aqm that make most of the difference in cake. Twiddling diffserv bits doesn’t help much, in general.
What I see people doing is putting a cake shaper in front of the wireguard interface, often with a RTT setting that is closer to where the VPN is going.
Jason
Hi Dave! Honoured to have your thoughts.
Putting a cake shaper in front, on a two NIC box, using something like IFB? If one had a server, eth0 inside, eth1 outside, and wg0, I assume that it can’t go on wg0, but would have to be “outbound” from eth0?
I just did some basic testing, and assume that if cake is put on wg0, it would see “inside”, serving it’s purpose as you describe above — the only minor issue is the double-queuing.
Also as a separate question, based on the same topology above, shaping inbound can be done fine on eth0, right (instead of using IFB)? It seems to work fine for me, but I guess also limits *all* traffic from the server, inbound to the local network (including itself, ie. backups etc).
Kirk Brocas
Hi Jason, Australian user here on NBN. Wbuntu 22.04 LTS laptop – is cake for use on router only – ie. is there benefit of using tc-cake on a laptop/desktop especially when using wifi?
Jason
Heya Kirk, thanks for dropping by!
An interesting question; I guess there’s nothing stopping you from doing it, you will just get the most benefit by doing it at the “choke” point of your bufferbloated service. I suspect any ECN would probably benefit more there too. It would probably mostly work, if that’s the only device using the NBN link.