[ENet-discuss] A unified rate-based throttle for reliable and unreliable packets

[Jim Purbrick]

Hi Lee,

You're hot on my heels again ;-) I sometimes wonder whether I should just go
on holiday and pick up your work when I get back :-)

I'm currently just finishing off my rate based version of ENet at the
moment. It's based on the Rate Adapting Protocol (RAP), a TCP friendly rate
based protocol. TCP friendliness is very important as otherwise you're
either going to make TCP users very upset or routers are going to drop your
packets for not playing fair.

The main wrinkle in using RAP is that it presumes a constant stream of
reliable data (video etc.) while game data is bursty and optionally
unreliable, so I'm having to extend RAP to support these features. The basic
RAP approach is to increase sending rate at 1 MTU per round-trip until a
packet loss is detected via a resend timeout at which point the rate is
halved. This additive increase/multiplicative decrease (AIMD) is the key to
TCP friendliness. The problem with sending unreliable data is that you don't
get any resend timeouts, so you never decrease the sending rate and it just
climbs forever. My current approach is to piggyback pings with unreliable
packets so that I send at least N reliable packets per RTT, which means I
get some feedback from the network and so can adjust the rate.

What to do with excess unreliable data is also a sticky point. On the one
hand if you let it back up then the application can respond to the back
pressure and reduce its data rate (adapting to the network conditions). On
the other hand just binning unreliable messages is simpler. A middle ground
would be to bin unreliable messages, but record the discard rate so the
application can adapt if it wants to. I'm considering adding a time to live
to both reliable and unreliable messages and binning messages if this is
exceeded before I send the data. I use a weighted fair queue to allow for
different message priorities, so low priority messages might have been
hanging around for a while before I send them.

Cheers,

Jim.