[ENet-discuss] Prioritisation

[Jim Purbrick]

> Well, there is some synergy between reliable and unreliable packets. I
> use reliable packets to measure the RTT (and its variance) of 
> the connection, 
> and send out pings regularly to make sure this information is always
> being collected periodically. 

This would stay.

> Abnormal RTTs (either too high 
> or too low)
> cause the throttle to move down or up, as appropriate.

Packet loss would affect send rate as per TCP friendly algorithm.

> And the throttle
> basically just tells how many packets to drop. I try to send the
> unreliable packets out ASAP or drop them, so I basically trade latency
> for packet loss. Given what ENet was written for, the latency was the
> way more critical issue.

My plan would be to keep sending rate and lifetime orthogonal. As well as
having a per-message priority you would have a time to live for unreliable
packets. When a packet is taken out of the priority queue to send it's time
to live is compared with the current time and, if stale, the packet is
dropped before sending.

This can result in a build up of messages, but this allows the client to
respond to back pressure. A maximum time to live for unreliable packets
would avoid chronic message build up and latency. A point would be reached
where all unreliable data would be dropped, as with the current ENet.
 
> Now, the reliable packets just use a window, which is 
> probably far from
> ideal. But on the other hand, it "magically" works with the unreliable
> throttling as lots of reliable traffic will cause the RTTs to 
> go up and
> the throttle will start dropping more unreliable packets in response.

With a rate based approach packet loss due to internet conjestion would
result in sending rate lowering in a TCP friendly way, avoiding conjestive
collapse. With ENet I think what happens is that routers will drop packets,
causing them to be resent and more unreliable packets to be dropped. As the
resends back off exponentially and the amount of unreliable data goes down,
ENet responds in the correct way, as the data rate goes down, but I'm not
sure if it's TCP friendly.
 
> Now, in an ideal world, ENet would probably not work by "magic" and
> would use some sort of uniform rate based approach, although 
> I am not so
> sure it would help the unreliable packet situation and would require
> either require more info about connection bandwidth or just some new
> "voodoo" to automatically determine some ideal rate to send 
> data out at.

You probe for network bandwidth in a similar way to TCP. You increase data
rate until you encounter packet loss caused by routers reporting conjestion,
then slow down the data rate in response to the packet loss. By additively
increasing data rate, but exponentially decreasing it you ensure that
conjestive collapse is avoided.

> In any case a possible start would be to have some good 
> metric of how much
> data in total is being sent out over a given time period, and have the
> throttle attack that, rather than merely unreliable packets. 

Yep.

> And I don't
> know if I would even want to change how unreliable packets 
> are handled,
> so much as making something more flexible than the window scheme for
> reliable data. 

Don't do anything drastic with ENet. I'll just tinker away and keep
badgering you with questions and I'll let you know how I get on.

> Then you start getting into the land of Quality of
> Service --- UGH! 

Real QoS can't easily be done over the internet as it's a best effort
system. You need to be able to reserve bandwidth. What I plan to do is use
prioritisation to provide Class of Service, so I can ensure that high
priority traffic gets 80% of my bandwidth - even though the bandwidth might
constantly change!

> A choice between magic or voodoo, hmm? :)

Not really. A choice between TCP-like sliding windows, used to control a
rate based unreliable service, or a pure rate based system using packet loss
detection to back off sending rate in the face off conjestion.

Both eminently do-able.