[ENet-discuss] Incorrectly reported connection under heavy load

[Krasimir Marinov]

I've been load testing an application, mainly based on ENet, and noticed
erroneous behaviour for some of the connection being initiated.

The test is performed on localhost and perform ~60K connections sending ~
120K messages.

The error is that initiating connection to let's say port X I got
confirmation (ENet connected callback) for port Y.
Here are log traces that visualise the sequence of events under which the
behaviour is observed:

1. Correct connect:
host.c:enet_host_connect:250: peer 0x7f4e604d1a10: connecting to port 40041
protocol.c:enet_protocol_handle_incoming_commands:1078: peer
0x7f4e604d1a10: port 40041
protocol.c:enet_protocol_handle_incoming_commands:1127: peer
0x7f4e604d1a10: command ENET_PROTOCOL_COMMAND_VERIFY_CONNECT
>>>2f00000000000000000000000000000000000000000000000000000000000000<00000000>:(conn
0x7f4dc8030190, peer 0x7f4e604d1a10): connected to port 40041

2. Incorrect connect:
protocol.c:enet_protocol_handle_incoming_commands:1078: peer
0x7f4e604d1a10: port 48418
protocol.c:enet_protocol_handle_incoming_commands:1133: peer
0x7f4e604d1a10: command ENET_PROTOCOL_COMMAND_DISCONNECT
peer.c:enet_peer_reset:372: peer 0x7f4e604d1a10
protocol.c:enet_protocol_handle_incoming_commands:1078: peer
0x7f4e604d1a10: port 56265
protocol.c:enet_protocol_handle_incoming_commands:1112: peer
0x7f4e604d1a10: command ENET_PROTOCOL_COMMAND_ACKNOWLEDGE
protocol.c:enet_protocol_handle_incoming_commands:1145: peer
0x7f4e604d1a10: command ENET_PROTOCOL_COMMAND_SEND_RELIABLE
>>>2f00000000000000000000000000000000000000000000000000000000000000<00000000>:(conn
0x7f4dc800ac00, peer 0x7f4e604d1a10): connected to port 56265

What happens, in 2., is that a peer that I've just created to connect
receives an "unexpected" disconnect message and resets the peer,
peer.c:enet_peer_reset:372: peer 0x7f4e604d1a10, which doesn't clear the
user supplied data (ENetPeer->data) and when the peer is reused I get
incorrect data attached to it.

I've applied the following patch to protocol.c, which seems to work.
Unfortunately I'm unable to predict all positive/negative consequences of
it, so I'm asking for another opinion/ideas :). Here is the patch (simply
skip processing disconnect messages when in state CONNECTING):

protocol.c
817c817,818
<     if (peer -> state == ENET_PEER_STATE_DISCONNECTED || peer -> state ==
ENET_PEER_STATE_ZOMBIE || peer -> state ==
ENET_PEER_STATE_ACKNOWLEDGING_DISCONNECT)
---
>     if (peer -> state == ENET_PEER_STATE_DISCONNECTED || peer -> state ==
ENET_PEER_STATE_ZOMBIE || peer -> state ==
ENET_PEER_STATE_ACKNOWLEDGING_DISCONNECT
>         || peer->state == ENET_PEER_STATE_CONNECTING)


Regards,
Krasi
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.cubik.org/pipermail/enet-discuss/attachments/20131127/60196e0a/attachment.html>