Interesting and useful data to know. I was wondering about the limits of ENet myself and I had just been about to do something similar to measure the max number of clients an Enet host can service without really starting to bog down.<br><br>I had a few questions.<br>1) Do you have data at other data points, for instance 100, 200, 300 clients and the CPU loading?<br>2) Or the effect of reliable data on the load?<br><br>Finally, is this benchmarking application something independent of your application? A general scalability test like this would be an useful addition to Enet. If required, I could help make your test a more general test harness for Enet.<br><br>-Madhan<br><br><b><i>Espen Overaae <minthos@gmail.com></i></b> wrote:<blockquote class="replbq" style="border-left: 2px solid rgb(16, 16, 255); margin-left: 5px; padding-left: 5px;"> I've been running some benchmarks to see how many clients I could get<br>a single server process to serve, and ran into some
interesting<br>bottlenecks.<br><br>I call enet_host_service 1000 times per second. I have 4 channels.<br>Each connected client sends a small packet every 100 ms, and gets<br>various amounts of data in return every 100 ms.<br><br>First the no connections and no traffic scenario:<br>With max peers at a low number like 64, cpu usage is 0%<br>With max peers at about 2 k, cpu usage is 2-3%<br>With max peers at about 8 k, cpu usage is about 9%, with most of it<br>being spent in this function:<br>72.74% - enet_protocol_dispatch_incoming_commands<br>Dropping the polling rate to 100 Hz reduces cpu usage to 1% for 8K max peers.<br><br><br>When I connect a bunch of peers and start sending data all over the place:<br><br>With 8 k max peers and 100 Hz polling rate, the server stays<br>responsive until about 2 k clients, and uses about 90% cpu.<br>Profiling shows nearly 25% of this time is spent in ENet:<br>12.83% - enet_protocol_dispatch_incoming_commands<br>9.31% -
enet_protocol_send_outgoing_commands<br><br>With 8 k max peers and 1 kHz polling rate, the server is more<br>responsive all over, but still only handles about 2 k clients, and cpu<br>usage rises to about 150% (the server is multithreaded and running on<br>a quad-core).<br>Profiling shows more than 50% of this time is spent in ENet, which<br>translates to about 80% cpu usage for the thread servicing ENet.<br>The big culprits are, according to gprof:<br>27.35% - enet_protocol_dispatch_incoming_commands<br>26.32% - enet_protocol_send_outgoing_commands.<br><br>Creating two server processes with 2 k max peers each and a 1 kHz<br>polling rate, allows me to connect a total of 3.5 k clients spread<br>over the two processes before the servers become unresponsive. CPU use<br>with two server processes is about 150% for each process, 40%<br>system(kernel - I guess this is the time spent inside system calls)<br>time, and only 5% idle (the remaining 55% probably spent in the
client<br>processes and other background processes).<br>Profiler still shows about 50% time spent in ENet:<br>29.43% - enet_protocol_dispatch_incoming_commands<br>18.00% - enet_protocol_send_outgoing_commands<br><br><br>These numbers do not show how much time is spent in system calls and<br>how much is spent in actual enet code, they only show the grand total<br>of time spent within the functions listed and all subfunctions. I<br>assume much of it is spent in ENet. Looking at the ENet code, I assume<br>increasing the number of channels would increase the cpu time spent in<br>ENet. The total throughput in these tests has been a few megabits per<br>second, most of it unreliable. Responsiveness is simply measured by<br>the percieved time it takes to connect a batch of 500 new clients and<br>seeing how many of them fail to connect.<br><br>The server processes did some computations on the data transmitted.<br>Previously I've done essentially nothing with them and the
profiler<br>showed enet to use a greater share of the cpu time, but the total<br>number of clients I could connect remained fairly constant. Even with<br>4 server processes and no computations, the servers got unresponsive<br>when the total client number approached 4 k.<br><br>To get useful profiling information from multiple threads, I used<br>this: http://sam.zoy.org/writings/programming/gprof.html<br><br><br>Espen Overaae<br>_______________________________________________<br>ENet-discuss mailing list<br>ENet-discuss@cubik.org<br>http://lists.cubik.org/mailman/listinfo/enet-discuss<br></blockquote><br>