Ok, i will share my xperiences here but i couldnt provide any graphical/statical benchmarks
i tried nginx, lighttpd and litespeed these monthes as my major download webserver which handles large static files larger than 200MB and more than 1k concurrent connetion average, some times upto 2k at peak
nginx is very well SMP implemented webserver with awesome bw throttle features, you can set max concurrent connection limits per each IP globally or at VHost or for any path/file type individual using PCRE conditions
nginx is suitable for smaller files about 10~20 MB but for larger files more than 100MB it gets in IO bottlenecks with my huge connections at peak
100% iowait for all cores which leads to high load and forces server to very lower throughput
litespeed is better than nginx for large static files, lower iowait and well balanced between cores ( depends on your license ), but the bad thing with litespeed is max concurrent connetions which is available only globally, not in VHost levels or more depth
lighttpd, super webserver for static contents, specially when compiled with LOCAL BUFFERING enabled in src\network_writev.c and lib async io
it rocks! for me it can handle 2x throughput againts litespeed/nginx without any iowait
iowait is almost 0 with that huge file sizes and awesome connetions
currently im using lighttpd for all my download servers and for two servers im pushing upto 900Mbit/s at peak while server load is less than 1
I strongly recommend LS developers to take an action about this, specially for AIO
Check this out:
http://litespeedtech.com/support/wiki/doku.php?id=litespeed:wiki:linux_kernel_tune
or take a look here:
http://litespeedtech.com/support/forum/showthread.php?t=2257
all tricks help you tuning your server IOWAIT magically