New LSWS vs. nginx vs. Apache Benchmarks! Feedback thread.

[Michael]

Howdy,

We've just released new benchmarks comparing LSWS 4.2.5, OpenLiteSpeed 1.2.6, nginx 1.4.3, Apache 2.4.6 with Event MPM, and Apache 2.2.25 with Pre-fork MPM: http://blog.litespeedtech.com/2013/...espeed-vs-apache-vs-nginx-for-static-content/

This is the first in a series of benchmarks that will cover many different facets of web service.

We're going to be using this thread for feedback. Tell us what you think!

 

[wanah]

Thanks ! What config were these run on ? What did the files contain ?

Edit : Sorry, just found the, full results !

Great to have some recent benchmarks.

 

[Michael]

Edit : Sorry, just found the, full results !

Great to have some recent benchmarks.

Glad you liked them. Let us know if you have feedback!

 

[eva2000]

Nice benchmarks !

Curious as you're using ApacheBench, which version of ApacheBench was used as i see your logs have ./ab_new ?? I ask as from my limited testing last year with ApacheBench, the version used can make a difference i.e ApacheBench included with Apache 2.4.x is faster than ApacheBench included in Apache 2.2.x for all web servers I tested, LiteSPeed, Apache and even Nginx http://vbtechsupport.com/1835/

Also, would be better to test at higher concurrency levels i.e. 250-500 and with Nginx 1.5.6 as well with more average/normal static file sizes

edit: Ah i see in the logs using ApacheBench, Version 2.3 <$Revision: 1430300 $> so from Apache 2.4.6 ?

edit2: Digging into config files, I noticed LSW vs OLS, the inMemBufSize differ with LSW = 120MB and OLS = 60MB ? Would make a difference ?

edit3: Also seems OLS static file gzip compression level was higher at 6 versus LSW at level 1.

-    <gzipCompressLevel>1</gzipCompressLevel>  
-    <compressibleTypes>text/*,application/x-javascript,application/javascript,application/xml</compressibleTypes>  
+    <gzipCompressLevel>6</gzipCompressLevel>  
+    <compressibleTypes>text/html</compressibleTypes>

 

[Michael]

Howdy Eva,

I responded to you on Twitter, but i figured I'd put it here too:

We're using ApacheBench 2.3 Revision 1430300.

It's interesting that different versions of ab come up with different results. (Maybe a faster version of ab would allow us to see the differences between servers more clearly...) But I figure it's not an issue as long as we're using the same version across all our tests.

m

 

[eva2000]

yeah was surprised different ApacheBench versions resulted in differences as well

Looking forward to more benchmarks from you folks - preferably at higher concurrency levels at which LiteSpeed starts to really show it's worth

 

[paul111]

paul

I was surprised different ApacheBench versions resulted in differences as well

 

[Michael]

Howdy,

Initially, I was surprised, but the more I've thought about it, the more it makes sense. If the code is different from one version to another, you'll see different amounts of drag caused by different versions of ab. This means a slower version of ab will cover up differences between web servers (as they all have added drag). With a faster version of ab, you get more pure results comparing only the speeds of the web servers. I'm not sure about how big the difference is between ab versions. Evangelion?

Michael