1-2 minute no response from server

[inxile]

In the logs there are NUMEROUS entries like this when the server doesnt seem to respond for a minute or two:

"No request delivery notification has been received from LSAPI process:5600, possible run away process.
Retry with new instance."

What is going on there? Which config settings are associated with this activity and what could I check?

Our SQL has never shown an issue.

 

[webizen]

see if this helps

http://www.litespeedtech.com/support/forum/showthread.php?t=4572

 

[mistwang]

a new build is available, please do a "force reinstall" from web console.

 

[inxile]

I just forced reinstalled v4.0.20.

phpinfo() says PHP v5.3.5, Litespeed v5.5.

Do I need to compile PHP? Drop down offers 5.3.6.

 

[webizen]

the new build is for 4.1. if you don't have 4.1 installed before, just download the package from http://www.litespeedtech.com/litespeed-web-server-downloads.html. unpack and run ./install.sh from the expanded folder and choose U(pgrade) option when prompted.

whether to upgrade php to 5.3.6 is up to you. not a requirement.

 

[inxile]

Upgraded to v4.1

We continue to see these issues, in fact, I'd say the previous runaway process and retry mention is occurring more now.

Going through the settings, there appear to be numerous max connections settings... what are each of these and do they all need to be the same?

 

[webizen]

There are not the same.

For example, Max Connections in Server Tuning (http://www.litespeedtech.com/docs/webserver/config/tuning/#maxConnections) defines maximum concurrent TCP connections can be accepted by server from clients.

if PHP suEXEC is enabled, PHP suEXEC Max Conn (http://www.litespeedtech.com/docs/webserver/config/general/#phpSuExecMaxConn) specifies the maximum concurrent PHP requestes can be processed when run PHP script in suEXEC mode. This is the maximum number of PHP processes each user can get. For a busy site, this can be set to 200-300.

Max Connections in External App (http://www.litespeedtech.com/docs/webserver/config/extapps/#maxConns) specifies the maximum concurrent connections that can be established between the web server and an external application. This setting controls how many requests can be processed concurrently by an external application.

 

[inxile]

I think we are seeing a more stable environment, however, it is still occurring. I was able to take snapshots of the realtime graph of requests... I think it is very interesting and may help identify sources:

I'm starting to lean towards the idea that the connections and timeouts could be tweaked. Currently, they are set to the following:

General:

PHP suEXEC Max Conn: 500

Tuning:

Connection Timeout (secs): 60
Max Keep-Alive Requests: 3
Smart Keep-Alive: Yes
Keep-Alive Timeout (secs): 2

lsphp5 External App:

Max Connections: 50
PHP_LSAPI_MAX_REQUESTS=256
PHP_LSAPI_CHILDREN=50
Initial Request Timeout (secs): 30
Retry Timeout (secs): 15
Connection Keepalive Timeout: 2
Process Soft Limt: 300
Process Hard Limit: 350

Based on the above graph, it appears 'things' are going on somewhere around 45 or 15 seconds... would the above behavior relate to the Initial Request Timeout and Retry Timeout? Are those current values high, would dropping them down to a few seconds effect the servers ability to recover?

 

[webizen]

The graph seems to indicate that some PHP processes (if not all of them) take 30-45 seconds to finish as requests in processing pile up from 5-30 seconds steadily. The Initial Request Timeout (30) and Retry Timeout (15) can explain why there is a dip for requests in processing at 50 sec mark (quite a few timed out as NOT yet finished processing).

So the issue is why PHP scripts took a long time (even timed out).

 

[cmanns]

The graph seems to indicate that some PHP processes (if not all of them) take 30-45 seconds to finish as requests in processing pile up from 5-30 seconds steadily. The Initial Request Timeout (30) and Retry Timeout (15) can explain why there is a dip for requests in processing at 50 sec mark (quite a few timed out as NOT yet finished processing).

So the issue is why PHP scripts took a long time (even timed out).

He's got resource exhaustion.

Id say with that req/in/proc that php is getting mysql threads to block.

You need a overall analysis of your system and tuning.when a busy webserver like that nears swap page loads drop to a halt.

Try setting vm.swappyness to like 20 too. Install XCache set it to 24mb mem to 32. Then adjust how many php procs. Aim for more child processes vs parent so xcache does its job.

However php doesn't io lock systems, mysql does. Set your max_connections on mysql a tad lower what happens is php keeps spawning and hitting mysql till it eats so many cons with each thread.