[Resolved] Multiple APC Cache Instances + Low APC Uptime

[Carl]

Hello,

I was experiencing problem of having multiple instances of APC cache on Enterprise v4.2.18 4-core license running PHP 5.3.29. It seemed each of the four process/core carried its own instance of APC and there was a lot of "swapping" going on so I suppose that each request was handled by one of the four cores which loads its own instance of APC. This was very problematic as when my site clears and updates certain user cache entries, that task was only done on the core that triggered the process but not the other three.

I have filed a support ticket for this problem a while ago and I was advised to use PHP suEXEC daemon mode, which I have done so. The problem still exist as it seems there are still TWO instances of the cache and they seem to be "swapping" at random. What PHP/LS settings should i tweak to fix this problem?

Besides having multiple instances. another problem seems to be the low uptime of APC, I have read related posts here on the forum saying it is caused by LS restarting itself, but I couldn't see any clear/working solutions provided. What can I do to avoid this?

Thanks,
Carl

 

[mistwang]

When LSWS restarts, it needs to restart all children processes started by the web server, what you see is likely due to server restarts. Maybe you can try to minimize server restarts?

If you run a single large web site, do not need changing user id for different account. you can manually start lsphp5 process, server restart wont cause PHP to restart, but you are responsible for managing the PHP process, it wont be started by web server if it died.

 

[Carl]

I understand that server restart causes APC to be wiped out, but my LS test server has been left on for weeks, in Apache the uptime would be exactly how many weeks it was left on, but that is not the case with LS, so does LS restarts on its own? I have read somewhere else that LS may restart to read/update statistics from log files or something, is that the case here?

But more importantly you have not answered my first question, how do I stop LS from spawning two instances of APC? This problem has nothing to do with server restarts, and can be easily seen by going to the APC stat page, pressing the "Refresh Data" button repetitively, and see it showing one of the two different set of APC stats.

 

[mistwang]

I understand that server restart causes APC to be wiped out, but my LS test server has been left on for weeks, in Apache the uptime would be exactly how many weeks it was left on, but that is not the case with LS, so does LS restarts on its own? I have read somewhere else that LS may restart to read/update statistics from log files or something, is that the case here?

Only something tell LiteSpeed to restart, sometimes, control panel will ask web server to restart to rotate log or apply configuration changes. it is logged by our server.

But more importantly you have not answered my first question, how do I stop LS from spawning two instances of APC? This problem has nothing to do with server restarts, and can be easily seen by going to the APC stat page, pressing the "Refresh Data" button repetitively, and see it showing one of the two different set of APC stats.

It is likely means that your PHP is not using suEXEC daemon mode. it should not happen with suEXEC daemon mode. You should check if PHP is in suEXEC daemon mode. Just do "ps -ef | grep php", there should be a lsphp5 process running as root.
If it there is, I need to check your server to find out what is going on.

 

[Carl]

Yes, I have said in the post that suEXEC is turned on for lsphp5. (Enabled Auto Start, Set "Run On Stat Up" to suEXEC Daemon). Unless there are other settings I need to check to make sure it is runs properly. suEXEC seems to have reduced the number of APCs from four to two. but nonetheless there are still two.

And ps -ef | grep php returned the following

 0  9452 11203  0 Fri11AM ??  0:12.89 lsphp5
 0 10464 11203  0 19Nov14 ??  0:47.54 lsphp5
 0 11206 11203  0 11Nov14 ??  1:07.40 lsphp5
 0 12955 11203  0 27Nov14 ??  0:30.82 lsphp5
 0 34108 11203  0 Sun11AM ??  0:08.56 lsphp5
 0 35727 11203  0 21Nov14 ??  0:43.36 lsphp5
 0 36907 11203  0 13Nov14 ??  1:02.40 lsphp5
 0 37174 11203  0 29Nov14 ??  0:25.82 lsphp5
 0 58597 11203  0 Tue11AM ??  0:04.47 lsphp5
 0 60036 11203  0  1Dec14 ??  0:21.37 lsphp5
 0 61437 11203  0 15Nov14 ??  0:57.46 lsphp5
 0 61817 11203  0 23Nov14 ??  0:39.93 lsphp5
 0 82963 11203  0 11:43AM ??  0:00.02 lsphp5
 0 82964 11203  0 11:43AM ??  0:00.02 lsphp5
 0 84751 11203  0  3Dec14 ??  0:17.15 lsphp5
 0 86057 11203  0 17Nov14 ??  0:52.72 lsphp5
 0 87064 11203  0 25Nov14 ??  0:34.51 lsphp5
 0 82998 51236  0 11:47AM ttys000  0:00.00 grep php

it appears that a lot of different lsphp5 processes are running, is this normal?

 

[mistwang]

That's normal. all of those processes has a common parent process, 11203, you need to check that process, it should be a lsphp5 process running as root. if it is lscgid or lshttpd, you are not using suEXEC daemon mode.

 

[Carl]

This is proces 11203:

0 11203  1  0 11Nov14 ??  4:49.64 litespeed (lshttpd)

It looks like the process name is actually lshttpd, not lsphp5, so suEXEC daemon mode is not running according to what you've said.

I have "Auto Start" set to "Yes" and "Run On Start Up" set to "suEXEC Daemon" under Configuration > Server > External App > lsphp5 Edit, is there other settings I need to check?

 

[mistwang]

Maybe you need to rebuild your lsphp5 binary, the current binary is using an older version of LSAPI which is not capable of running suEXEC daemon mode. please check phpinfo() output for the SAPI version, the latest version is "LiteSpeed V6.7".

Are you using a control panel?

 

[Carl]

phpinfo says I'm running LS v6.6.

How would I go about rebuilding lsphp5?

And sorry I don't know what's the control panel you are referring to, but I guess I am not using any.

 

[mistwang]

http://www.litespeedtech.com/support/wiki/doku.php?id=litespeed_wiki:php:lsapi

make sure to copy your existing configuration from phpinfo().

 

[Carl]

rebuilt php with same settings, did a hard restart on the server with /usr/local/lsws/bin/lswsctrl stop/start, and the parent process is still lshttpd.. any other suggestions?

 

[mistwang]

Are you sure that you changed the correct external app configuration? server level? vhost level?

 

[Carl]

It is configured at the server level under Configuration > Server > External App > lsphp5, and there is no apps at the VH level under Virtual Host > (My VH) > view/edit > External App

 

[mistwang]

Please show us the full lsphp5 extapp configuration.

 

[Carl]

Here is the screenshot

 

[mistwang]

Looks fine, is there any other PHP related external app configuration?
maybe we need to login to take a look.

 

[mistwang]

It turns out to be a problem with MACOSX package, addressed in latest 4.2.19 build.

 

[Carl]

I am using that build, do I have to reinstall the server then?

 

[Carl]

Here is an update on the processes

0  466  1  0 10:46AM ??  0:00.01 litespeed (lshttpd)
0  469  466  0 10:46AM ??  0:00.00 httpd (lscgid)
0  472  466  0 10:46AM ??  0:00.02 lsphp5
0  473  466  0 10:46AM ??  0:00.01 lsphp5
-2  556  473  0 10:47AM ??  0:00.37 lsphp5
-2  559  472  0 10:47AM ??  0:00.10 lsphp5

it turns out that there are two lsphp5 processes with parent 466 - litespeed (lshttpd) running under user root (0)
but there are two more lsphp5s running with parent 473 and 472 - lsphp5 running under user nobody (-2)

I'm thinking the 466 process should also be lsphp5 if sever is running in suEXEC daemon mode?

 

[mistwang]

Just click "Force reinstall" link under "Version Manager" page,
Your output looks like the older build is still in use. There should be only "472", "473" should not be started. Only one lsphp5 process started by litespeed process.