503 Errors - APC with cPanel matched PHP binary

Z

ziggy95

Member
#1
#1
Hi there. I have successfully installed litespeed and matched the PHP 5.2.8 binary from cPanel. I have installed APC and included it in php.ini correctly.

When I restart litespeed, I confirm that APC is added to phpinfo().

PHP pages load fine for a few seconds, and then quickly go to 503 errors. There is nothing recorded to stderr.log.

In error_log it says:

2009-02-18 12:36:59.838 [NOTICE] [97.77.97.51:1386-0#APVH_sitename.com:lsapi] Redirect: #2, URL: /vbseo.php
2009-02-18 12:36:59.838 [ERROR] [97.77.97.51:1386-0#APVH_sitename.com:lsapi] detect loop redirection.
2009-02-18 12:37:00.829 [INFO] [97.77.97.51:1389-0#APVH_sitename.com] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
2009-02-18 12:37:00.831 [INFO] [97.77.97.51:1389-0#APVH_sitename.com] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
2009-02-18 12:37:00.833 [INFO] [97.77.97.51:1389-0#APVH_sitename.com] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
2009-02-18 12:37:00.833 [NOTICE] [97.77.97.51:1389-0#APVH_sitename.com] oops! 503 Service Unavailable
2009-02-18 12:37:00.833 [NOTICE] [97.77.97.51:1389-0#APVH_sitename.com] Content len: 0, Request line:
GET / HTTP/1.1
2009-02-18 12:37:00.834 [INFO] [97.77.97.51:1389-0#APVH_sitename.com:lsapi] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
2009-02-18 12:37:00.835 [INFO] [97.77.97.51:1389-0#APVH_sitename.com:lsapi] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
2009-02-18 12:37:00.836 [INFO] [97.77.97.51:1389-0#APVH_sitename.com:lsapi] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
2009-02-18 12:37:00.836 [NOTICE] [97.77.97.51:1389-0#APVH_sitename.com:lsapi] oops! 503 Service Unavailable
2009-02-18 12:37:00.836 [NOTICE] [97.77.97.51:1389-0#APVH_sitename.com:lsapi] Content len: 0, Request line:
GET / HTTP/1.1
2009-02-18 12:37:00.836 [NOTICE] [97.77.97.51:1389-0#APVH_sitename.com:lsapi] Redirect: #2, URL: /vbseo.php
2009-02-18 12:37:00.836 [ERROR] [97.77.97.51:1389-0#APVH_sitename.com:lsapi] detect loop redirection.
2009-02-18 12:37:32.870 [INFO] [66.249.70.226:45454-0#APVH_sitename.com] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
2009-02-18 12:37:32.871 [INFO] [66.249.70.226:45454-0#APVH_sitename.com] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
2009-02-18 12:37:32.873 [INFO] [66.249.70.226:45454-0#APVH_sitename.com] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
2009-02-18 12:37:32.874 [NOTICE] [66.249.70.226:45454-0#APVH_sitename.com] oops! 503 Service Unavailable
2009-02-18 12:37:32.874 [NOTICE] [66.249.70.226:45454-0#APVH_sitename.com] Content len: 0, Request line:
GET /members/sexculer.html HTTP/1.1
2009-02-18 12:37:32.874 [NOTICE] [66.249.70.226:45454-0#APVH_sitename.com] Redirect: #1, URL: /vbseo.php
2009-02-18 12:37:32.875 [INFO] [66.249.70.226:45454-0#APVH_sitename.com:lsapi] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
2009-02-18 12:37:32.878 [INFO] [66.249.70.226:45454-0#APVH_sitename.com:lsapi] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
2009-02-18 12:37:32.880 [INFO] [66.249.70.226:45454-0#APVH_sitename.com:lsapi] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
2009-02-18 12:37:32.880 [NOTICE] [66.249.70.226:45454-0#APVH_sitename.com:lsapi] oops! 503 Service Unavailable
2009-02-18 12:37:32.880 [NOTICE] [66.249.70.226:45454-0#APVH_sitename.com:lsapi] Content len: 0, Request line:
GET /members/sexculer.html HTTP/1.1
2009-02-18 12:37:32.880 [NOTICE] [66.249.70.226:45454-0#APVH_sitename.com:lsapi] Redirect: #3, URL: /vbseo.php
2009-02-18 12:37:32.880 [ERROR] [66.249.70.226:45454-0#APVH_sitename.com:lsapi] detect loop redirection.

I would like to use APC, but must disable it until I can resolve this.
 
Z

ziggy95

Member
#2
#2
Static pages continue to load fine. At first I thought maybe the problem was a conflict with vbseo.php, but then after a while even phpinfo() goes to 503.
 
Z

ziggy95

Member
#4
#4
I tried increasing soft/hard for lsphp5 to from 250M/300M to 2500M/2500M and restarted and still get same result. Do I need to change it somewhere else also?
 
M

mistwang

LiteSpeed Staff
#9
#9
Are you using a VPS?
What is the cache memory size for APC in php.ini? maybe you need to reduce the cache memory size, especially when you use PHP suEXEC. Maybe you should disable PHP suEXEC if you use VPS to host a few web sites, not for shared hosting.
 
Z

ziggy95

Member
#10
#10
It is a dedicated server cluster, one web server and one database server. I am not using suEXEC. I haven't specified cache size in php.ini for APC, I'll look into that, though I am doubtful that the default would cause overload - especially since I tried raising limit to 3,000MB.
 
Z

ziggy95

Member
#11
#11
After trying again we are not getting the 503 errors anymore. There was a problem where the database connection was routing through the network LAN vs the local LAN between the two boxes. I'm not sure how this would have played into things, but now that the connection is direct the errors aren't happenning.... yet.

Will keep you posted.
 
Z

ziggy95

Member
#13
#13
mistwang said:
You should use LAN for DB connection. always use the fastest available connection.
Click to expand...
Yes, of course we understand this. :) Someone had made a mistake and it was corrected.

However, we are back to the 503 errors. So far we have only moved a single low traffic site to the server. We are testing and near moving a very high traffic site (vbulletin forum) over.

I have tried setting memory in Security and External App to 3,000M. No good.

2009-02-19 13:17:31.799 [NOTICE] [68.83.66.8:49373-0#APVH_sitename.com] Redirect: #1, URL: /vbseo.php
2009-02-19 13:17:31.801 [INFO] [68.83.66.8:49373-0#APVH_sitename.com:lsapi] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
2009-02-19 13:17:31.804 [INFO] [68.83.66.8:49373-0#APVH_sitename.com:lsapi] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
2009-02-19 13:17:31.807 [INFO] [68.83.66.8:49373-0#APVH_sitename.com:lsapi] connection to [uds://tmp/lshttpd/lsphp5.sock] on request #0, Connection reset by peer!
2009-02-19 13:17:31.807 [NOTICE] [68.83.66.8:49373-0#APVH_sitename.com:lsapi] oops! 503 Service Unavailable

is what shows in error_log.

I guess we don't have to have APC, though it would likely help.

Any advice?
 
M

mistwang

LiteSpeed Staff
#14
#14
Maybe you have set process limits of lsphp5 too low?
check how many lsphp5 processes do you get?

I think for some reason, APC crashes PHP, maybe look into other cache solutions like eAccelerator or xcache.
 
M

mistwang

LiteSpeed Staff
#16
#16
Since you do not use PHP suEXEC, you can try strace the parent lsphp5 process with command

strace -f -p <pid_of_lsphp5>

when you get 503 errors. It can give you some hints on why PHP crashed for some reason.
 
Z

ziggy95

Member
#17
#17
I installed and configured xcache. Within seconds of restarting the server, the 503 errors come back. I setting max processes to 0/0, no luck. Here is the result of strace:

# strace -f -p 16225
Process 16225 attached - interrupt to quit
select(6, [5], NULL, NULL, {0, 853000}) = 0 (Timeout)
getppid() = 16224
select(6, [5], NULL, NULL, {1, 0}) = 1 (in [5], left {0, 226000})
read(5, "LS\1\0\355\6\0\0\315\3\0\0\0\0\0\0\201\1\0\0\372\1\0\0\265\1\0\0n\2\0\0"..., 8192) = 1773
write(5, "LS\7\0\10\0\0\0", 8) = 8
setitimer(ITIMER_PROF, {it_interval={0, 0}, it_value={60, 0}}, NULL) = 0
rt_sigaction(SIGPROF, {0x685d80, [PROF], SA_RESTORER|SA_RESTART, 0x3ed4e301b0}, {0x685d80, [PROF], SA_RESTORER|SA_RESTART, 0x3ed4e301b0}, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [PROF], NULL, 8) = 0
getcwd("/usr/local/lsws/fcgi-bin", 4095) = 25
chdir("/home/sitename/public_html") = 0
setitimer(ITIMER_PROF, {it_interval={0, 0}, it_value={30, 0}}, NULL) = 0
rt_sigaction(SIGPROF, {0x685d80, [PROF], SA_RESTORER|SA_RESTART, 0x3ed4e301b0}, {0x685d80, [PROF], SA_RESTORER|SA_RESTART, 0x3ed4e301b0}, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [PROF], NULL, 8) = 0
open("/home/sitename/public_html/vbseo.php", O_RDONLY) = 6
fstat(6, {st_mode=S_IFREG|0644, st_size=48445, ...}) = 0
mmap(NULL, 48445, PROT_READ, MAP_SHARED, 6, 0) = 0x2aaaafef6000
munmap(0x2aaaafef6000, 48445) = 0
close(6) = 0
open("/home/sitename/public_html/vbseo.php", O_RDONLY) = 6
fstat(6, {st_mode=S_IFREG|0644, st_size=48445, ...}) = 0
fcntl(6, F_GETFL) = 0x8000 (flags O_RDONLY|O_LARGEFILE)
fstat(6, {st_mode=S_IFREG|0644, st_size=48445, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2aaaafef6000
lseek(6, 0, SEEK_CUR) = 0
lseek(6, 0, SEEK_SET) = 0
read(6, "<?php\n\n/************************"..., 4096) = 4096
lseek(6, 4096, SEEK_SET) = 4096
stat("/home/sitename/public_html/vbseo.php", {st_mode=S_IFREG|0644, st_size=48445, ...}) = 0
fcntl(1, F_SETLKW, {type=F_WRLCK, whence=SEEK_SET, start=0, len=1}) = 0
fcntl(1, F_SETLKW, {type=F_UNLCK, whence=SEEK_SET, start=0, len=1}) = 0
lseek(6, -4096, SEEK_CUR) = 0
close(6) = 0
munmap(0x2aaaafef6000, 4096) = 0
--- SIGSEGV (Segmentation fault) @ 0 (0) ---
Process 16225 detached
 
M

mistwang

LiteSpeed Staff
#18
#18
Which version of PHP is used?
Maybe you can try downgrade PHP version.

How about Apache mod_php? will APC or Xcache crash it?
The problem is between PHP and opcode cache, you can try to let lsphp5 dump core file and exam the core file with GDB.

To let lsphp5 dump core file, you need to add environment variable to lsphp5 external application

LSAPI_ALLOW_CORE_DUMP=1
 
You must log in or register to reply here.
Top