503 Errors

[freeballt]

2011-06-02 19:12:06.105 [NOTICE] [***********:49925-0#TEST] POST request in process stage, fail with 503
2011-06-02 19:12:06.105 [NOTICE] [***********:49925-0#TEST] oops! 503 Service Unavailable
2011-06-02 19:12:06.105 [NOTICE] [***********:49925-0#TEST] Content len: 106, Request line: 'POST /ajax.php?do=shownewnotifications HTTP/1.1'
2011-06-02 19:12:06.105 [INFO] [***********:49925-0#TEST] Cookie len: 540, bblastvisit=1306471780; bblastactivity=0; __utma=215042507.2144092086.1306471796.1306987648.1307055923.18; __utmz=215042507.1306698997.10.4.utmcsr=google|utmc$
2011-06-02 19:12:06.105 [NOTICE] [***********:49925-0#TEST] Redirect: #1, URL: /vbseo.php


2011-06-02 19:12:11.261 [INFO] [***********:50686-0#TEST] connection to [/tmp/lshttpd/lsphp5.sock.597] on request #23, confirmed, 1, associated process: 20549, running: 1, error: Connection reset by peer!

Myself and other users of my site have been getting 503 errors in abundance and I've been trying to figure out why. It seems to be linked to two error messages, both listed above. The first has to do with cookie length, and if that is the case, what is the maximum cookie length permitted by Litespeed? If there is a limit, is there a way of disabling it?

The second error, I am unable to replicate, but appears all through out my error logs.

Running vBulletin with 4.1.1...

An assistance would be appreciated.

 

[NiteWave]

there is a setting:
Server->Tuning->Max Request Header Size (bytes)

but usually it's not a problem.

another clue is to check if ip_conntrack tables is full.

check:
#cat /proc/sys/net/ipv4/ip_conntrack_max
#cat /proc/sys/net/ipv4/netfilter/ip_conntrack_count

 

[freeballt]

Clearing bbthread_lastview and bbforum_view seems to fix the issue whenever I have the first error (and they are the bulk of the cookie when they get that big). Other vbulletin forums hosted with litespeed seem to have this common issue.

Also, Max Request Header Size (bytes) 16380. Cookie len: 540 is no where near 16380.

[root@Server1 logs]# cat /proc/sys/net/ipv4/ip_conntrack_max
cat: /proc/sys/net/ipv4/ip_conntrack_max: No such file or directory
[root@Server1 logs]# cat /proc/sys/net/ipv4/netfilter/ip_conntrack_count
cat: /proc/sys/net/ipv4/netfilter/ip_conntrack_count: No such file or directory

What is the cause for the second error? Is it a generic error or does it have another purpose?

 

[NiteWave]

the 2nd "error" is not error, just info, can ignore it safely in general.

 

[freeballt]

Hmm, well someone emailed me and told me that they had a 503, they gave me their IP and that was the only thing on the log.

Error #1 still remains with cookies.

 

[NiteWave]

Clearing bbthread_lastview and bbforum_view seems to fix the issue

in browser?

 

[freeballt]

in browser?

Yes, they make the cookie very large. Every time someone views a thread, it adds that thread id to the cookie. They can get very large. I've had reports of people getting 503 errors while viewing threads, the only way to fix it is to remove the cookies.

The first error shows up in the log when that occurs.

 

[NiteWave]

Thanks for the info!

maybe need modify the vBulletin code, limit size of cookie vaule bbthread_lastview and bbforum_view, truncate them when too big.

 

[freeballt]

Yeah, but when should I truncate it? Max Request Header Size does not represent the cookie maximum size, and since it seems to be a cookie related error, I don't know what the limit is. If I were to completely remove those two cookies, it would completely limit several aspects of my forum, including latest post, subscribed threads, and forums that have been read (not including the functions I am aware of). Could you give me any more information about the limits?

 

[NiteWave]

first, need locate the vB source code where set the cookies.
in example of bbforum_view, it keeps recent-visited-forum history of a user, limit to recent 10 forums, not appending it forever.

 

[freeballt]

Doing that would still cause forum issues.

Why does that size of the cookie work fine with apache but not litespeed?

 

[webizen]

see this thread about cookie issue (more of a PHP issue) http://www.litespeedtech.com/support/forum/showthread.php?t=4657&page=2

 

[freeballt]

see this thread about cookie issue (more of a PHP issue) http://www.litespeedtech.com/support/forum/showthread.php?t=4657&page=2

Right, but this error doesn't occur with the same version of php w/ apache. Apache is too slow, that's why I use litespeed. Litespeed has an issue with cookies (perhaps since PHP is compiled with whatever optimizations you guys add?), preventing some of my users from even browsing the forum = big problem.

Just fixed someone's problem on MSN Messenger:

Me says:
*clear ur cookies for the site
Him says:
*done
*ill see now
*Yeah solved, But still a bit slow

His error was "Error 101 (net::ERR_CONNECTION_RESET):" with Chrome.

Edit: other websites are having the same issue http://www.litespeedtech.com/support/forum/archive/index.php/t-3887.html

 

[NiteWave]

please do Force Reinstall 4.1.1 now. the cookie issue may disappear. Will work better than apache in this regards as well!

 

[freeballt]

please do Force Reinstall 4.1.1 now. the cookie issue may disappear. Will work better than apache in this regards as well!

Same thing as before. I've experienced it personally and I reinstalled it last night. You guys can easily replicate it on your side, save a large cookie and test it with this board which I am assuming is hosted with litespeed.

 

[mistwang]

can we login and take a look at your server.
It is not that easy to reproduce in our lab, otherwise, it should have been fixed easily.

 

[freeballt]

If compiling it without Suhosin doesn't fix it, then sure.

Could that be the issue? PHP hardening... if the cookie gets too big reject it and 503?

 

[NiteWave]

if the cookie gets too big reject it and 503?

no, should not. if you can work out a test case, can reproduce the 503 error, that should be very helpful.

 

[cmanns]

If compiling it without Suhosin doesn't fix it, then sure.

Could that be the issue? PHP hardening... if the cookie gets too big reject it and 503?

Disabling shuosin will probably def help.

We havnt had any issues and we hosted large vb with millions of posts and hundreds of active users.

 

[superlink]

Anyone knows how to fix this issue?, same problem here using vBulletin. These 503 errors only happens with POST requests from different php files randomly.

Thanks