This is an old revision of the document!
How to submit a bug report
To submit a useful bug report you need to provide as much information as you can, including:
- Your Server OS (Linux/FreeBSD/Solaris/MacOSX)
- CPU/Processor family (32bit/64bit)
- If you think LSWS does not work properly with a certain feature, you should provide as detailed information as possible on the feature and how you use it, so we can reproduce the problem in our lab.
- If lshttpd crashed, LSWS Enterprise users should switch to the debug build of lshttpd, then submit the debug report generated and a core file if possible.
- GDB backtrace information on core file.
If you receive some email alert on the crash, you can forward that email to us. If not, you can use gdb
to backtrace the crash and send to us for further investigation. If the error log shows the server crashed but no core file was generated, you may need to enable a core dump for your system and reboot your server.
LSWS crash report
When you receive some email alert similar to the following:
Web server example.com on example.com automatically restarted --------- Forwarded message --------- From: root <root@abc.example.com> Date: seg, 10 de dez de 2018 às 12:38 Subject: Web server example.com on example.com automatically restarted To: <root@localhost> At [10/Dec/2018:12:38:26 -0200], web server with pid=12264 received unexpected signal=11, a core file has been created. A new instance of web server will be started automatically! Please forward the following debug information to bug@litespeedtech.com. Environment: Server: LiteSpeed/5.3.4 Enterprise Build 4 OS: Linux Release: 3.10.0-862.14.4.el7.x86_64 Version: #1 SMP Wed Sep 26 15:12:11 UTC 2018 Machine: x86_64 If the call stack information does not show up here, please compress and forward the core file located in /tmp/lshttpd/. [New LWP 12264] [New LWP 12265] [New LWP 12266] Core was generated by `litespeed'. Program terminated with signal 11, Segmentation fault. #0 0x0000000000557bc2 in ?? () #0 0x0000000000557bc2 in ?? () No symbol table info available. #1 0x00000000042763e0 in ?? () No symbol table info available. #2 0x00000000042763e0 in ?? () No symbol table info available. #3 0x00000000042e0110 in ?? () No symbol table info available. #4 0x0000000000557ccb in ?? () No symbol table info available. #5 0x000000000420c988 in ?? () No symbol table info available. #6 0x000000000404b000 in ?? () No symbol table info available. #7 0x00000000042e0110 in ?? () No symbol table info available. #8 0x0000000000000008 in ?? () No symbol table info available. #9 0x00000000042e0d98 in ?? () No symbol table info available. #10 0x000000000055a0e3 in ?? () No symbol table info available. #11 0x00000000042e0110 in ?? () No symbol table info available. #12 0x00000000042e0158 in ?? () No symbol table info available. #13 0x0000000000000000 in ?? () No symbol table info available. rax 0x4023200 67252736 rbx 0x42763e0 69690336 rcx 0x42763d0 69690320 rdx 0x0 0 rsi 0x404b000 67416064 rdi 0x42763e0 69690336 rbp 0x42e0110 0x42e0110 rsp 0x7ffcc447b170 0x7ffcc447b170 r8 0x1 1 r9 0x404aff0 67416048 r10 0x0 0 r11 0x0 0 r12 0x404b000 67416064 r13 0x3 3 r14 0x64 100 r15 0x0 0 rip 0x557bc2 0x557bc2 eflags 0x10246 [ PF ZF IF RF ] cs 0x33 51 ss 0x2b 43 ds 0x0 0 es 0x0 0 fs 0x0 0 gs 0x0 0
it means LSWS server crashed and a core file has been created at /tmp/lshttpd/bak_core
ll /tmp/lshttpd/bak_core drwxr-xr-x 2 root nobody 4096 May 3 2018 ./ drwxr-x--x 4 nobody nobody 20480 Dec 11 11:10 ../ -rw------- 1 nobody nobody 53587968 May 3 2018 core.2578440
Before you log a bug report to us, please check your version and build number and run force upgrade command to the latest build of lastest version and it may have fixed your problem already.
/usr/local/lsws/admin/misc/lsup.sh -f -v <latest_version_number>
If you receive an email similar to the following email without backtrace information, please install gdb firstly.
At [10/Dec/2018:07:15:19 +0000], web server with pid=4059 received unexpected signal=6, a core file has been created. A new instance of web server will be started automatically! Please forward the following debug information to bug@litespeedtech.com. Environment: Server: LiteSpeed/5.3.4 Enterprise Build 6 OS: Linux Release: 3.10.0-514.21.1.el7.x86_64 Version: #1 SMP Thu May 25 17:04:51 UTC 2017 Machine: x86_64 If the call stack information does not show up here, please compress and forward the core file located in /tmp/lshttpd/.
If you have installed gdb and also LSWS crashed on the latest build of the latest version, please switch to debug build:
/usr/local/lsws/admin/misc/lsup.sh -d -f -v <latest_version_number>
then wait for the next crash to happen and log a bug report ticket by providiong tmp root ssh login for our developer to take a further analysis.
Debug build of LSWS Enterprise
Starting from version 5.0.x, you do not need a separate debug build since the production LSWS binary installed has debug info enabled by default. It has no impact on your server performance.
For versions earlier than 5.0, there are two versions of the lshttpd binary shipped in the LSWS Enterprise package. One is the release build. The other is the debug build. The debug build is not normally used when you run LSWS, but, when debugging, the GDB stack backtrace and core file generated by the debug build will be used for bug analysis and bug fixes. If there is no debug build of lshttpd binary in your current installation, you can download the related version and arch of the debug build through the following command:
/usr/local/lsws/admin/misc/lsup.sh -d -f -v <your_lsws_version>
How to enable a core dump
For PHP
By default, core dumping is disabled in LSAPI PHP external applications. To enable it, you need to add an environment variable to your LSPHP external application configuration (WebAdmin console > Configuration > External App > your external application > Environment):
LSAPI_ALLOW_CORE_DUMP=1
Server-wide
If you only need a particular program to be troubleshooted, you can use soft limits:
ulimit -S -c
Edit /etc/security/limits.conf
and add the following line at the bottom of the file (before # End of file
)
* soft core unlimited
Check ulimit core file size value. It should be unlimited.
ulimit -c -> unlimited
Core dumping is generally disabled by default on Linux. The following guide will lead you through the steps necessary to enable it: How to enable core dump for Linux
Install GDB and backtrace the core file
Install GDB on your server if not installed yet, and use the syntax gdb /usr/local/lsws/bin/litespeed <path/to/core/file>
to check the debug infomation in core file.
Once you have opened the core file with GDB, use the bt
command to print a backtrace of the stack (the steps that the application took leading up to the crash). This will often reveal what caused the crash.
You can forward the above backtrace information + core files to bug@litespeedtech.com .
Email Alert
If you enable email alert on your server and GDB is enabled, you should receive some alert email like the following. You can forward the email to bug@litespeedtech.com.
##---------- Forwarded message ----------## From: root <root@example.com> To: root@localhost Cc: Bcc: Date: Wed, 01 Nov 2017 01:00:27 -0500 Subject: Web server example.com on example.com automatically restarted At [01/Nov/2017:01:00:25 -0500], web server with pid=12964 received unexpected signal=11, a core file has been created. A new instance of web server will be started automatically! Please forward the following debug information to [[bug@litespeedtech.com]]. Environment: Server: LiteSpeed/5.2.2 Enterprise Build 2 OS: Linux Release: 3.10.0-693.5.2.el7.x86_64 Version: #1 SMP Fri Oct 20 20:32:50 UTC 2017 Machine: x86_64 If the call stack information does not show up here, please compress and forward the core file located in /tmp/lshttpd/. [New LWP 12964] [New LWP 12971] [New LWP 12972] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `litespeed'. Program terminated with signal 11, Segmentation fault. #0 0x0000000000607440 in lsShm_hElem_s::getLruLinkPtr (this=0x0) at /home/gwang/release/litespeed/src/shm/lsshmhash.h:102 #0 0x0000000000607440 in lsShm_hElem_s::getLruLinkPtr (this=0x0) at /home/gwang/release/litespeed/src/shm/lsshmhash.h:102 No locals. #1 0x00000000006a6553 in lsShm_hElem_s::setLruLinkNext (this=0x0, off=...) at /home/gwang/release/litespeed/src/shm/lsshmhash.h:110 No locals. #2 0x00000000006a681c in LsShmHash::set_linkNext (this=0x144a880, offThis=..., offNext=...) at /home/gwang/release/litespeed/src/shm/lsshmhash.h:686 No locals. #3 0x00000000006a5953 in LsShmHash::linkSetTop (this=0x144a880, pElem=0x7f7d53dc4b80, offElem=...) at /home/gwang/release/litespeed/src/shm/lsshmhash.cpp:1581 pLru = 0x7f7d53d00a1c prev = {m_iOffset = 610112} __PRETTY_FUNCTION__ = "void LsShmHash::linkSetTop(LsShmHElem*, LsShmHash::iteroffset)" pLink = 0x7f7d53dc4b9c next = {m_iOffset = 12479344} #4 0x00000000006a34c1 in LsShmHash::doGet (this=0x144a880, iterOff=..., key=4039776365, pParms=0x7ffd84259560, pFlag=0x7ffd84259614) at /home/gwang/release/litespeed/src/shm/lsshmhash.cpp:809 iter = 0x7f7d53dc4b80 #5 0x00000000006a4a9a in LsShmHash::getPtr (pThis=0x144a880, pParms=0x7ffd84259560, pFlag=0x7ffd84259614) at /home/gwang/release/litespeed/src/shm/lsshmhash.cpp:1206 key = 4039776365 iterOff = {m_iOffset = 805760} #6 0x000000000069f9fb in LsShmHash::getIterator (this=0x144a880, pParms=0x7ffd84259560, pFlag=0x7ffd84259614) at /home/gwang/release/litespeed/src/shm/lsshmhash.h:401 iterOff = {m_iOffset = 44971056} #7 0x000000000069f8fd in LsShmHash::get (this=0x144a880, pKey=0x2ae3430, keyLen=8, valLen=0x7ffd84259618, pFlag=0x7ffd84259614) at /home/gwang/release/litespeed/src/shm/lsshmhash.h:323 iterOff = {m_iOffset = 2217055712} parms = {key = {ptr = 0x2ae3430 "\357\t\304\330\326\373e8", len = 8}, val = {ptr = 0x0, len = 12}} iter = 0x2ae3430 #8 0x000000000069f2bb in ShmCacheManager::addTracking2 (this=0x15024a0, pEntry=0x2ae3410, pTracker=0x144a880) at /home/gwang/release/litespeed/src/cache/shmcachemanager.cpp:1141 pData = 0x943b51 valLen = 12 flag = 0 offVal = 0 #9 0x000000000069f248 in ShmCacheManager::addTracking (this=0x15024a0, pEntry=0x2ae3410) at /home/gwang/release/litespeed/src/cache/shmcachemanager.cpp:1128 pTracker = 0x144a880 #10 0x000000000055baef in HttpSession::preparePageCache (this=0x30f9ea0) at /home/gwang/release/litespeed/src/http/httpsession.cpp:6463 pEntry = 0xfffffffffffffffe pIP = 0x0 ipLen = 0 age = 604800 priv_cache = 0 pResp = 0x30fa358 pCtx = 0x149afd0 #11 0x0000000000555d79 in HttpSession::setupDynRespBody (this=0x30f9ea0, prepareCache=1) at /home/gwang/release/litespeed/src/http/httpsession.cpp:4439 No locals. #12 0x0000000000556641 in HttpSession::appendDynBody (this=0x30f9ea0, pBuf=0xd7b420 <HttpGlobals::g_achBuf> "<!DOCTYPE html>\n<html lang=\"en-US\" prefix=\"og: http://ogp.me/ns# fb: http://ogp.me/ns/fb#\">\n<head >\n<meta charset=\"UTF-8\" />\n<title>My Bookmark - MyReadingManga</title><meta name=\"viewport\" content=\"w"..., len=16384) at /home/gwang/release/litespeed/src/http/httpsession.cpp:4631 buffered = 32765 ret = 0 #13 0x000000000056eec8 in HttpExtConnector::processRespBodyData (this=0x14a4800, pBuf=0xd7b420 <HttpGlobals::g_achBuf> "<!DOCTYPE html>\n<html lang=\"en-US\" prefix=\"og: http://ogp.me/ns# fb: http://ogp.me/ns/fb#\">\n<head >\n<meta charset=\"UTF-8\" />\n<title>My Bookmark - MyReadingManga</title><meta name=\"viewport\" content=\"w"..., len=16384) at /home/gwang/release/litespeed/src/http/httpextconnector.cpp:371 ret = 16384 #14 0x000000000066d7f7 in LsapiConn::readRespBody (this=0x314a9e0) at /home/gwang/release/litespeed/src/extensions/lsapi/lsapiconn.cpp:943 len = 0 packetLen = 16384 pBuf = 0xd7b420 <HttpGlobals::g_achBuf> "<!DOCTYPE html>\n<html lang=\"en-US\" prefix=\"og: http://ogp.me/ns# fb: http://ogp.me/ns/fb#\">\n<head >\n<meta charset=\"UTF-8\" />\n<title>My Bookmark - MyReadingManga</title><meta name=\"viewport\" content=\"w"... toRead = 16384 pHEC = 0x14a4800 ret = 16384 bufLen = 16384 respState = @0x14a4874: 526339 total = 16384 #15 0x000000000066cb13 in LsapiConn::processResp (this=0x314a9e0) at /home/gwang/release/litespeed/src/extensions/lsapi/lsapiconn.cpp:683 ret = 8 count = 2 #16 0x000000000066c2c3 in LsapiConn::doRead (this=0x314a9e0) at /home/gwang/release/litespeed/src/extensions/lsapi/lsapiconn.cpp:361 ret = 0 #17 0x000000000067e727 in ExtConn::onRead (this=0x314a9e0) at /home/gwang/release/litespeed/src/extensions/extconn.cpp:344 ret = 0 #18 0x000000000068720f in EdStream::handleEvents (this=0x314a9e0, event=1) at /home/gwang/release/litespeed/src/edio/ediostream.cpp:70 ret = 0 #19 0x000000000068b6e2 in epoll::processEvents (this=0x135fa80, events=1) at /home/gwang/release/litespeed/src/edio/epoll.cpp:371 fd = 76 pReactor = 0x314a9e0 p = 0x1370130 __PRETTY_FUNCTION__ = "virtual int epoll::processEvents(int)" #20 0x000000000068b5c5 in epoll::waitAndProcessEvents (this=0x135fa80, iTimeoutMilliSec=100) at /home/gwang/release/litespeed/src/edio/epoll.cpp:344 ret = 1 #21 0x000000000054192d in EventDispatcher::run (this=0x1345c78) at /home/gwang/release/litespeed/src/http/eventdispatcher.cpp:244 ret = 1 evt_waiting = 0 sigEvent = 0 errors = 0 pQuicEngine = 0x13a3a50 #22 0x00000000004ef55b in HttpServerImpl::start (this=0x1345c50) at /home/gwang/release/litespeed/src/main/httpserver.cpp:529 No locals. #23 0x00000000004f2bfa in HttpServer::start (this=0xd752d0 <TSingleton<HttpServer>::getInstance()::s_instance>) at /home/gwang/release/litespeed/src/main/httpserver.cpp:2031 No locals. #24 0x00000000004e83d1 in LshttpdMain::main (this=0x1345870, argc=1, argv=0x7ffd84259ca8) at /home/gwang/release/litespeed/src/main/lshttpdmain.cpp:1950 ret = 0 #25 0x00000000004e36e6 in main (argc=1, argv=0x7ffd84259ca8) at /home/gwang/release/litespeed/src/main.cpp:140 ret = 0 rax 0x0 0 rbx 0x1345870 20207728 rcx 0x7f7d53d00a1c 140176253782556 rdx 0x10 16 rsi 0xc4b80 805760 rdi 0x0 0 rbp 0x7ffd84259bc0 0x7ffd84259bc0 rsp 0x7ffd84259b90 0x7ffd84259b90 r8 0x7ffd84259614 140726820509204 r9 0x7f7d59fca340 140176357368640 r10 0x4 4 r11 0x7f7d5a0032d0 140176357602000 r12 0x4e3578 5125496 r13 0x7ffd84259ca0 140726820510880 r14 0x0 0 r15 0x0 0 rip 0x4e36e6 0x4e36e6 <main(int, char**)+102> eflags 0x10206 [ PF IF RF ] cs 0x33 51 ss 0x2b 43 ds 0x0 0 es 0x0 0 fs 0x0 0 gs 0x0 0
Where to send the bug report
Bug reports can be sent to our bug report email.
We recommend that Enterprise customers use our ticket system (accessible from the client area) for their bug reports.