Reduce load/disk usage

[DoM]

Hello,
i would like to know if you can suggest few tips in order to reduce disk usage.

Server Hardware Config:

2 X Intel Xeon Quad Core E5420
8GB RAM
4 X HDD 1TB SATA II in RAID10

Mysql is remote.

I also have server with SAS HDD and i can see of course difference from web server with SAS hdd and SATA hdd but anyway i think that this difference could be win with right directives configuration of LiteSpeedTech web server.

This server host about 540 cPanel account.

I use version 4.1RC4.
Server has 5GB RAM free always.

Hdparm show following:

hdparm -tT /dev/sda

/dev/sda:
Timing cached reads: 18572 MB in 1.99 seconds = 9312.96 MB/sec
Timing buffered disk reads: 580 MB in 3.00 seconds = 193.12 MB/sec

During day( without backup or stats calculation), Load is always about 7-8 and few times up to 11-12.

Directives of litespeed that i tried to setup are:

Memory I/O Buffer 180M
Max Keep-Alive Requests 100
Smart Keep-Alive yes
Keep-Alive Timeout (secs) 5
Total MMAP Cache Size (bytes) 80M
Use Asynchronized disk I/O (AIO) yes
AIO Block Size 1M

What i have to do in order to reduce LOAD or reduce disk usage instead of RAM ?

What do you suggest ?

Waiting for your reply

Regards

 

[NiteWave]

what's the IOwait value?
#mpstat -P ALL

 

[DoM]

Hi,
right now (09:00 A.M.) with low load (about 3-4) those are values:

12/23/2010

08:54:14 AM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
08:54:14 AM all 5.80 0.08 1.87 2.14 0.01 0.10 0.00 90.00 1731.24
08:54:14 AM 0 7.38 0.11 1.65 1.37 0.00 0.14 0.00 89.35 1072.08
08:54:14 AM 1 4.92 0.10 1.89 5.37 0.01 0.07 0.00 87.64 36.12
08:54:14 AM 2 4.94 0.09 1.62 1.65 0.03 0.03 0.00 91.65 53.55
08:54:14 AM 3 4.05 0.04 1.29 1.32 0.00 0.02 0.00 93.29 2.51
08:54:14 AM 4 5.23 0.07 1.90 1.16 0.00 0.02 0.00 91.62 0.00
08:54:14 AM 5 5.11 0.10 2.08 2.45 0.00 0.03 0.00 90.23 7.54
08:54:14 AM 6 5.19 0.09 2.51 2.53 0.01 0.28 0.00 89.40 374.30
08:54:14 AM 7 9.59 0.05 2.03 1.31 0.01 0.21 0.00 86.80 185.15


I want test also during day when load increase.

What do you think ?


Waiting for your reply

Best regards

 

[NiteWave]

overall average iowait only 2.14, it's very low.
the value is statistics for the time since system startup (boot).

to watch current iowait status:

#mpstat -P ALL 2 5
Display five reports of statistics for all processors at two second intervals.

(from "man mpstat")

 

[DoM]

Hi,
yes i was using directly iostat to check instead of mpstat

Now at 10 A.M. following results:

10:19:13 AM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
10:19:15 AM all 45.62 0.00 5.75 3.00 0.00 0.88 0.00 44.75 4084.08
10:19:15 AM 0 65.17 0.00 9.45 2.99 0.00 5.97 0.00 16.42 3924.38
10:19:15 AM 1 42.21 0.00 4.52 12.06 0.00 1.01 0.00 40.20 151.74
10:19:15 AM 2 57.00 0.00 5.50 1.00 0.00 0.50 0.00 36.00 8.96
10:19:15 AM 3 29.85 0.00 3.48 0.50 0.00 0.00 0.00 66.17 0.00
10:19:15 AM 4 43.22 0.00 7.04 2.01 0.00 0.00 0.00 47.74 0.00
10:19:15 AM 5 45.27 0.00 6.97 3.98 0.00 0.00 0.00 43.78 0.00
10:19:15 AM 6 23.00 0.00 5.50 1.00 0.00 0.00 0.00 70.50 0.00
10:19:15 AM 7 59.00 0.00 4.50 0.50 0.00 0.00 0.00 36.00 0.00

10:19:15 AM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
10:19:17 AM all 32.98 0.00 6.31 7.06 0.06 0.87 0.00 52.72 3910.89
10:19:17 AM 0 57.92 0.00 11.39 2.97 0.50 5.45 0.00 21.78 3680.20
10:19:17 AM 1 36.82 0.00 7.46 22.39 0.00 0.50 0.00 32.84 221.29
10:19:17 AM 2 29.50 0.00 6.00 4.50 0.00 0.00 0.00 60.00 8.91
10:19:17 AM 3 34.00 0.00 5.00 7.50 0.00 0.50 0.00 53.00 0.00
10:19:17 AM 4 27.36 0.00 4.48 4.98 0.00 0.50 0.00 62.69 0.00
10:19:17 AM 5 17.91 0.00 4.98 6.47 0.00 0.50 0.00 70.15 0.00
10:19:17 AM 6 31.50 0.00 5.00 4.50 0.00 0.00 0.00 59.00 0.00
10:19:17 AM 7 27.86 0.00 5.47 3.98 0.00 0.50 0.00 62.19 0.00

10:19:17 AM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
10:19:19 AM all 49.34 0.00 6.75 1.19 0.06 1.50 0.00 41.16 4892.89
10:19:19 AM 0 62.94 0.00 9.64 0.00 0.00 9.64 0.00 17.77 4736.04
10:19:19 AM 1 46.23 0.00 7.54 7.04 0.00 0.50 0.00 38.69 147.21
10:19:19 AM 2 45.05 0.00 7.43 1.49 0.00 0.00 0.00 46.04 9.14
10:19:19 AM 3 39.70 0.00 4.02 0.50 0.00 0.00 0.00 55.78 0.00
10:19:19 AM 4 48.24 0.00 6.53 0.00 0.00 0.00 0.00 45.23 0.00
10:19:19 AM 5 50.50 0.00 5.50 0.00 0.00 0.00 0.00 44.00 0.00
10:19:19 AM 6 49.25 0.00 8.46 0.50 0.00 0.50 0.00 41.29 0.00
10:19:19 AM 7 53.73 0.00 5.47 0.00 0.00 0.00 0.00 40.80 0.00

10:19:19 AM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
10:19:21 AM all 23.69 0.00 3.38 0.88 0.00 0.88 0.00 71.19 3724.00
10:19:21 AM 0 54.50 0.00 7.50 1.50 0.50 6.50 0.00 29.50 3642.00
10:19:21 AM 1 26.00 0.00 3.50 4.00 0.00 0.00 0.00 66.50 73.00
10:19:21 AM 2 22.11 0.00 3.02 0.00 0.00 0.00 0.00 74.87 9.00
10:19:21 AM 3 21.29 0.00 1.98 0.99 0.00 0.50 0.00 75.25 0.00
10:19:21 AM 4 9.05 0.00 2.51 0.50 0.00 0.00 0.00 87.94 0.00
10:19:21 AM 5 27.00 0.00 2.50 0.00 0.00 0.50 0.00 70.00 0.00
10:19:21 AM 6 12.56 0.00 1.51 0.00 0.00 0.00 0.00 85.93 0.00
10:19:21 AM 7 16.16 0.00 4.04 0.00 0.00 0.00 0.00 79.80 0.00

10:19:21 AM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
10:19:23 AM all 25.25 0.00 3.56 1.06 0.06 0.75 0.00 69.31 3379.70
10:19:23 AM 0 62.87 0.00 7.43 5.45 0.50 4.95 0.00 18.81 3238.61
10:19:23 AM 1 11.44 0.00 2.49 1.99 0.00 0.50 0.00 83.58 131.68
10:19:23 AM 2 13.50 0.00 1.00 0.00 0.00 0.00 0.00 85.50 8.91
10:19:23 AM 3 14.00 0.00 3.00 0.00 0.00 0.00 0.00 83.00 0.00
10:19:23 AM 4 14.50 0.00 2.50 0.00 0.00 0.00 0.00 83.00 0.00
10:19:23 AM 5 21.11 0.00 4.52 1.01 0.00 0.00 0.00 73.37 0.00
10:19:23 AM 6 30.35 0.00 3.98 0.00 0.00 0.00 0.00 65.67 0.00
10:19:23 AM 7 33.83 0.00 3.98 0.00 0.00 0.00 0.00 62.19 0.00

Average: CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
Average: all 35.38 0.00 5.15 2.64 0.04 0.97 0.00 55.82 3994.31
Average: 0 60.68 0.00 9.08 2.59 0.30 6.49 0.00 20.86 3840.12
Average: 1 32.50 0.00 5.10 9.50 0.00 0.50 0.00 52.40 145.11
Average: 2 33.47 0.00 4.60 1.40 0.00 0.10 0.00 60.44 8.98
Average: 3 27.74 0.00 3.49 1.90 0.00 0.20 0.00 66.67 0.00
Average: 4 28.46 0.00 4.61 1.50 0.00 0.10 0.00 65.33 0.00
Average: 5 32.37 0.00 4.90 2.30 0.00 0.20 0.00 60.24 0.00
Average: 6 29.37 0.00 4.90 1.20 0.00 0.10 0.00 64.44 0.00
Average: 7 38.16 0.00 4.70 0.90 0.00 0.10 0.00 56.14 0.00


Do you think i had to adjust config settings or do i need to switch directly to SAS ?


Waiting for your reply

Regards

 

[NiteWave]

it's already fast enough, you can compare with other servers. if switch to SAS, it may be even faster, not bad

 

[DoM]

As you can see, right now (12:20 A.M.) there is more I/O

12:20:34 PM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
12:20:36 PM all 27.78 0.31 7.24 21.10 0.06 0.56 0.00 42.95 3688.56
12:20:36 PM 0 18.41 0.00 4.48 16.42 0.00 0.00 0.00 60.70 995.52
12:20:36 PM 1 25.87 0.50 10.95 62.19 0.00 0.50 0.00 0.00 312.94
12:20:36 PM 2 21.89 0.50 4.48 9.95 0.00 0.00 0.00 63.18 8.96
12:20:36 PM 3 12.44 0.50 5.47 18.41 0.00 0.00 0.00 63.18 0.00
12:20:36 PM 4 22.11 0.50 7.04 25.63 0.00 0.00 0.00 44.72 0.00
12:20:36 PM 5 25.76 0.00 7.58 13.13 0.00 0.00 0.00 53.54 0.00
12:20:36 PM 6 38.00 0.50 8.50 17.50 0.00 1.00 0.00 34.50 998.51
12:20:36 PM 7 57.71 0.50 9.95 5.97 0.00 1.99 0.00 23.88 1371.64

12:20:36 PM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
12:20:38 PM all 43.19 0.19 10.25 23.19 0.00 0.81 0.00 22.38 4904.00
12:20:38 PM 0 36.50 0.00 7.50 17.50 0.00 0.50 0.00 38.00 1000.50
12:20:38 PM 1 46.77 0.00 9.95 42.29 0.50 0.50 0.00 0.00 293.00
12:20:38 PM 2 30.65 0.00 12.56 35.18 0.00 0.50 0.00 21.11 9.00
12:20:38 PM 3 41.21 0.00 5.53 11.56 0.00 0.00 0.00 41.71 0.00
12:20:38 PM 4 52.00 0.00 16.00 19.50 0.00 0.50 0.00 12.00 0.00
12:20:38 PM 5 37.00 0.50 11.50 20.50 0.00 0.50 0.00 30.00 0.00
12:20:38 PM 6 42.79 0.00 10.95 28.36 0.50 1.00 0.00 16.42 968.00
12:20:38 PM 7 58.50 0.00 7.50 10.50 0.50 4.00 0.00 19.00 2634.00

12:20:38 PM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
12:20:40 PM all 37.69 0.50 8.50 18.31 0.06 0.62 0.00 34.31 4112.63
12:20:40 PM 0 35.86 0.51 6.57 10.10 0.00 0.00 0.00 46.97 1010.61
12:20:40 PM 1 26.50 2.00 12.00 59.50 0.00 0.00 0.00 0.00 256.06
12:20:40 PM 2 29.15 0.00 6.53 7.54 0.00 0.00 0.00 56.78 9.09
12:20:40 PM 3 34.48 0.49 4.43 3.94 0.00 0.49 0.00 56.16 0.00
12:20:40 PM 4 31.50 0.50 8.50 15.50 0.00 0.00 0.00 44.00 0.00
12:20:40 PM 5 32.67 0.99 9.90 22.28 0.00 0.00 0.00 34.16 0.00
12:20:40 PM 6 42.50 0.50 11.50 22.50 0.00 1.50 0.00 21.50 1204.55
12:20:40 PM 7 68.50 0.00 9.00 4.50 0.00 2.50 0.00 15.50 1632.32

12:20:40 PM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
12:20:42 PM all 33.92 0.31 28.04 17.93 0.00 0.50 0.00 19.30 3268.81
12:20:42 PM 0 49.01 0.50 19.80 4.46 0.00 0.50 0.00 25.74 990.59
12:20:42 PM 1 29.15 0.50 24.12 44.72 0.00 1.01 0.00 0.50 388.61
12:20:42 PM 2 20.40 0.50 37.81 9.45 0.00 0.00 0.00 31.84 8.91
12:20:42 PM 3 35.68 0.00 14.57 6.53 0.00 0.00 0.00 43.22 0.00
12:20:42 PM 4 28.36 0.50 41.79 18.41 0.00 0.00 0.00 10.95 0.00
12:20:42 PM 5 22.00 0.00 40.50 29.00 0.00 0.50 0.00 8.00 0.00
12:20:42 PM 6 22.50 0.00 37.50 26.50 0.00 1.00 0.00 12.50 1005.45
12:20:42 PM 7 63.00 0.50 8.00 6.00 0.00 1.00 0.00 21.50 875.74

12:20:42 PM CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
12:20:44 PM all 28.92 0.69 30.92 14.55 0.00 0.44 0.00 24.48 3492.46
12:20:44 PM 0 29.65 0.50 9.55 14.57 0.00 1.01 0.00 44.72 1005.53
12:20:44 PM 1 31.00 1.50 21.00 45.00 0.00 0.50 0.00 1.00 372.86
12:20:44 PM 2 26.24 0.00 25.25 6.44 0.00 0.50 0.00 41.58 9.05
12:20:44 PM 3 35.68 0.50 22.61 9.55 0.00 0.00 0.00 31.66 0.00
12:20:44 PM 4 23.23 0.00 62.63 3.03 0.00 0.00 0.00 11.11 0.00
12:20:44 PM 5 29.65 1.01 29.15 19.10 0.00 0.00 0.00 21.11 0.00
12:20:44 PM 6 23.12 0.50 47.74 7.04 0.00 1.01 0.00 20.60 1212.06
12:20:44 PM 7 34.50 1.00 29.50 10.50 0.00 0.50 0.00 24.00 892.46

Average: CPU %user %nice %sys %iowait %irq %soft %steal %idle intr/s
Average: all 34.30 0.40 16.99 19.02 0.02 0.59 0.00 28.69 3891.80
Average: 0 33.90 0.30 9.60 12.60 0.00 0.40 0.00 43.20 1000.50
Average: 1 31.87 0.90 15.58 50.75 0.10 0.50 0.00 0.30 324.90
Average: 2 25.65 0.20 17.37 13.67 0.00 0.20 0.00 42.91 9.00
Average: 3 31.87 0.30 10.49 9.99 0.00 0.10 0.00 47.25 0.00
Average: 4 31.46 0.30 27.15 16.43 0.00 0.10 0.00 24.55 0.00
Average: 5 29.43 0.50 19.72 20.82 0.00 0.20 0.00 29.33 0.00
Average: 6 33.80 0.30 23.20 20.40 0.10 1.10 0.00 21.10 1077.10
Average: 7 56.44 0.40 12.79 7.49 0.10 2.00 0.00 20.78 1480.20


Do you think settings are good enough and this is a hardware issue ? Or we can do more optimization with lst ?

Waiting for your reply

Regards

 

[NiteWave]

please adjust
AIO Block Size 1M
to 2M at peak hour, to see if any difference. also, increase memory 8G-->16G, 32G etc can improve the I/O too. the linux kernel will fully use the memory, as disk cache for example.

average iowait is 19.02 during peak hour --- still acceptable I think.

Can any of our experienced customers to share how to improve the disk I/O performance? Thanks in advance.

 

[Tony]

Depending on the workload of the server deadline scheduler can help quite a bit: http://blog.hawkhost.com/2009/02/23/deadline-scheduler/

Also if it's mysql causing the slowness can increase it's tmp table size or even use a ramdisk for temporary tables. This though depends on you having plenty of memory to spare and want to offload intensive disk operations of mysql to memory no matter what.