Por que a carga do servidor é significativa quando o I / O da CPU e a memória são usados apenas levemente

Question

Eu tenho um servidor muito poderoso.

32 núcleos e 64G de memória com SSD

Aqui está o resultado do iostat -xk 5

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
          13.43    0.00    8.68    1.74    0.00   76.15

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await  svctm  %util
sda               0.77     0.00    0.50    0.00     1.97     0.00     7.94     0.00    0.10   0.10   0.00
sdb            1487.51   562.92 3596.86  299.21 21888.08  3448.81    13.01     9.94    2.55   0.10  40.26
sdd               5.03    57.98   65.03   86.51   679.05   578.13    16.59     0.28    1.82   0.12   1.80
sdc               2.21     0.00    0.82    0.00    12.24     0.00    29.67     0.00    0.27   0.25   0.02

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
          12.98    0.00    8.46    0.41    0.00   78.15

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await  svctm  %util
sda               0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
sdb              35.80   377.40  656.80  237.00  3714.40  2457.60    13.81     1.34    1.50   0.17  14.96
sdd               0.00   106.20    2.00  143.80     8.00  1000.80    13.84     0.43    2.96   0.12   1.80
sdc               0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00

avg-cpu:  %user   %nice %system %iowait  %steal   %idle
          10.96    0.00    7.11    0.16    0.00   81.78

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await  svctm  %util
sda               0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00
sdb              10.60   190.60  429.80  220.40  2637.60  1644.00    13.17     0.50    0.76   0.12   7.94
sdd               0.00    58.40   17.80   69.80   345.60   512.80    19.60     0.23    2.65   0.12   1.04
sdc               0.00     0.00    0.00    0.00     0.00     0.00     0.00     0.00    0.00   0.00   0.00

e topo

root@host [/]# top
top - 18:20:27 up 15 min,  1 user,  load average: 4.81, 5.63, 3.90
Tasks: 802 total,   3 running, 786 sleeping,   0 stopped,  13 zombie
Cpu(s): 13.4%us,  8.4%sy,  0.0%ni, 76.2%id,  1.7%wa,  0.0%hi,  0.2%si,  0.0%st
Mem:  15981924k total,  1705512k used, 14276412k free,   457408k buffers
Swap:  4194296k total,        0k used,  4194296k free,   455800k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
14351 investgr  20   0     0    0    0 Z 28.3  0.0   0:00.15 php <defunct>
14377 sexsmovi  20   0     0    0    0 Z 24.5  0.0   0:00.13 php <defunct>
14359 hearsttr  20   0     0    0    0 Z 20.7  0.0   0:00.11 php <defunct>
14348 sexsmovi  20   0     0    0    0 Z 18.8  0.0   0:00.10 php <defunct>
14350 sexsmovi  20   0 35420  11m 4668 S 17.0  0.1   0:00.09 php
14362 cuntschi  20   0     0    0    0 Z 17.0  0.0   0:00.09 php <defunct>
14367 investgr  20   0     0    0    0 Z 15.1  0.0   0:00.08 php <defunct>
14366 investgr  20   0 33568 9064 4472 S 13.2  0.1   0:00.07 php
14372 investgr  20   0 33568 8904 4420 S 13.2  0.1   0:00.07 php
14373 investgr  20   0 33568 9028 4456 S 13.2  0.1   0:00.07 php
14376 hearsttr  20   0     0    0    0 Z 11.3  0.0   0:00.06 php <defunct>
14336 hearsttr  20   0 34656  10m 4684 S  9.4  0.1   0:00.11 php
14340 root      20   0  3096 1444  760 R  5.7  0.0   0:00.05 top
14388 sexsmovi  20   0 30720 5996 3896 R  5.7  0.0   0:00.03 php
 1161 root      20   0     0    0    0 S  1.9  0.0   0:00.40 flush-8:48
 1959 root      20   0     0    0    0 S  1.9  0.0   0:01.22 kondemand/1
 5546 nobody    20   0  153m  78m 1568 S  1.9  0.5   0:00.25 httpd
 5842 nobody    20   0  153m  78m 1544 S  1.9  0.5   0:00.26 httpd
 5843 nobody    20   0  153m  78m 1544 S  1.9  0.5   0:00.23 httpd
 6012 nobody    20   0  153m  78m 1568 S  1.9  0.5   0:00.32 httpd
 6041 nobody    20   0  153m  78m 1536 S  1.9  0.5   0:00.19 httpd
 9143 nobody    20   0  153m  78m 1556 S  1.9  0.5   0:00.26 httpd
 9158 nobody    20   0  153m  78m 1536 S  1.9  0.5   0:00.17 httpd
11916 nobody    20   0  153m  78m 1552 S  1.9  0.5   0:00.07 httpd
12508 nobody    20   0  153m  78m 1504 S  1.9  0.5   0:00.03 httpd
12523 nobody    20   0  153m  78m 1504 S  1.9  0.5   0:00.04 httpd
14051 investgr  20   0 33568 9360 4612 S  1.9  0.1   0:00.08 php
14076 investgr  20   0 33568 9328 4640 S  1.9  0.1   0:00.08 php
14306 hearsttr  20   0 34392  10m 4636 S  1.9  0.1   0:00.09 php
16554 nobody    20   0  153m  78m 1572 S  1.9  0.5   0:01.11 httpd
20263 nobody    20   0  153m  78m 1584 S  1.9  0.5   0:00.99 httpd
26535 nobody    20   0  153m  78m 1576 S  1.9  0.5   0:00.64 httpd
26738 nobody    20   0  153m  78m 1584 S  1.9  0.5   0:00.67 httpd
26740 nobody    20   0  153m  78m 1580 S  1.9  0.5   0:00.68 httpd
26963 nobody    20   0  153m  78m 1572 S  1.9  0.5   0:00.78 httpd
30587 nobody    20   0  153m  78m 1568 S  1.9  0.5   0:01.78 httpd
    1 root      20   0  2900 1436 1212 S  0.0  0.0   0:01.99 init
    2 root      20   0     0    0    0 S  0.0  0.0   0:00.00 kthreadd
    3 root      RT   0     0    0    0 S  0.0  0.0   0:00.39 migration/0

    

Answer 1

Sua CPU não está tão carregada. Está mostrando apenas ~ 13%. Seu HDD (/ dev / sdb) está um pouco ocupado:

Device:         rrqm/s   wrqm/s     r/s     w/s    rkB/s    wkB/s avgrq-sz avgqu-sz   await  svctm  %util
sdb              35.80   377.40  656.80  237.00  3714.40  2457.60    13.81     1.34    1.50   0.17  14.96

A carga reportada no topo @ ~ 4.81 apenas significa que há uma média de ~ 4 processos em um estado pronto para execução e estão aguardando que os recursos do sistema fiquem disponíveis para que possam ser executados.

Os processos "defuntos" são provavelmente o seu problema, eles provavelmente estão bloqueando algum recurso para todos os outros processos.