Apache forkbombing aleatoriamente

Navegue suas respostas
1

Recentemente, aparentemente do nada, o daemon Apache2 em nosso VPS tem problemas aleatórios com o que parece ser forkbombing.

Primeiras entradas de log antes que as coisas saiam do inferno 

[Tue Jun 25 23:07:18 2013] [error] [client 173.245.51.242] PHP Warning: Invalid argument supplied for foreach() in /var/www/libraries/joomla/access/access.php on line 409 
[Tue Jun 25 23:07:19 2013] [error] [client 108.162.224.23] PHP Warning: array_key_exists() expects parameter 2 to be array, null given in /var/www/libraries/joomla/language/helper.php on line 55

Logs anteriores a quaisquer problemas mostram mais erros de PHP do que eu esperava, desenvolvedores que mantêm o site não encontraram nenhum problema aqui 

[Tue Jun 25 23:18:01 2013] [error] [client 103.22.200.24] PHP Notice: Trying to get property of non-object in /var/www/libraries/joomla/plugin/helper.php on line 123 
[Tue Jun 25 23:18:05 2013] [error] [client 173.245.51.242] PHP Warning: array_key_exists() expects parameter 2 to be array, null given in /var/www/libraries/joomla/language/helper.php on line 55 
[Tue Jun 25 23:18:19 2013] [error] [client 103.22.200.63] PHP Warning: array_key_exists() expects parameter 2 to be array, null given in /var/www/libraries/joomla/language/helper.php on line 55 
[Tue Jun 25 23:18:24 2013] [error] [client 103.22.200.24] PHP Notice: Trying to get property of non-object in /var/www/libraries/joomla/plugin/helper.php on line 123 
[Tue Jun 25 23:18:33 2013] [error] [client 173.245.53.153] PHP Notice: Trying to get property of non-object in /var/www/libraries/joomla/plugin/helper.php on line 123 
[Tue Jun 25 23:18:35 2013] [error] [client 108.162.225.139] PHP Warning: array_key_exists() expects parameter 2 to be array, null given in /var/www/libraries/joomla/language/helper.php on line 55 
[Tue Jun 25 23:18:52 2013] [error] [client 108.162.231.144] PHP Notice: Trying to get property of non-object in /var/www/administrator/includes/application.php on line 276 
[Tue Jun 25 23:18:59 2013] [error] [client 108.162.231.144] PHP Notice: Undefined property: stdClass::$params in /var/www/administrator/includes/application.php on line 277 
[Tue Jun 25 23:19:05 2013] [error] [client 108.162.231.243] PHP Warning: array_key_exists() expects parameter 2 to be array, null given in /var/www/libraries/joomla/language/helper.php on line 55 
[Tue Jun 25 23:19:22 2013] [error] [client 108.162.219.100] PHP Notice: Trying to get property of non-object in /var/www/libraries/joomla/plugin/helper.php on line 123 
[Tue Jun 25 23:19:39 2013] [error] [client 103.22.200.163] PHP Warning: array_key_exists() expects parameter 2 to be array, null given in /var/www/libraries/joomla/language/helper.php on line 55 
[Tue Jun 25 23:19:38 2013] [error] [client 173.245.49.138] PHP Warning: array_key_exists() expects parameter 2 to be array, null given in /var/www/libraries/joomla/language/helper.php on line 55 
[Tue Jun 25 23:19:50 2013] [error] [client 103.22.200.24] PHP Notice: Trying to get property of non-object in /var/www/administrator/includes/application.php on line 276 
[Tue Jun 25 23:19:50 2013] [error] [client 103.22.200.24] PHP Notice: Undefined property: stdClass::$params in /var/www/administrator/includes/application.php on line 277

Eu vi isso acontecer no htop, a memória virtual fica esgotada e começa a ver isso 

[Tue Jun 25 23:54:45 2013] [warn] child process 30976 still did not exit, sending a SIGTERM 
[Tue Jun 25 23:54:45 2013] [warn] child process 30978 still did not exit, sending a SIGTERM 
[Tue Jun 25 23:54:45 2013] [warn] child process 30979 still did not exit, sending a SIGTERM 
[Tue Jun 25 23:54:45 2013] [warn] child process 30829 still did not exit, sending a SIGTERM 
[Tue Jun 25 23:54:45 2013] [warn] child process 30830 still did not exit, sending a SIGTERM 
[Tue Jun 25 23:54:45 2013] [warn] child process 32009 still did not exit, sending a SIGTERM 
[Tue Jun 25 23:54:45 2013] [warn] child process 31929 still did not exit, sending a SIGTERM 
[Tue Jun 25 23:54:45 2013] [warn] child process 31037 still did not exit, sending a SIGTERM

Neste ponto, tudo fica lento e o sistema é basicamente inutilizável 

[Tue Jun 25 23:54:50 2013] [error] could not make child process 30976 exit, attempting to continue anyway 
[Tue Jun 25 23:54:50 2013] [error] could not make child process 30830 exit, attempting to continue anyway 
[Tue Jun 25 23:54:50 2013] [error] could not make child process 31929 exit, attempting to continue anyway 
[Tue Jun 25 23:54:50 2013] [error] could not make child process 30834 exit, attempting to continue anyway 
[Tue Jun 25 23:54:50 2013] [error] could not make child process 31050 exit, attempting to continue anyway 
[Tue Jun 25 23:54:50 2013] [error] could not make child process 30438 exit, attempting to continue anyway 
[Tue Jun 25 23:54:50 2013] [error] could not make child process 31052 exit, attempting to continue anyway 
[Tue Jun 25 23:54:50 2013] [error] could not make child process 32080 exit, attempting to continue anyway 
[Tue Jun 25 23:54:50 2013] [error] could not make child process 30838 exit, attempting to continue anyway 
[Tue Jun 25 23:54:50 2013] [error] could not make child process 32179 exit, attempting to continue anyway

Reiniciar o httpd corrigiu as coisas, embora às vezes o sistema seja muito lento e a VM precise ser reiniciada.

Algumas noções básicas: 

Linux version 2.6.32-358.11.1.el6.x86_64 ([email protected]) (gcc version 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC) ) #1 SMP Wed Jun 12 03:34:52 UTC 2013

[user@server ~]$ httpd -V
Server version: Apache/2.2.15 (Unix)
Server built:   May 16 2012 22:32:26
Server's Module Magic Number: 20051115:24
Server loaded:  APR 1.3.9, APR-Util 1.3.9
Compiled using: APR 1.3.9, APR-Util 1.3.9
Architecture:   64-bit
Server MPM:     Prefork
  threaded:     no
    forked:     yes (variable process count)
Server compiled with....
 -D APACHE_MPM_DIR="server/mpm/prefork"
 -D APR_HAS_SENDFILE
 -D APR_HAS_MMAP
 -D APR_HAVE_IPV6 (IPv4-mapped addresses enabled)
 -D APR_USE_SYSVSEM_SERIALIZE
 -D APR_USE_PTHREAD_SERIALIZE
 -D SINGLE_LISTEN_UNSERIALIZED_ACCEPT
 -D APR_HAS_OTHER_CHILD
 -D AP_HAVE_RELIABLE_PIPED_LOGS
 -D DYNAMIC_MODULE_LIMIT=128
 -D HTTPD_ROOT="/etc/httpd"
 -D SUEXEC_BIN="/usr/sbin/suexec"
 -D DEFAULT_PIDLOG="run/httpd.pid"
 -D DEFAULT_SCOREBOARD="logs/apache_runtime_status"
 -D DEFAULT_LOCKFILE="logs/accept.lock"
 -D DEFAULT_ERRORLOG="logs/error_log"
 -D AP_TYPES_CONFIG_FILE="conf/mime.types"
 -D SERVER_CONFIG_FILE="conf/httpd.conf"

[user@server ~]$ php -v
PHP 5.3.3 (cli) (built: Jul 12 2013 20:35:47)
Copyright (c) 1997-2010 The PHP Group
Zend Engine v2.3.0, Copyright (c) 1998-2010 Zend Technologies

[user@server ~]$ mysql -v
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 264224
Server version: 5.1.69 Source distribution

top - 00:52:46 up 19 days,  2:47,  1 user,  load average: 0.00, 0.00, 0.00
Tasks: 152 total,   1 running, 151 sleeping,   0 stopped,   0 zombie
Cpu(s):  0.3%us,  0.3%sy,  0.0%ni, 99.3%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   1016516k total,   883812k used,   132704k free,    61112k buffers
Swap:  2064376k total,   157088k used,  1907288k free,   227368k cached
  • O servidor está definido para os padrões de max_childs e outros valores de mpm_worker
  • O site está por trás da Cloudflare, portanto, os poucos endereços IP nos registros acima
  • A CPU da VM é 1 Ghz, 1 GB de RAM, não há limites de disco rígido e nenhum dos meus discos virtuais está cheio
  • Tentou mudar para o mpm_prefork, mas teve problemas com o suporte do MySQL
  • Módulos PHP não são nada fora do comum
  • O site executa o webmin como um front-end, sua função é o servidor da Web de front-end, MTA (postfix), MDB (dovecot) e servidor de arquivos (proftpd)

Não sei em que direção devo ir, posso habilitar a depuração do PHP se é algo no Joomla que está causando o problema, mas minha experiência é limitada, então prefiro saber para onde ir primeiro exemplo.

Atualmente, isso aconteceu apenas 3 vezes nos últimos meses e, embora não exista um padrão real, ele não parece estar relacionado à carga, como aconteceu tarde da noite.

Qualquer sugestão seria muito apreciada.

    
por Grant Moritz 29.07.2013 / 17:29

1 resposta

2

Você definitivamente tem um loop PHP que está ficando preso. Verifique estes arquivos:

  • /var/www/libraries/joomla/access/access.php
  • /var/www/libraries/joomla/language/helper.php
  • /var/www/administrator/includes/application.php
por 29.07.2013 / 17:58

Tags

Não é possível adicionar usuário local no sistema usando a autenticação ldap para o samba [closed] Monitoramento da integridade do sistema do servidor não gerenciado usando o WMI / SNMP