Eu rodei um servidor Web debian com nextcloud. Toda noite, um trabalho de backup (storeBackup) é agendado e só é bem sucedido em poucos casos. Mas todas as noites a revista está cheia de erros OOM (veja abaixo).
A máquina tem 16G RAM, portanto, eu estou querendo saber, o que é muita RAM consumindo.
Saída de journalctl -p err -e por uma noite:
Jun 21 03:28:01 balkonien kernel: Out of memory: Kill process 4723 (clamd) score 19 or sacrifice child
Jun 21 03:28:01 balkonien kernel: Killed process 4723 (clamd) total-vm:760960kB, anon-rss:49044kB, file-rss:6504kB, shmem-rss:0kB
Jun 21 03:35:00 balkonien kernel: Out of memory: Kill process 22111 (forkit) score 12 or sacrifice child
Jun 21 03:35:00 balkonien kernel: Killed process 22139 (loolkit) total-vm:1488072kB, anon-rss:53768kB, file-rss:0kB, shmem-rss:0kB
Jun 21 03:38:11 balkonien kernel: Out of memory: Kill process 8246 (loolkit) score 13 or sacrifice child
Jun 21 03:38:11 balkonien kernel: Killed process 8246 (loolkit) total-vm:1488072kB, anon-rss:58388kB, file-rss:25816kB, shmem-rss:0kB
Jun 21 03:40:49 balkonien kernel: Out of memory: Kill process 10165 (loolkit) score 13 or sacrifice child
Jun 21 03:40:49 balkonien kernel: Killed process 10165 (loolkit) total-vm:1488072kB, anon-rss:58336kB, file-rss:26224kB, shmem-rss:0kB
Jun 21 03:47:38 balkonien kernel: Out of memory: Kill process 12575 (loolkit) score 13 or sacrifice child
Jun 21 03:47:38 balkonien kernel: Killed process 12575 (loolkit) total-vm:1488072kB, anon-rss:58212kB, file-rss:26224kB, shmem-rss:0kB
Jun 21 03:47:39 balkonien kernel: Out of memory: Kill process 22111 (forkit) score 12 or sacrifice child
Jun 21 03:47:39 balkonien kernel: Killed process 22111 (forkit) total-vm:1487792kB, anon-rss:53880kB, file-rss:2080kB, shmem-rss:0kB
Jun 21 03:48:45 balkonien kernel: Out of memory: Kill process 16652 (forkit) score 17 or sacrifice child
Jun 21 03:48:45 balkonien kernel: Killed process 16783 (loolkit) total-vm:1488068kB, anon-rss:350976kB, file-rss:27500kB, shmem-rss:0kB
Jun 21 04:07:00 balkonien kernel: Out of memory: Kill process 16652 (forkit) score 12 or sacrifice child
Jun 21 04:07:00 balkonien kernel: Killed process 16801 (loolkit) total-vm:1488068kB, anon-rss:349752kB, file-rss:7536kB, shmem-rss:0kB
Jun 21 04:07:00 balkonien kernel: Out of memory: Kill process 16652 (forkit) score 12 or sacrifice child
Jun 21 04:07:00 balkonien kernel: Killed process 16652 (forkit) total-vm:1487720kB, anon-rss:349536kB, file-rss:18508kB, shmem-rss:0kB
Jun 21 04:49:00 balkonien kernel: Out of memory: Kill process 27040 (forkit) score 12 or sacrifice child
Jun 21 04:49:00 balkonien kernel: Killed process 27082 (loolkit) total-vm:1488068kB, anon-rss:102080kB, file-rss:428kB, shmem-rss:0kB
Jun 21 04:49:00 balkonien kernel: Out of memory: Kill process 27040 (forkit) score 12 or sacrifice child
Jun 21 04:49:00 balkonien kernel: Killed process 27088 (loolkit) total-vm:1488068kB, anon-rss:101984kB, file-rss:416kB, shmem-rss:0kB
Jun 21 04:56:00 balkonien kernel: Out of memory: Kill process 12165 (loolkit) score 12 or sacrifice child
Jun 21 04:56:00 balkonien kernel: Killed process 12165 (loolkit) total-vm:1488068kB, anon-rss:100852kB, file-rss:7140kB, shmem-rss:0kB
Jun 21 04:56:00 balkonien kernel: Out of memory: Kill process 27040 (forkit) score 12 or sacrifice child
Jun 21 04:56:00 balkonien kernel: Killed process 27040 (forkit) total-vm:1487720kB, anon-rss:99072kB, file-rss:60kB, shmem-rss:0kB
Jun 21 05:07:01 balkonien kernel: Out of memory: Kill process 13137 (forkit) score 12 or sacrifice child
Jun 21 05:07:01 balkonien kernel: Killed process 13170 (loolkit) total-vm:1488068kB, anon-rss:348584kB, file-rss:840kB, shmem-rss:0kB
Jun 21 05:09:00 balkonien kernel: Out of memory: Kill process 13137 (forkit) score 12 or sacrifice child
Jun 21 05:09:00 balkonien kernel: Killed process 13174 (loolkit) total-vm:1488068kB, anon-rss:340688kB, file-rss:0kB, shmem-rss:0kB
Jun 21 05:09:43 balkonien kernel: Out of memory: Kill process 14902 (loolkit) score 13 or sacrifice child
Jun 21 05:09:43 balkonien kernel: Killed process 14902 (loolkit) total-vm:1488068kB, anon-rss:340476kB, file-rss:26308kB, shmem-rss:0kB
Jun 21 05:54:52 balkonien kernel: Out of memory: Kill process 15401 (loolkit) score 12 or sacrifice child
Jun 21 05:54:52 balkonien kernel: Killed process 15401 (loolkit) total-vm:1488068kB, anon-rss:295604kB, file-rss:1080kB, shmem-rss:0kB
Jun 21 05:56:00 balkonien kernel: Out of memory: Kill process 15519 (loolkit) score 13 or sacrifice child
Jun 21 05:56:00 balkonien kernel: Killed process 15519 (loolkit) total-vm:1488068kB, anon-rss:292004kB, file-rss:25992kB, shmem-rss:0kB
Jun 21 05:56:00 balkonien kernel: Out of memory: Kill process 13137 (forkit) score 12 or sacrifice child
Jun 21 05:56:00 balkonien kernel: Killed process 13137 (forkit) total-vm:1487720kB, anon-rss:291656kB, file-rss:1272kB, shmem-rss:0kB
Jun 21 05:56:07 balkonien kernel: Out of memory: Kill process 27684 (perl) score 10 or sacrifice child
Jun 21 05:56:07 balkonien kernel: Killed process 27688 (perl) total-vm:34744kB, anon-rss:4820kB, file-rss:3848kB, shmem-rss:0kB
Jun 21 05:56:07 balkonien kernel: Out of memory: Kill process 27684 (perl) score 10 or sacrifice child
Jun 21 05:56:07 balkonien kernel: Killed process 27703 (perl) total-vm:34728kB, anon-rss:5364kB, file-rss:3812kB, shmem-rss:0kB
Jun 21 05:56:15 balkonien kernel: Out of memory: Kill process 16287 (forkit) score 13 or sacrifice child
Jun 21 05:56:15 balkonien kernel: Killed process 16287 (forkit) total-vm:1381924kB, anon-rss:248144kB, file-rss:129808kB, shmem-rss:0kB
Jun 21 05:58:18 balkonien kernel: Out of memory: Kill process 16503 (forkit) score 16 or sacrifice child
Jun 21 05:58:18 balkonien kernel: Killed process 16655 (loolkit) total-vm:1488068kB, anon-rss:348072kB, file-rss:25648kB, shmem-rss:0kB
Jun 21 05:59:18 balkonien kernel: Out of memory: Kill process 16503 (forkit) score 16 or sacrifice child
Jun 21 05:59:18 balkonien kernel: Killed process 16675 (loolkit) total-vm:1488068kB, anon-rss:345868kB, file-rss:24916kB, shmem-rss:0kB
Jun 21 06:07:00 balkonien kernel: Out of memory: Kill process 16503 (forkit) score 12 or sacrifice child
Jun 21 06:07:00 balkonien kernel: Killed process 17546 (loolkit) total-vm:1488068kB, anon-rss:342780kB, file-rss:3176kB, shmem-rss:0kB
Jun 21 06:07:00 balkonien kernel: Out of memory: Kill process 16503 (forkit) score 12 or sacrifice child
Jun 21 06:07:00 balkonien kernel: Killed process 16503 (forkit) total-vm:1487720kB, anon-rss:342536kB, file-rss:3128kB, shmem-rss:0kB
Jun 21 06:07:05 balkonien kernel: Out of memory: Kill process 27684 (perl) score 10 or sacrifice child
Jun 21 06:07:05 balkonien kernel: Killed process 27702 (bzip2) total-vm:13624kB, anon-rss:6392kB, file-rss:952kB, shmem-rss:0kB
Jun 21 06:08:45 balkonien systemd[1]: Failed to start systemd-cron daily script service.
Saída de ps faux (referente à tarefa de backup):
samuel 9458 0.0 0.0 26016 2212 ? Ss Jun16 0:00 SCREEN -R
samuel 9459 0.0 0.0 25516 5812 pts/2 Ss Jun16 0:00 \_ /bin/bash
root 14092 0.0 0.0 45336 3804 pts/2 S+ 12:08 0:00 \_ sudo ./storeBackup
root 14093 0.0 0.0 4312 672 pts/2 S+ 12:08 0:00 \_ /bin/sh ./storeBackup
root 14099 38.4 1.9 369208 306664 pts/2 SN+ 12:08 33:13 \_ perl /usr/local/bin/storeBackup.pl -f /etc/storeBackup/storebackup.conf
root 14103 0.0 0.0 34752 7432 pts/2 SN+ 12:08 0:00 \_ perl /opt/storeBackup/lib/stbuLog.pl --readFile /tmp/prLog-54a4c1c781f71013 --out /var/log/storeBackup/storeBackup.log --maxFilelen 1000000 --noOfOldFiles 5
root 14116 1.6 0.0 13624 7500 pts/2 SN+ 12:08 1:22 \_ bzip2
root 14117 0.0 0.0 34732 7328 pts/2 SN+ 12:08 0:00 \_ perl /opt/storeBackup/lib/stbuLog.pl --readFile /tmp/prLog-bd0ccc7b5ac7d895 --out /media/Backup/storeBackup/default/2018.06.21_12.08.27/.storeBackup.log --maxFilelen 0 --noOfOldFiles 1
Saída de iotop (primeiras linhas):
Total DISK READ : 7.06 K/s | Total DISK WRITE : 0.00 B/s
Actual DISK READ: 84.75 K/s | Actual DISK WRITE: 0.00 B/s
TID PRIO USER DISK READ DISK WRITE SWAPIN IO> COMMAND
14099 be/7 root 7.06 K/s 0.00 B/s 0.00 % 0.06 % perl /usr/local/bin/storeBackup.pl -f /etc/storeBackup/storebackup.conf
1 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % init
2 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [kthreadd]
3 be/4 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [ksoftirqd/0]
5 be/0 root 0.00 B/s 0.00 B/s 0.00 % 0.00 % [kworker/0:0H]
Saída de top (primeiras linhas):
top - 13:38:35 up 5 days, 6:38, 1 user, load average: 1,00, 1,15, 1,22
Tasks: 592 total, 2 running, 589 sleeping, 0 stopped, 1 zombie
%Cpu(s): 0,5 us, 25,7 sy, 1,5 ni, 71,8 id, 0,2 wa, 0,0 hi, 0,2 si, 0,0 st
KiB Mem: 15908456 total, 15733624 used, 174832 free, 2555436 buffers
KiB Swap: 12695892 total, 263052 used, 12432840 free. 10882640 cached Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
14099 root 38 18 369208 306664 4920 R 44,5 1,9 34:54.76 perl
47 root 20 0 0 0 0 S 3,3 0,0 67:54.66 kswapd0
18400 root 20 0 26224 3432 2568 R 2,0 0,0 0:00.21 top
7 root 20 0 0 0 0 S 1,6 0,0 29:58.23 rcu_sched
Saída de free -h :
total used free shared buffers cached
Mem: 15G 14G 214M 18M 2,4G 10G
-/+ buffers/cache: 2,2G 12G
Swap: 12G 256M 11G
Tags ram debian linux out-of-memory