Eu ia postar isso no ServerFault originalmente, mas achei que poderia ser um lugar melhor. Deixe-me saber se você acha que existe um lugar melhor para postar essa pergunta.
Eu tenho um aplicativo de espaço de usuário que executa a rede por meio da API do Java NIO (também conhecido como epoll
no Linux) Para fins de demonstração e diagnóstico, eu tenho um utilitário de teste de linha. É basicamente a mesma coisa que iperf
.
Algumas informações sobre o ambiente e como o teste é executado.
- Ubuntu 16.04 Desktop atualizado hoje (4.4.0-34-generic)
- o saldo está fora
- Receptor Intel X504T1 10GbE (ixgbe) < - > Remetente Solarflare 10GbE (sfc)
- Usa 10, 000 soquetes TCP
- Sockets usam as configurações padrão do SO
- O buffer de leitura do espaço do usuário é de 32 KB
- a leitura não ocorre mais que 40hz
O teste de linha consiste em um único cliente que transmite o máximo de informações possível sobre os soquetes TCP.
- cada read () por soquete pode ser chamado mais de uma vez para obter até 98KB por hz (o buffer de 32KB teria que ser lido 3 vezes para atingir o limite máximo)
- Isso significa que em 40hz e no teto de 98KB que read () pode ser chamado até 120 vezes por segundo por conexão; lendo um total de 3, 840KB.
- O Line tester mostra que read () é chamado de um total de cerca de 110.000 vezes por segundo.
O teste de linha saturará totalmente o adaptador 10GbE facilmente usando cerca de 8% softirq
top - 22:04:29 up 51 min, 1 user, load average: 1.31, 1.02, 0.66
Tasks: 258 total, 1 running, 257 sleeping, 0 stopped, 0 zombie
%Cpu(s): 2.2 us, 3.6 sy, 0.0 ni, 85.6 id, 1.1 wa, 0.0 hi, 7.4 si, 0.0 st
KiB Mem : 16378912 total, 12909832 free, 2383088 used, 1085992 buff/cache
KiB Swap: 16721916 total, 16721916 free, 0 used. 13746736 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
4922 jon 20 0 1553556 492552 127160 S 125.0 3.0 0:54.61 firefox
5099 jon 20 0 7212040 218396 16872 S 75.0 1.3 2:59.88 java
3194 root 20 0 722144 163812 134052 S 18.8 1.0 1:25.63 Xorg
4149 jon 20 0 1588648 147848 75344 S 6.2 0.9 0:28.63 compiz
4197 jon 20 0 544660 40600 26804 S 6.2 0.2 0:01.20 indicator-+
5186 jon 20 0 41948 3696 3084 R 6.2 0.0 0:00.01 top
1 root 20 0 119744 5884 3964 S 0.0 0.0 0:00.84 systemd
2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd
3 root 20 0 0 0 0 S 0.0 0.0 5:01.01 ksoftirqd/0
5 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:+
7 root 20 0 0 0 0 S 0.0 0.0 0:01.06 rcu_sched
8 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_bh
9 root rt 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
10 root rt 0 0 0 0 S 0.0 0.0 0:00.04 watchdog/0
11 root rt 0 0 0 0 S 0.0 0.0 0:00.01 watchdog/1
12 root rt 0 0 0 0 S 0.0 0.0 0:00.00 migration/1
13 root 20 0 0 0 0 S 0.0 0.0 0:08.16 ksoftirqd/1
cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7
0: 17 0 0 0 0 0 0 0 IR-IO-APIC 2-edge timer
1: 0 1 0 0 1 0 0 0 IR-IO-APIC 1-edge i8042
5: 0 0 0 0 0 0 0 0 IR-IO-APIC 5-edge parport0
8: 0 0 0 0 0 1 0 0 IR-IO-APIC 8-edge rtc0
9: 0 0 0 0 0 0 0 0 IR-IO-APIC 9-fasteoi acpi
12: 2 0 1 0 1 0 0 0 IR-IO-APIC 12-edge i8042
16: 50 6 2 6 10 0 0 3 IR-IO-APIC 16-fasteoi ehci_hcd:usb1
17: 1138 35 14 24 227 25 35 24 IR-IO-APIC 17-fasteoi snd_hda_intel
19: 0 1 0 0 0 1 0 0 IR-IO-APIC 19-fasteoi firewire_ohci
23: 11 4 10 1 7 0 0 0 IR-IO-APIC 23-fasteoi ehci_hcd:usb2
24: 0 0 0 0 0 0 0 0 DMAR-MSI 0-edge dmar0
27: 4571 1431 1142 812 1286 1442 985 730 IR-PCI-MSI 327680-edge xhci_hcd
28: 26230 3078 1744 1325 6297 2715 1703 1258 IR-PCI-MSI 512000-edge 0000:00:1f.2
29: 754 43 28 30 215 176 129 76 IR-PCI-MSI 2097152-edge eth0-rx-0
30: 0 0 0 0 0 0 0 0 IR-PCI-MSI 2097153-edge eth0-tx-0
31: 0 0 0 0 1 0 0 0 IR-PCI-MSI 2097154-edge eth0
32: 757 64 28 33 205 169 129 66 IR-PCI-MSI 2621440-edge eth1-rx-0
33: 0 0 0 0 0 0 0 0 IR-PCI-MSI 2621441-edge eth1-tx-0
34: 1 0 0 0 0 0 0 0 IR-PCI-MSI 2621442-edge eth1
35: 1042128 233608 58916 16705 1612687 1484813 1121118 630363 IR-PCI-MSI 1048576-edge enp2s0-TxRx-0
36: 858271 736510 372134 165262 1704892 1127381 1265752 767377 IR-PCI-MSI 1048577-edge enp2s0-TxRx-1
37: 816359 711664 426719 192686 1475309 1307882 807216 712562 IR-PCI-MSI 1048578-edge enp2s0-TxRx-2
38: 934786 714007 432100 217627 1905295 1622682 1150693 517990 IR-PCI-MSI 1048579-edge enp2s0-TxRx-3
39: 0 0 0 0 14185366 0 0 0 IR-PCI-MSI 1048580-edge enp2s0-TxRx-4
40: 0 0 0 0 0 14332864 0 0 IR-PCI-MSI 1048581-edge enp2s0-TxRx-5
41: 0 0 0 0 0 0 14617282 0 IR-PCI-MSI 1048582-edge enp2s0-TxRx-6
42: 0 0 0 0 0 0 0 14840029 IR-PCI-MSI 1048583-edge enp2s0-TxRx-7
43: 57 88 47 34 77 64 75 58 IR-PCI-MSI 1048584-edge enp2s0
44: 0 0 0 0 0 13 1 1 IR-PCI-MSI 360448-edge mei_me
45: 246 20 30 4 345 132 128 142 IR-PCI-MSI 442368-edge snd_hda_intel
46: 63933 9794 7233 4753 28843 19323 17678 11191 IR-PCI-MSI 524288-edge nvidia
NMI: 57 43 35 42 103 98 83 76 Non-maskable interrupts
LOC: 300755 258293 257168 289802 373725 262211 218677 196510 Local timer interrupts
SPU: 0 0 0 0 0 0 0 0 Spurious interrupts
PMI: 57 43 35 42 103 98 83 76 Performance monitoring interrupts
IWI: 0 0 0 0 1 0 0 0 IRQ work interrupts
RTR: 0 0 0 0 0 0 0 0 APIC ICR read retries
RES: 7721466 2192716 1958606 3095012 1106115 1189666 309133 169884 Rescheduling interrupts
CAL: 2598 2206 2194 1751 1976 2255 2130 2211 Function call interrupts
TLB: 5450 6659 6103 5640 4352 5128 4535 4470 TLB shootdowns
TRM: 0 0 0 0 0 0 0 0 Thermal event interrupts
THR: 0 0 0 0 0 0 0 0 Threshold APIC interrupts
DFR: 0 0 0 0 0 0 0 0 Deferred Error APIC interrupts
MCE: 0 0 0 0 0 0 0 0 Machine check exceptions
MCP: 11 11 11 11 11 11 11 11 Machine check polls
ERR: 0
MIS: 0
PIN: 0 0 0 0 0 0 0 0 Posted-interrupt notification event
PIW: 0 0 0 0 0 0 0 0 Posted-interrupt wakeup event
Agora, vamos aplicar o controle de taxa ao leitor de soquete.
- O controle da taxa de entrada está definido como 50 KB por conexão
- Que é cerca de 500MB / s, pois temos 10, 000 conexões
O controle de taxa - define a frequência de leitura como 5 Hz, abaixo dos 40 Hz do exemplo anterior.
A frequência do - rate control não está alinhada, o que significa que nem todas as conexões funcionam usando a mesma referência inicial, no entanto, todas são controladas por um único clock.
- o relógio é 40hz; significando que existem 40 oportunidades para que leituras programadas de controle de taxa ocorram.
- durante cada uma dessas leituras de controle de taxa de 5 Hz, o soquete só pode ler até 10 KB. Então, 5 vezes por segundo, ele lê 10 KB do buffer de soquete.
- O Line tester mostra que read () é chamado de um total de cerca de 47, 000 vezes por segundo.
A quantidade de softirq
salta de 8% para 50-65%; o número de interrupções quase triplicou e há 26-58 milhões de interrupções de RES (por núcleo) em comparação com 1-7 milhões antes.
top - 22:31:50 up 1:19, 1 user, load average: 2.30, 2.30, 1.96
Tasks: 259 total, 2 running, 257 sleeping, 0 stopped, 0 zombie
%Cpu(s): 3.3 us, 5.5 sy, 0.0 ni, 41.2 id, 0.0 wa, 0.0 hi, 50.0 si, 0.0 st
KiB Mem : 16378912 total, 11752520 free, 2189080 used, 2437312 buff/cache
KiB Swap: 16721916 total, 16721916 free, 0 used. 12590400 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3 root 20 0 0 0 0 S 82.1 0.0 26:57.43 ksoftirqd/0
5194 jon 20 0 7212040 233488 16720 S 46.2 1.4 12:08.73 java
28 root 20 0 0 0 0 S 40.2 0.0 9:04.84 ksoftirqd/4
33 root 20 0 0 0 0 S 30.9 0.0 7:26.84 ksoftirqd/5
43 root 20 0 0 0 0 R 21.6 0.0 4:26.41 ksoftirqd/7
38 root 20 0 0 0 0 S 21.3 0.0 5:37.16 ksoftirqd/6
4922 jon 20 0 1533388 475124 127784 S 5.6 2.9 2:41.82 firefox
3194 root 20 0 722448 163872 134052 S 5.3 1.0 2:50.84 Xorg
5154 jon 20 0 589896 83876 53964 S 1.7 0.5 0:26.08 plugin-con+
13 root 20 0 0 0 0 S 1.3 0.0 0:42.60 ksoftirqd/1
4548 jon 20 0 5492168 634252 43104 S 1.3 3.9 2:18.86 java
4149 jon 20 0 1604016 169732 75348 S 1.0 1.0 0:52.62 compiz
18 root 20 0 0 0 0 S 0.7 0.0 0:35.31 ksoftirqd/2
23 root 20 0 0 0 0 S 0.3 0.0 0:22.65 ksoftirqd/3
cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7
0: 17 0 0 0 0 0 0 0 IR-IO-APIC 2-edge timer
1: 0 1 0 0 1 0 0 0 IR-IO-APIC 1-edge i8042
5: 0 0 0 0 0 0 0 0 IR-IO-APIC 5-edge parport0
8: 0 0 0 0 0 1 0 0 IR-IO-APIC 8-edge rtc0
9: 0 0 0 0 0 0 0 0 IR-IO-APIC 9-fasteoi acpi
12: 2 0 1 0 1 0 0 0 IR-IO-APIC 12-edge i8042
16: 50 6 2 6 10 0 0 3 IR-IO-APIC 16-fasteoi ehci_hcd:usb1
17: 1138 35 14 24 227 25 35 24 IR-IO-APIC 17-fasteoi snd_hda_intel
19: 0 1 0 0 0 1 0 0 IR-IO-APIC 19-fasteoi firewire_ohci
23: 11 4 10 1 7 0 0 0 IR-IO-APIC 23-fasteoi ehci_hcd:usb2
24: 0 0 0 0 0 0 0 0 DMAR-MSI 0-edge dmar0
27: 6518 1966 1471 1031 4361 3847 2501 1673 IR-PCI-MSI 327680-edge xhci_hcd
28: 26732 3381 1957 1447 6687 3367 2112 1502 IR-PCI-MSI 512000-edge 0000:00:1f.2
29: 930 184 150 114 283 344 232 142 IR-PCI-MSI 2097152-edge eth0-rx-0
30: 0 0 0 0 0 0 0 0 IR-PCI-MSI 2097153-edge eth0-tx-0
31: 0 0 0 0 1 0 0 0 IR-PCI-MSI 2097154-edge eth0
32: 899 234 138 104 277 348 236 143 IR-PCI-MSI 2621440-edge eth1-rx-0
33: 0 0 0 0 0 0 0 0 IR-PCI-MSI 2621441-edge eth1-tx-0
34: 1 0 0 0 0 0 0 0 IR-PCI-MSI 2621442-edge eth1
35: 1339704 330929 97391 31445 2023348 1859243 1369358 782238 IR-PCI-MSI 1048576-edge enp2s0-TxRx-0
36: 1863223 3328011 1764431 788048 2411300 2677922 2540016 1742062 IR-PCI-MSI 1048577-edge enp2s0-TxRx-1
37: 1911973 3426913 2084294 955668 2216702 2894499 2008907 1723010 IR-PCI-MSI 1048578-edge enp2s0-TxRx-2
38: 2064515 3379490 2155421 1093171 2652077 3162801 2369659 1442568 IR-PCI-MSI 1048579-edge enp2s0-TxRx-3
39: 0 0 0 0 23079493 0 0 0 IR-PCI-MSI 1048580-edge enp2s0-TxRx-4
40: 0 0 0 0 0 23379687 0 0 IR-PCI-MSI 1048581-edge enp2s0-TxRx-5
41: 0 0 0 0 0 0 24721093 0 IR-PCI-MSI 1048582-edge enp2s0-TxRx-6
42: 0 0 0 0 0 0 0 25752073 IR-PCI-MSI 1048583-edge enp2s0-TxRx-7
43: 211 430 277 179 142 219 240 197 IR-PCI-MSI 1048584-edge enp2s0
44: 0 0 0 0 0 13 1 1 IR-PCI-MSI 360448-edge mei_me
45: 246 20 30 4 345 132 128 142 IR-PCI-MSI 442368-edge snd_hda_intel
46: 87961 29805 21965 14718 43334 42053 34617 23830 IR-PCI-MSI 524288-edge nvidia
NMI: 218 130 107 105 252 247 225 214 Non-maskable interrupts
LOC: 716630 636798 640606 679852 641275 555921 488433 446196 Local timer interrupts
SPU: 0 0 0 0 0 0 0 0 Spurious interrupts
PMI: 218 130 107 105 252 247 225 214 Performance monitoring interrupts
IWI: 0 0 0 0 3 0 0 0 IRQ work interrupts
RTR: 0 0 0 0 0 0 0 0 APIC ICR read retries
RES: 38554509 4165414 4123561 5839087 2680226 2883656 1297965 812274 Rescheduling interrupts
CAL: 3292 2356 2373 2014 2215 2496 2375 2474 Function call interrupts
TLB: 10997 21211 21364 22716 11757 23899 28023 27646 TLB shootdowns
TRM: 0 0 0 0 0 0 0 0 Thermal event interrupts
THR: 0 0 0 0 0 0 0 0 Threshold APIC interrupts
DFR: 0 0 0 0 0 0 0 0 Deferred Error APIC interrupts
MCE: 0 0 0 0 0 0 0 0 Machine check exceptions
MCP: 17 17 17 17 17 17 17 17 Machine check polls
ERR: 0
MIS: 0
PIN: 0 0 0 0 0 0 0 0 Posted-interrupt notification event
PIW: 0 0 0 0 0 0 0 0 Posted-interrupt wakeup event
Alguém pode explicar por que isso está acontecendo e, possivelmente, como evitá-lo?
Para referência, aqui está top
ao usar o Controle de taxa de saída @ 500MB / s
top - 01:26:15 up 4:13, 1 user, load average: 0.38, 0.31, 1.00
Tasks: 254 total, 1 running, 253 sleeping, 0 stopped, 0 zombie
%Cpu(s): 1.7 us, 3.7 sy, 0.0 ni, 93.3 id, 0.1 wa, 0.0 hi, 1.2 si, 0.0 st
KiB Mem : 16378912 total, 12912528 free, 2209912 used, 1256472 buff/cache
KiB Swap: 16721916 total, 16721916 free, 0 used. 13873312 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
6560 jon 20 0 7212040 204656 16836 S 38.9 1.2 0:21.37 java
3194 root 20 0 871176 206844 175404 S 1.0 1.3 12:11.62 Xorg
4149 jon 20 0 1909092 221972 99348 S 0.7 1.4 3:21.75 compiz
4548 jon 20 0 5879804 662312 45948 S 0.7 4.0 6:48.86 java
3940 jon 20 0 350840 13196 5468 S 0.3 0.1 0:20.41 ibus-daemon
4922 jon 20 0 1779380 686992 145824 S 0.3 4.2 20:38.42 firefox
5827 root 20 0 0 0 0 S 0.3 0.0 0:00.64 kworker/4:1
6341 root 20 0 0 0 0 S 0.3 0.0 0:00.93 kworker/1:2
6539 root 20 0 0 0 0 S 0.3 0.0 0:00.31 kworker/0:2
1 root 20 0 185280 5896 3964 S 0.0 0.0 0:01.01 systemd
2 root 20 0 0 0 0 S 0.0 0.0 0:00.02 kthreadd
3 root 20 0 0 0 0 S 0.0 0.0 107:56.20 ksoftirqd/0
5 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:+
Anexar 2, 500 conexões TCP e usar o controle de taxa vê uma taxa de pacotes internos de tcp de 20K pps; saltando para 5, 000 conexões TCP vê esse número saltar para 105K pps; saltando para 7, 500 TCP faz o salto de saída para 190K pps (estes são apenas os pacotes que reconhecem as leituras - ou presumo) **
2: Colocar a placa Solarflare no servidor e no Intel X540T1 no cliente; Eu vejo a fixação de IRQ para ksoftirqd/0
usando 100% e o total si
para 12.5%
, que é cerca de um núcleo. Com o Solarflare, as interrupções de RES
não excedem 10.000 por núcleo. **
O seguinte é o servidor ao usar a placa Solarflare .. mas apenas cerca de 360-400MB / s está sendo recebido em vez do alvo 500MB / s
top - 11:07:55 up 16 min, 1 user, load average: 1.49, 1.09, 0.62
Tasks: 259 total, 3 running, 256 sleeping, 0 stopped, 0 zombie
%Cpu(s): 1.5 us, 2.5 sy, 0.0 ni, 83.5 id, 0.0 wa, 0.0 hi, 12.5 si, 0.0 st
KiB Mem : 16378912 total, 12294300 free, 2356136 used, 1728476 buff/cache
KiB Swap: 16721916 total, 16721916 free, 0 used. 13067464 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
3 root 20 0 0 0 0 R 99.7 0.0 5:20.82 ksoftirqd/0
4620 jon 20 0 7212040 246176 16712 S 25.6 1.5 1:24.67 java
3241 root 20 0 716936 161772 133628 R 3.3 1.0 0:15.42 Xorg
4659 jon 20 0 654928 36356 27820 S 1.0 0.2 0:00.63 gnome-term+
4103 jon 20 0 1567768 141048 75340 S 0.7 0.9 0:06.44 compiz
4542 jon 20 0 5688204 601804 43040 S 0.7 3.7 1:03.91 java
7 root 20 0 0 0 0 S 0.3 0.0 0:00.93 rcu_sched
4538 root 20 0 0 0 0 S 0.3 0.0 0:00.68 kworker/4:2
1 root 20 0 119844 5980 4028 S 0.0 0.0 0:00.84 systemd
2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd
5 root 0 -20 0 0 0 S 0.0 0.0 0:00.00 kworker/0:+
8 root 20 0 0 0 0 S 0.0 0.0 0:00.00 rcu_bh
9 root rt 0 0 0 0 S 0.0 0.0 0:00.00 migration/0
10 root rt 0 0 0 0 S 0.0 0.0 0:00.02 watchdog/0
11 root rt 0 0 0 0 S 0.0 0.0 0:00.00 watchdog/1
12 root rt 0 0 0 0 S 0.0 0.0 0:00.00 migration/1
13 root 20 0 0 0 0 S 0.0 0.0 0:00.02 ksoftirqd/1
cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7
0: 17 0 0 0 0 0 0 0 IR-IO-APIC 2-edge timer
1: 1 0 0 1 0 0 0 0 IR-IO-APIC 1-edge i8042
5: 0 0 0 0 0 0 0 0 IR-IO-APIC 5-edge parport0
8: 0 0 0 0 0 1 0 0 IR-IO-APIC 8-edge rtc0
9: 0 0 0 0 0 0 0 0 IR-IO-APIC 9-fasteoi acpi
12: 1 0 1 0 1 0 1 0 IR-IO-APIC 12-edge i8042
16: 61 2 1 3 7 2 1 0 IR-IO-APIC 16-fasteoi ehci_hcd:usb1
17: 1166 55 10 19 245 45 13 19 IR-IO-APIC 17-fasteoi snd_hda_intel
19: 0 0 0 0 2 0 0 0 IR-IO-APIC 19-fasteoi firewire_ohci
23: 26 1 2 0 1 2 0 1 IR-IO-APIC 23-fasteoi ehci_hcd:usb2
24: 0 0 0 0 0 0 0 0 DMAR-MSI 0-edge dmar0
27: 1723 170 168 126 1603 166 135 47 IR-PCI-MSI 327680-edge xhci_hcd
28: 24980 1714 933 754 7492 1546 1202 936 IR-PCI-MSI 512000-edge 0000:00:1f.2
29: 298 2 1 7 159 4 6 1 IR-PCI-MSI 2097152-edge eth0-rx-0
30: 0 0 0 0 0 0 0 0 IR-PCI-MSI 2097153-edge eth0-tx-0
31: 1 0 0 0 0 0 0 0 IR-PCI-MSI 2097154-edge eth0
32: 16878 5179 2952 3044 18575 7842 3822 3939 IR-PCI-MSI 1048576-edge enp2s0f0-0
33: 16174 4967 2787 2583 19305 7883 3507 3862 IR-PCI-MSI 1048577-edge enp2s0f0-1
34: 16707 5192 2952 2659 18031 8588 3496 4393 IR-PCI-MSI 1048578-edge enp2s0f0-2
35: 17726 5431 2951 2746 17183 8105 3529 4238 IR-PCI-MSI 1048579-edge enp2s0f0-3
36: 6 1 0 3 6 3 0 1 IR-PCI-MSI 1050624-edge enp2s0f1-0
37: 1 1 0 0 0 0 0 0 IR-PCI-MSI 1050625-edge enp2s0f1-1
38: 1 1 0 0 0 0 0 0 IR-PCI-MSI 1050626-edge enp2s0f1-2
39: 1 1 0 0 0 0 0 0 IR-PCI-MSI 1050627-edge enp2s0f1-3
40: 414 12 9 3 0 14 18 8 IR-PCI-MSI 2621440-edge eth1-rx-0
41: 0 0 0 0 0 0 0 0 IR-PCI-MSI 2621441-edge eth1-tx-0
42: 1 0 0 0 0 0 0 0 IR-PCI-MSI 2621442-edge eth1
43: 0 0 0 0 10 0 5 0 IR-PCI-MSI 360448-edge mei_me
44: 95 26 8 33 398 384 51 16 IR-PCI-MSI 442368-edge snd_hda_intel
45: 17400 1413 1135 806 17781 1714 1401 988 IR-PCI-MSI 524288-edge nvidia
NMI: 37 3 5 3 2 1 1 1 Non-maskable interrupts
LOC: 112894 53399 87350 46718 43552 19663 25436 19705 Local timer interrupts
SPU: 0 0 0 0 0 0 0 0 Spurious interrupts
PMI: 37 3 5 3 2 1 1 1 Performance monitoring interrupts
IWI: 0 0 0 0 0 0 0 0 IRQ work interrupts
RTR: 0 0 0 0 0 0 0 0 APIC ICR read retries
RES: 1808 7668 9364 1244 4161 2554 9171 954 Rescheduling interrupts
CAL: 1900 2028 1497 1984 1862 1931 2118 2004 Function call interrupts
TLB: 1991 2539 3176 2985 3176 2458 1612 2087 TLB shootdowns
TRM: 0 0 0 0 0 0 0 0 Thermal event interrupts
THR: 0 0 0 0 0 0 0 0 Threshold APIC interrupts
DFR: 0 0 0 0 0 0 0 0 Deferred Error APIC interrupts
MCE: 0 0 0 0 0 0 0 0 Machine check exceptions
MCP: 5 5 5 5 5 5 5 5 Machine check polls
ERR: 0
MIS: 0
PIN: 0 0 0 0 0 0 0 0 Posted-interrupt notification event
PIW: 0 0 0 0 0 0 0 0 Posted-interrupt wakeup event