Por que não consigo usar todo o potencial de 100% da CPU?

1

Estou usando os clusters de meu computador para executar um programa MD C, mas não posso usar todo o potencial desses clusters.

Masonódestetem16CPUseeutambémdouapenas15trabalhosparaessesprocessadores.Maseunãopossousartotalmenteessespotenciais.

abaixoéoresultadodepsaux

USERPID%CPU%MEMVSZRSSTTYSTATSTARTTIMECOMMANDroot10.00.0236361624?SsJun150:01/sbin/initroot20.00.000?SJun150:00[kthreadd]root30.00.000?SJun150:00[migration/0]root40.00.000?SJun150:00[ksoftirqd/0]root50.00.000?SJun150:00[migration/0]root60.00.000?SJun150:00[watchdog/0]root70.00.000?SJun150:00[migration/1]root80.00.000?SJun150:00[migration/1]root90.00.000?SJun150:05[ksoftirqd/1]root100.00.000?SJun150:00[watchdog/1]root110.00.000?SJun150:00[migration/2]root120.00.000?SJun150:00[migration/2]root130.00.000?SJun150:01[ksoftirqd/2]root140.00.000?SJun150:02[watchdog/2]root150.00.000?SJun150:00[migration/3]root160.00.000?SJun150:00[migration/3]root170.00.000?SJun150:00[ksoftirqd/3]root180.00.000?SJun150:17[watchdog/3]root190.00.000?SJun150:00[migration/4]root200.00.000?SJun150:00[migration/4]root210.00.000?SJun150:00[ksoftirqd/4]root220.00.000?SJun150:02[watchdog/4]root230.00.000?SJun150:00[migration/5]root240.00.000?SJun150:00[migration/5]root250.00.000?SJun150:00[ksoftirqd/5]root260.00.000?SJun150:00[watchdog/5]root270.00.000?SJun150:00[migration/6]root280.00.000?SJun150:00[migration/6]root290.00.000?SJun150:00[ksoftirqd/6]root300.00.000?SJun150:00[watchdog/6]root310.00.000?SJun150:00[migration/7]root320.00.000?SJun150:00[migration/7]root330.00.000?SJun150:00[ksoftirqd/7]root340.00.000?SJun150:00[watchdog/7]root350.00.000?SJun150:00[migration/8]root360.00.000?SJun150:00[migration/8]root370.00.000?SJun150:01[ksoftirqd/8]root380.00.000?SJun150:00[watchdog/8]root390.00.000?SJun150:00[migration/9]root400.00.000?SJun150:00[migration/9]root410.00.000?SJun150:00[ksoftirqd/9]root420.00.000?SJun150:00[watchdog/9]root430.00.000?SJun150:00[migration/10]root440.00.000?SJun150:00[migration/10]root450.00.000?SJun150:00[ksoftirqd/10]root460.00.000?SJun150:00[watchdog/10]root470.00.000?SJun150:00[migration/11]root480.00.000?SJun150:00[migration/11]root490.00.000?SJun150:00[ksoftirqd/11]root500.00.000?SJun150:00[watchdog/11]root510.00.000?SJun150:00[migration/12]root520.00.000?SJun150:00[migration/12]root530.00.000?SJun150:00[ksoftirqd/12]root540.00.000?SJun150:00[watchdog/12]root550.00.000?SJun150:00[migration/13]root560.00.000?SJun150:00[migration/13]root570.00.000?SJun150:00[ksoftirqd/13]root580.00.000?SJun150:00[watchdog/13]root590.00.000?SJun150:00[migration/14]root600.00.000?SJun150:00[migration/14]root610.00.000?SJun150:00[ksoftirqd/14]root620.00.000?SJun150:00[watchdog/14]root630.00.000?SJun150:00[migration/15]root640.00.000?SJun150:00[migration/15]root650.00.000?SJun150:00[ksoftirqd/15]root660.00.000?SJun150:00[watchdog/15]root670.00.000?SJun150:00[events/0]root680.00.000?SJun150:16[events/1]root690.00.000?SJun150:26[events/2]root700.00.000?SJun151:24[events/3]root710.00.000?SJun154:17[events/4]root720.00.000?SJun154:11[events/5]root730.00.000?SJun150:31[events/6]root740.00.000?SJun152:34[events/7]root750.00.000?SJun151:11[events/8]root760.00.000?SJun1511:39[events/9]root770.00.000?SJun151:15[events/10]root780.00.000?SJun150:01[events/11]root790.00.000?SJun150:00[events/12]root800.00.000?SJun150:01[events/13]root810.00.000?SJun150:00[events/14]root820.00.000?SJun150:00[events/15]root830.00.000?SJun150:00[cpuset]root840.00.000?SJun150:00[khelper]root850.00.000?SJun150:00[netns]root860.00.000?SJun150:00[async/mgr]root870.00.000?SJun150:00[pm]root880.00.000?SJun150:09[sync_supers]root890.00.000?SJun150:07[bdi-default]root900.00.000?SJun150:00[kintegrityd/0]root910.00.000?SJun150:00[kintegrityd/1]root920.00.000?SJun150:00[kintegrityd/2]root930.00.000?SJun150:00[kintegrityd/3]root940.00.000?SJun150:00[kintegrityd/4]root950.00.000?SJun150:00[kintegrityd/5]root960.00.000?SJun150:00[kintegrityd/6]root970.00.000?SJun150:00[kintegrityd/7]root980.00.000?SJun150:00[kintegrityd/8]root990.00.000?SJun150:00[kintegrityd/9]root1000.00.000?SJun150:00[kintegrityd/10]root1010.00.000?SJun150:00[kintegrityd/11]root1020.00.000?SJun150:00[kintegrityd/12]root1030.00.000?SJun150:00[kintegrityd/13]root1040.00.000?SJun150:00[kintegrityd/14]root1050.00.000?SJun150:00[kintegrityd/15]root1060.00.000?SJun150:00[kblockd/0]root1070.00.000?SJun150:00[kblockd/1]root1080.00.000?SJun150:01[kblockd/2]root1090.00.000?SJun150:00[kblockd/3]root1100.00.000?SJun150:00[kblockd/4]root1110.00.000?SJun150:00[kblockd/5]root1120.00.000?SJun150:00[kblockd/6]root1130.00.000?SJun150:00[kblockd/7]root1140.00.000?SJun150:00[kblockd/8]root1150.00.000?SJun150:00[kblockd/9]root1160.00.000?SJun150:00[kblockd/10]root1170.00.000?SJun150:00[kblockd/11]root1180.00.000?SJun150:00[kblockd/12]root1190.00.000?SJun150:00[kblockd/13]root1200.00.000?SJun150:00[kblockd/14]root1210.00.000?SJun150:00[kblockd/15]root1220.00.000?SJun150:00[kacpid]root1230.00.000?SJun150:00[kacpi_notify]root1240.00.000?SJun150:00[kacpi_hotplug]root1250.00.000?SJun150:00[ata/0]root1260.00.000?SJun150:00[ata/1]root1270.00.000?SJun150:00[ata/2]root1280.00.000?SJun150:00[ata/3]root1290.00.000?SJun150:00[ata/4]root1300.00.000?SJun150:00[ata/5]root1310.00.000?SJun150:00[ata/6]root1320.00.000?SJun150:00[ata/7]root1330.00.000?SJun150:00[ata/8]root1340.00.000?SJun150:00[ata/9]root1350.00.000?SJun150:00[ata/10]root1360.00.000?SJun150:00[ata/11]root1370.00.000?SJun150:00[ata/12]root1380.00.000?SJun150:00[ata/13]root1390.00.000?SJun150:00[ata/14]root1400.00.000?SJun150:00[ata/15]root1410.00.000?SJun150:00[ata_aux]root1420.00.000?SJun150:00[ksuspend_usbd]root1430.00.000?SJun150:00[khubd]root1440.00.000?SJun150:00[kseriod]root1450.00.000?SJun150:00[md/0]root1460.00.000?SJun150:00[md/1]root1470.00.000?SJun150:00[md/2]root1480.00.000?SJun150:00[md/3]root1490.00.000?SJun150:00[md/4]root1500.00.000?SJun150:00[md/5]root1510.00.000?SJun150:00[md/6]root1520.00.000?SJun150:00[md/7]root1530.00.000?SJun150:00[md/8]root1540.00.000?SJun150:00[md/9]root1550.00.000?SJun150:00[md/10]root1560.00.000?SJun150:00[md/11]root1570.00.000?SJun150:00[md/12]root1580.00.000?SJun150:00[md/13]root1590.00.000?SJun150:00[md/14]root1600.00.000?SJun150:00[md/15]root1610.00.000?SJun150:00[md_misc/0]root1620.00.000?SJun150:00[md_misc/1]root1630.00.000?SJun150:00[md_misc/2]root1640.00.000?SJun150:00[md_misc/3]root1650.00.000?SJun150:00[md_misc/4]root1660.00.000?SJun150:00[md_misc/5]root1670.00.000?SJun150:00[md_misc/6]root1680.00.000?SJun150:00[md_misc/7]root1690.00.000?SJun150:00[md_misc/8]root1700.00.000?SJun150:00[md_misc/9]root1780.00.000?SJun150:00[kswapd0]root1790.00.000?SJun150:00[kswapd1]root1800.00.000?SNJun150:00[ksmd]root1810.00.000?SNJun150:10[khugepaged]root1820.00.000?SJun150:00[aio/0]root1830.00.000?SJun150:00[aio/1]root1840.00.000?SJun150:00[aio/2]root1850.00.000?SJun150:00[aio/3]root1860.00.000?SJun150:00[aio/4]root1870.00.000?SJun150:00[aio/5]root1880.00.000?SJun150:00[aio/6]root1890.00.000?SJun150:00[aio/7]root1900.00.000?SJun150:00[aio/8]root1910.00.000?SJun150:00[aio/9]root1920.00.000?SJun150:00[aio/10]root1930.00.000?SJun150:00[aio/11]root1940.00.000?SJun150:00[aio/12]root1950.00.000?SJun150:00[aio/13]root1960.00.000?SJun150:00[aio/14]root1970.00.000?SJun150:00[aio/15]root1980.00.000?SJun150:00[crypto/0]root1990.00.000?SJun150:00[crypto/1]root2000.00.000?SJun150:00[crypto/2]root2010.00.000?SJun150:00[crypto/3]root2020.00.000?SJun150:00[crypto/4]root2030.00.000?SJun150:00[crypto/5]root2040.00.000?SJun150:00[crypto/6]root2050.00.000?SJun150:00[crypto/7]root2060.00.000?SJun150:00[crypto/8]root2070.00.000?SJun150:00[crypto/9]root2080.00.000?SJun150:00[crypto/10]root2090.00.000?SJun150:00[crypto/11]root2100.00.000?SJun150:00[crypto/12]root2110.00.000?SJun150:00[crypto/13]root2120.00.000?SJun150:00[crypto/14]root2130.00.000?SJun150:00[crypto/15]root2180.00.000?SJun150:00[kthrotld/0]root2190.00.000?SJun150:00[kthrotld/1]root2200.00.000?SJun150:00[kthrotld/2]root2210.00.000?SJun150:00[kthrotld/3]root2220.00.000?SJun150:00[kthrotld/4]root2230.00.000?SJun150:00[kthrotld/5]root2240.00.000?SJun150:00[kthrotld/6]root2250.00.000?SJun150:00[kthrotld/7]root2260.00.000?SJun150:00[kthrotld/8]root2270.00.000?SJun150:00[kthrotld/9]root2280.00.000?SJun150:00[kthrotld/10]root2290.00.000?SJun150:00[kthrotld/11]root2300.00.000?SJun150:00[kthrotld/12]root2310.00.000?SJun150:00[kthrotld/13]root2320.00.000?SJun150:00[kthrotld/14]root2330.00.000?SJun150:00[kthrotld/15]root2460.00.000?SJun150:00[kpsmoused]root2470.00.000?SJun150:00[usbhid_resumer]root2770.00.000?SJun150:00[kstriped]root6110.00.000?SJun150:00[scsi_eh_0]root6120.00.000?SJun150:00[scsi_eh_1]root6130.00.000?SJun150:00[scsi_eh_2]root6140.00.000?SJun150:00[scsi_eh_3]root6150.00.000?SJun150:00[scsi_eh_4]root6160.00.000?SJun150:00[scsi_eh_5]root7630.00.000?SJun150:00[scsi_eh_6]root7640.00.000?SJun150:00[scsi_wq_6]root7690.00.000?SJun150:00[scsi_eh_7]root7700.00.000?SJun150:00[fw_event0]root7730.00.000?SJun150:54[poll_0_status]root8180.00.000?SJun150:12[jbd2/sda3-8]root8190.00.000?SJun150:00[ext4-dio-unwrit]root8200.00.000?SJun150:00[ext4-dio-unwrit]root8210.00.000?SJun150:00[ext4-dio-unwrit]root8220.00.000?SJun150:00[ext4-dio-unwrit]root8230.00.000?SJun150:00[ext4-dio-unwrit]root8240.00.000?SJun150:00[ext4-dio-unwrit]root8250.00.000?SJun150:00[ext4-dio-unwrit]root8260.00.000?SJun150:00[ext4-dio-unwrit]root8270.00.000?SJun150:00[ext4-dio-unwrit]root8280.00.000?SJun150:00[ext4-dio-unwrit]root8290.00.000?SJun150:00[ext4-dio-unwrit]root8300.00.000?SJun150:00[ext4-dio-unwrit]root8310.00.000?SJun150:00[ext4-dio-unwrit]root8320.00.000?SJun150:00[ext4-dio-unwrit]root8330.00.000?SJun150:00[ext4-dio-unwrit]root8340.00.000?SJun150:00[ext4-dio-unwrit]root9210.00.0116721652?S<sJun150:00/sbin/udevd-droot12920.00.000?SJun151:03[edac-poller]root18920.00.000?SJun150:00[mlx4]root18940.00.000?SJun150:00[mlx4_opreq]root18950.00.000?SJun150:12[flush-8:0]root18960.00.000?SJun150:12[mlx4_sense]root19050.00.000?SJun150:00[mlx4_en]root20770.00.0116841664?S<Jun150:00/sbin/udevd-droot21250.00.000?SJun150:00[jbd2/sda1-8]root21260.00.000?SJun150:00[ext4-dio-unwrit]root21270.00.000?SJun150:00[ext4-dio-unwrit]root21280.00.000?SJun150:00[ext4-dio-unwrit]root21290.00.000?SJun150:00[ext4-dio-unwrit]root21300.00.000?SJun150:00[ext4-dio-unwrit]root21310.00.000?SJun150:00[ext4-dio-unwrit]root21320.00.000?SJun150:00[ext4-dio-unwrit]root21330.00.000?SJun150:00[ext4-dio-unwrit]root21340.00.000?SJun150:00[ext4-dio-unwrit]root21350.00.000?SJun150:00[ext4-dio-unwrit]root21360.00.000?SJun150:00[ext4-dio-unwrit]root21370.00.000?SJun150:00[ext4-dio-unwrit]root21380.00.000?SJun150:00[ext4-dio-unwrit]root21390.00.000?SJun150:00[ext4-dio-unwrit]root21400.00.000?SJun150:00[ext4-dio-unwrit]root21410.00.000?SJun150:00[ext4-dio-unwrit]root21420.00.000?SJun150:00[jbd2/sda5-8]root21430.00.000?SJun150:00[ext4-dio-unwrit]root21440.00.000?SJun150:00[ext4-dio-unwrit]root21450.00.000?SJun150:00[ext4-dio-unwrit]root21460.00.000?SJun150:00[ext4-dio-unwrit]root21470.00.000?SJun150:00[ext4-dio-unwrit]root21480.00.000?SJun150:00[ext4-dio-unwrit]root21490.00.000?SJun150:00[ext4-dio-unwrit]root21500.00.000?SJun150:00[ext4-dio-unwrit]root21510.00.000?SJun150:00[ext4-dio-unwrit]root21520.00.000?SJun150:00[ext4-dio-unwrit]root21530.00.000?SJun150:00[ext4-dio-unwrit]root21540.00.000?SJun150:00[ext4-dio-unwrit]root21550.00.000?SJun150:00[ext4-dio-unwrit]root21560.00.000?SJun150:00[ext4-dio-unwrit]root21570.00.000?SJun150:00[ext4-dio-unwrit]root21580.00.000?SJun150:00[ext4-dio-unwrit]root22020.00.000?SJun150:00[kauditd]root22500.00.000?SJun150:00[mthcacatas]root22530.00.000?SJun150:00[mlx4_ib]root22540.00.000?SJun150:00[ib_mad1]root22610.00.000?SJun150:00[iw_cxgb3]root22650.00.000?SJun150:00[nesewq]root22660.00.000?SJun150:00[nesdwq]root22700.00.0116681660?S<Jun150:00/sbin/udevd-droot22730.00.000?SJun150:00[ib_mcast]root22740.00.000?SJun150:00[ib_inform]root22750.00.000?SJun150:00[local_sa]root22760.00.000?SJun150:00[ib_cm/0]root22770.00.000?SJun150:00[ib_cm/1]root22780.00.000?SJun150:00[ib_cm/2]root22790.00.000?SJun150:00[ib_cm/3]root22800.00.000?SJun150:00[ib_cm/4]root22810.00.000?SJun150:00[ib_cm/5]root22820.00.000?SJun150:00[ib_cm/6]root22830.00.000?SJun150:00[ib_cm/7]root22840.00.000?SJun150:00[ib_cm/8]root22850.00.000?SJun150:00[ib_cm/9]root22860.00.000?SJun150:00[ib_cm/10]root22870.00.000?SJun150:00[ib_cm/11]root22880.00.000?SJun150:00[ib_cm/12]root22890.00.000?SJun150:00[ib_cm/13]root22900.00.000?SJun150:00[ib_cm/14]root22910.00.000?SJun150:00[ib_cm/15]root22920.00.000?SJun150:51[ipoib]root22930.00.000?SJun153:17[ipoib_auto_mode]root23580.00.000?SJun150:01[ib_addr]root23590.00.000?SJun150:00[iw_cm_wq]root23600.00.000?SJun150:00[rdma_cm]root25940.00.093224896?S<slJun150:04auditd15526640.00.0607889360?SJun150:11/usr/libexec/systemtap/stap-serverd-r2.6.32-220.el6.x86_64-ax86_64--log=/var/log/stap-server/logroot26910.00.02508561560?SlJun150:01/sbin/rsyslogd-i/var/run/syslogd.pid-c4root27140.00.000?SJun150:00[kondemand/0]root27150.00.000?SJun150:24[kondemand/1]root27160.00.000?SJun150:48[kondemand/2]root27170.00.000?SJun152:03[kondemand/3]root27180.00.000?SJun155:41[kondemand/4]root27190.00.000?SJun151:25[kondemand/5]root27200.00.000?SJun151:26[kondemand/6]root27210.00.000?SJun152:09[kondemand/7]root27220.00.000?SJun150:40[kondemand/8]root27230.00.000?SJun151:28[kondemand/9]root27240.00.000?SJun150:09[kondemand/10]root27250.00.000?SJun150:00[kondemand/11]root27260.00.000?SJun150:00[kondemand/12]root27270.00.000?SJun150:00[kondemand/13]root27280.00.000?SJun150:00[kondemand/14]root27290.00.000?SJun150:00[kondemand/15]root27400.00.09204644?SsJun1517:34irqbalancerpc27540.00.019024984?SsJun150:02rpcbindrpcuser27720.00.0232001204?SsJun150:00rpc.statdroot28000.00.000?SJun153:22[rpciod/0]root28010.00.000?SJun153:22[rpciod/1]root28020.00.000?SJun153:21[rpciod/2]root28030.00.000?SJun153:18[rpciod/3]root28040.00.000?SJun153:16[rpciod/4]root28050.00.000?SJun153:13[rpciod/5]root28060.00.000?SJun153:10[rpciod/6]root28070.00.000?SJun153:08[rpciod/7]root28080.00.000?SJun1526:27[rpciod/8]root28090.00.000?SJun153:58[rpciod/9]root28100.00.000?SJun153:34[rpciod/10]root28110.00.000?SJun153:17[rpciod/11]root28120.00.000?SJun153:11[rpciod/12]root28130.00.000?SJun153:18[rpciod/13]root28140.00.000?SJun153:07[rpciod/14]

Comoasaídaexcedeolimitedowebsite,euarmazenooarquivoexatonoGoogleDrive,seissopuderserútil.

saída de ps

Então, existem alguns métodos que podem resolver este problema para que eu possa usar todo o potencial das CPUs?

    
por sikisis 07.08.2015 / 12:25

2 respostas

8

Como mencionado em um comentário e sem ver nenhum de seu código ou outras informações (o que não seria aqui, de qualquer maneira), tudo o que posso dizer é que seu programa parece estar com o IO vinculado.

Os meios, enquanto seus cálculos podem usar mais da sua CPU, eles têm que esperar nos dados e gastar muitos ciclos de espera, em vez de calcular. Isso pode ser devido à maneira como você escreve código (otimização de loop, vetorização, etc). Um problema comum é acessar seus dados de uma maneira que cause muitos erros de cache. Seus múltiplos núcleos também podem compartilhar um cache L3 e, se todos eles estiverem trabalhando em dados diferentes, você provavelmente terá muitas falhas. Buscando memória da DRAM principal é ordens de magnitude mais lenta do que a memória cache on-die. Se seus dados vierem dos discos, você também terá que lidar com a latência de leitura de disco, além da latência de DRAM. Se os dados estiverem na rede ethernet ou em alguma outra interconexão, você também terá que considerar a latência nessas leituras. Você também pode incorrer em muita espera em gravações em disco, especialmente se estiver escrevendo com frequência em pequenos blocos, em vez de armazenar em buffer para gravações grandes.

Em suma, há muitas considerações de desempenho de IO que limitam sua capacidade de manter 100% de uso da CPU em seus núcleos. Minha recomendação é criar um perfil do seu código, descobrir suas limitações de IO e garantir que seu código seja eficiente e avance a partir daí.

    
por 07.08.2015 / 16:37
2

Isso significa que seus aplicativos são vinculados a E / S - seu hardrive / rede etc. não consegue acompanhar seu processador e, consequentemente, o processador gasta muito tempo aguardando dados de E / S, não usando todo o seu potencial.

Se o IO de que seus aplicativos dependem for da rede e você não estiver baixando na largura de banda total, poderá aumentar a eficiência adicionando mais processos / threads, supondo que cada processo / thread adicione um canal de download que aumentará seu uso de largura de banda para o seu máximo.

    
por 07.08.2015 / 16:33