Acelerar a criação de arquivos temporários no linux

Navegue suas respostas
1

Eu tenho um aplicativo para linux que eu uso com bastante frequência, que adora fazer muitos arquivos temporários enquanto faz seu trabalho. Basicamente, seu software que roteia placas de circuito automaticamente e durante seus procedimentos de roteamento, cria e exclui muitos arquivos temporários.

Minha saída de strace ao procurar pelo comando open se parece com isso: 

  12:09:52.677525 open("/tmp/e000134A.tmp", O_RDWR|O_CREAT|O_APPEND, 0666) = 8
  12:09:52.677777 open("/tmp/e000134A.tmp", O_RDWR|O_CREAT|O_TRUNC, 0666) = 8
  12:09:52.871198 open("/tmp/e000134B.tmp", O_RDWR|O_CREAT|O_APPEND, 0666) = 8
  12:09:52.871342 open("/tmp/e000134B.tmp", O_RDWR|O_CREAT|O_TRUNC, 0666) = 8
  12:09:53.092441 open("/tmp/e000134C.tmp", O_RDWR|O_CREAT|O_APPEND, 0666) = 8
  12:09:53.092563 open("/tmp/e000134C.tmp", O_RDWR|O_CREAT|O_TRUNC, 0666) = 8
  12:09:53.292663 open("/tmp/e000134D.tmp", O_RDWR|O_CREAT|O_APPEND, 0666) = 8
  12:09:53.292779 open("/tmp/e000134D.tmp", O_RDWR|O_CREAT|O_TRUNC, 0666) = 8
  12:09:53.452220 open("/tmp/e000134E.tmp", O_RDWR|O_CREAT|O_APPEND, 0666) = 8
  12:09:53.452310 open("/tmp/e000134E.tmp", O_RDWR|O_CREAT|O_TRUNC, 0666) = 8
  12:09:53.630176 open("/tmp/e000134F.tmp", O_RDWR|O_CREAT|O_APPEND, 0666) = 8
  12:09:53.630275 open("/tmp/e000134F.tmp", O_RDWR|O_CREAT|O_TRUNC, 0666) = 8
  12:09:53.797461 open("/tmp/e0001350.tmp", O_RDWR|O_CREAT|O_APPEND, 0666) = 8
  12:09:53.797566 open("/tmp/e0001350.tmp", O_RDWR|O_CREAT|O_TRUNC, 0666) = 8
  12:09:53.968627 open("/tmp/e0001351.tmp", O_RDWR|O_CREAT|O_APPEND, 0666) = 8
  12:09:53.968727 open("/tmp/e0001351.tmp", O_RDWR|O_CREAT|O_TRUNC, 0666) = 8
  12:09:54.145310 open("/tmp/e0001352.tmp", O_RDWR|O_CREAT|O_APPEND, 0666) = 8
  12:09:54.145417 open("/tmp/e0001352.tmp", O_RDWR|O_CREAT|O_TRUNC, 0666) = 8

e as chamadas continuam se repetindo até que eu pare a operação.

Eu notei na minha pasta temporária que um arquivo criado tinha cerca de 90K, mas na maioria das vezes eles são 0 bytes.

Eu tentei aumentar a velocidade das operações das seguintes formas:

Eu ajustei meus parâmetros de inicialização para permitir ram drives maiores, então adicionei essas duas linhas antes de iniciar qualquer coisa que precise da pasta / tmp: 

mke2fs -cc -b 1024 -m 0 /dev/ram0 50000
mount -o async,nodev,noatime,nodiratime,noiversion,nomand,norelatime /dev/ram0 /tmp

Eu também executei o aplicativo com ionice: 

ionice -c1 -n0 appname

Aqui está mais a minha ideia em relação à forma como lida com um arquivo temporário da criação à exclusão: 

  12:17:02.905620 stat64("/tmp/", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=1024, ...}) = 0
  12:17:02.905719 access("/tmp/", R_OK|W_OK|X_OK) = 0
  12:17:02.905782 open("/tmp/e0001C6D.tmp", O_RDWR|O_CREAT|O_APPEND, 0666) = 8
  12:17:02.905851 close(8)                = 0
  12:17:02.905886 open("/tmp/e0001C6D.tmp", O_RDWR|O_CREAT|O_TRUNC, 0666) = 8
  12:17:02.905933 fstat64(8, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
  12:17:02.905994 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7c5c000
  12:17:02.906044 write(8, "
  12:09:52.677525 open("/tmp/e000134A.tmp", O_RDWR|O_CREAT|O_APPEND, 0666) = 8
  12:09:52.677777 open("/tmp/e000134A.tmp", O_RDWR|O_CREAT|O_TRUNC, 0666) = 8
  12:09:52.871198 open("/tmp/e000134B.tmp", O_RDWR|O_CREAT|O_APPEND, 0666) = 8
  12:09:52.871342 open("/tmp/e000134B.tmp", O_RDWR|O_CREAT|O_TRUNC, 0666) = 8
  12:09:53.092441 open("/tmp/e000134C.tmp", O_RDWR|O_CREAT|O_APPEND, 0666) = 8
  12:09:53.092563 open("/tmp/e000134C.tmp", O_RDWR|O_CREAT|O_TRUNC, 0666) = 8
  12:09:53.292663 open("/tmp/e000134D.tmp", O_RDWR|O_CREAT|O_APPEND, 0666) = 8
  12:09:53.292779 open("/tmp/e000134D.tmp", O_RDWR|O_CREAT|O_TRUNC, 0666) = 8
  12:09:53.452220 open("/tmp/e000134E.tmp", O_RDWR|O_CREAT|O_APPEND, 0666) = 8
  12:09:53.452310 open("/tmp/e000134E.tmp", O_RDWR|O_CREAT|O_TRUNC, 0666) = 8
  12:09:53.630176 open("/tmp/e000134F.tmp", O_RDWR|O_CREAT|O_APPEND, 0666) = 8
  12:09:53.630275 open("/tmp/e000134F.tmp", O_RDWR|O_CREAT|O_TRUNC, 0666) = 8
  12:09:53.797461 open("/tmp/e0001350.tmp", O_RDWR|O_CREAT|O_APPEND, 0666) = 8
  12:09:53.797566 open("/tmp/e0001350.tmp", O_RDWR|O_CREAT|O_TRUNC, 0666) = 8
  12:09:53.968627 open("/tmp/e0001351.tmp", O_RDWR|O_CREAT|O_APPEND, 0666) = 8
  12:09:53.968727 open("/tmp/e0001351.tmp", O_RDWR|O_CREAT|O_TRUNC, 0666) = 8
  12:09:54.145310 open("/tmp/e0001352.tmp", O_RDWR|O_CREAT|O_APPEND, 0666) = 8
  12:09:54.145417 open("/tmp/e0001352.tmp", O_RDWR|O_CREAT|O_TRUNC, 0666) = 8

46
mke2fs -cc -b 1024 -m 0 /dev/ram0 50000
mount -o async,nodev,noatime,nodiratime,noiversion,nomand,norelatime /dev/ram0 /tmp

ionice -c1 -n0 appname

  12:17:02.905620 stat64("/tmp/", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=1024, ...}) = 0
  12:17:02.905719 access("/tmp/", R_OK|W_OK|X_OK) = 0
  12:17:02.905782 open("/tmp/e0001C6D.tmp", O_RDWR|O_CREAT|O_APPEND, 0666) = 8
  12:17:02.905851 close(8)                = 0
  12:17:02.905886 open("/tmp/e0001C6D.tmp", O_RDWR|O_CREAT|O_TRUNC, 0666) = 8
  12:17:02.905933 fstat64(8, {st_mode=S_IFREG|0644, st_size=0, ...}) = 0
  12:17:02.905994 mmap2(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0xb7c5c000
  12:17:02.906044 write(8, "%pre%46%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%"..., 1024) = 1024
  12:17:02.906117 write(8, "%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%"..., 63488) = 63488
  12:17:02.906305 write(8, "@%pre%1%pre%1%pre%1%pre%1%pre%1%pre%3%pre%3%pre%3%pre%2%pre%3%pre%3%pre%3%pre%Q%pre%Q%pre%b%pre%Q"..., 1024) = 1024
  12:17:02.906361 write(8, "Q%pre%Q%pre%2%pre%b%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%"..., 33792) = 33792
  12:17:02.924577 gettimeofday({1519233422, 924592}, NULL) = 0
  12:17:02.935856 gettimeofday({1519233422, 935870}, NULL) = 0
  12:17:02.947417 gettimeofday({1519233422, 947430}, NULL) = 0
  12:17:02.947453 write(8, "%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%"..., 544) = 544
  12:17:02.947523 _llseek(8, 0, [0], SEEK_SET) = 0
  12:17:02.947558 read(8, "%pre%46%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%"..., 1024) = 1024
  12:17:02.947616 read(8, "%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%"..., 63488) = 63488
  12:17:02.947727 read(8, "@%pre%1%pre%1%pre%1%pre%1%pre%1%pre%3%pre%3%pre%3%pre%2%pre%3%pre%3%pre%3%pre%Q%pre%Q%pre%b%pre%Q"..., 1024) = 1024
  12:17:02.947781 read(8, "Q%pre%Q%pre%2%pre%b%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%"..., 33792) = 33792
  12:17:02.947864 read(8, "%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%"..., 1024) = 544
  12:17:02.948680 close(8)                = 0
  12:17:02.948713 munmap(0xb7c5c000, 4096) = 0
  12:17:02.948751 unlink("/tmp/e0001C6D.tmp") = 0

%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%"..., 1024) = 1024
  12:17:02.906117 write(8, "%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%"..., 63488) = 63488
  12:17:02.906305 write(8, "@%pre%1%pre%1%pre%1%pre%1%pre%1%pre%3%pre%3%pre%3%pre%2%pre%3%pre%3%pre%3%pre%Q%pre%Q%pre%b%pre%Q"..., 1024) = 1024
  12:17:02.906361 write(8, "Q%pre%Q%pre%2%pre%b%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%"..., 33792) = 33792
  12:17:02.924577 gettimeofday({1519233422, 924592}, NULL) = 0
  12:17:02.935856 gettimeofday({1519233422, 935870}, NULL) = 0
  12:17:02.947417 gettimeofday({1519233422, 947430}, NULL) = 0
  12:17:02.947453 write(8, "%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%"..., 544) = 544
  12:17:02.947523 _llseek(8, 0, [0], SEEK_SET) = 0
  12:17:02.947558 read(8, "%pre%46%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%"..., 1024) = 1024
  12:17:02.947616 read(8, "%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%"..., 63488) = 63488
  12:17:02.947727 read(8, "@%pre%1%pre%1%pre%1%pre%1%pre%1%pre%3%pre%3%pre%3%pre%2%pre%3%pre%3%pre%3%pre%Q%pre%Q%pre%b%pre%Q"..., 1024) = 1024
  12:17:02.947781 read(8, "Q%pre%Q%pre%2%pre%b%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%"..., 33792) = 33792
  12:17:02.947864 read(8, "%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%%pre%"..., 1024) = 544
  12:17:02.948680 close(8)                = 0
  12:17:02.948713 munmap(0xb7c5c000, 4096) = 0
  12:17:02.948751 unlink("/tmp/e0001C6D.tmp") = 0

Existe alguma maneira de reduzir ou acelerar essas chamadas?

    
por Mike 21.02.2018 / 18:27

1 resposta

0

Um sistema de arquivos ext2 suportado pela RAM é muito menos eficiente que um ramfs ou tmpfs. Esta é uma das razões pelas quais o kernel mudou de initrd para initramfs. Tente substituir seu ponto de montagem / tmp por um tmpfs.

    
por 04.09.2018 / 01:19

Tags

Novo laptop Ryzen - suspend = panic, é uma correção no pipeline? SSH pede para adicionar o mesmo host como conhecido pela segunda vez