Eu tenho executado simulações em supercomputadores, muitas vezes escrevendo grandes arquivos de saída no formato h5 (em torno de 750MB). Para fazer backup dos dados após as simulações no meu computador local, usei o comando rsync .
No início, o comando analisa a lista de arquivos e inicia o processo de cópia. No caminho, os arquivos grandes (no formato h5) desaparecem (eu acho que eles são excluídos no próprio computador) por razões desconhecidas. O log de cópia está abaixo;
sathish@HP-EliteBook:~/Documents/Research/journal/results/oblate2.5$ rsync -av --exclude '*.xdr' [email protected]:/home/ncteh317/Desktop/lb3d/drag_test/test/bouzidi/Oblate_2.5/Re0.1ang* .
receiving incremental file list
Re0.1ang0/
Re0.1ang0/angle.sh
Re0.1ang0/chk_uid-info
Re0.1ang0/init.cfg
Re0.1ang0/input-default
Re0.1ang0/input-default.md
Re0.1ang0/myjob.cmd
Re0.1ang0/slurm-2430487.out
Re0.1ang0/Production/
Re0.1ang0/Production/init.cfg
Re0.1ang0/Production/input-default
Re0.1ang0/Production/input-default.md
Re0.1ang0/Production/md-cfg-desc_out_t00000000-0882612060.txt
Re0.1ang0/Production/md-cfg_out_p********-0882612060.asc
Re0.1ang0/Production/md-summary_out_t00030000-0882612060.txt
Re0.1ang0/Production/od_out_t00030000-0882612060.h5
Re0.1ang0/Production/velx_out_t00030000-0882612060.h5
Re0.1ang0/Production/vely_out_t00030000-0882612060.h5
WARNING: Re0.1ang0/Production/vely_out_t00030000-0882612060.h5 failed verification -- update discarded (will try again).
rsync: read errors mapping "/home/ncteh317/Desktop/lb3d/drag_test/test/bouzidi/Oblate_2.5/Re0.1ang0/Production/vely_out_t00030000-0882612060.h5": Stale file handle (116)
file has vanished: "/home/ncteh317/Desktop/lb3d/drag_test/test/bouzidi/Oblate_2.5/Re0.1ang0/Production/velz_out_t00030000-0882612060.h5"
Re0.1ang10/
Re0.1ang10/angle.sh
Re0.1ang10/chk_uid-info
Re0.1ang10/init.cfg
Re0.1ang10/input-default
Re0.1ang10/input-default.md
Re0.1ang10/myjob.cmd
Re0.1ang10/slurm-2430488.out
Re0.1ang10/Production/
Re0.1ang10/Production/init.cfg
Re0.1ang10/Production/input-default
Re0.1ang10/Production/input-default.md
Re0.1ang10/Production/md-cfg-desc_out_t00000000-0882624620.txt
Re0.1ang10/Production/md-cfg_out_p00000000-0882624620.asc
Re0.1ang10/Production/md-summary_out_t00030000-0882624620.txt
file has vanished: "/home/ncteh317/Desktop/lb3d/drag_test/test/bouzidi/Oblate_2.5/Re0.1ang10/Production/od_out_t00030000-0882624620.h5"
file has vanished: "/home/ncteh317/Desktop/lb3d/drag_test/test/bouzidi/Oblate_2.5/Re0.1ang10/Production/velx_out_t00030000-0882624620.h5"
file has vanished: "/home/ncteh317/Desktop/lb3d/drag_test/test/bouzidi/Oblate_2.5/Re0.1ang10/Production/vely_out_t00030000-0882624620.h5"
file has vanished: "/home/ncteh317/Desktop/lb3d/drag_test/test/bouzidi/Oblate_2.5/Re0.1ang10/Production/velz_out_t00030000-0882624620.h5"
Re0.1ang30/
Re0.1ang30/angle.sh
Re0.1ang30/chk_uid-info
Re0.1ang30/init.cfg
Re0.1ang30/input-default
Re0.1ang30/input-default.md
Re0.1ang30/myjob.cmd
Re0.1ang30/slurm-2430489.out
Re0.1ang30/Production/
Re0.1ang30/Production/init.cfg
Re0.1ang30/Production/input-default
Re0.1ang30/Production/input-default.md
Re0.1ang30/Production/md-cfg-desc_out_t00000000-0882622039.txt
Re0.1ang30/Production/md-cfg_out_p********-0882622039.asc
Re0.1ang30/Production/md-summary_out_t00030000-0882622039.txt
file has vanished: "/home/ncteh317/Desktop/lb3d/drag_test/test/bouzidi/Oblate_2.5/Re0.1ang30/Production/od_out_t00030000-0882622039.h5"
file has vanished: "/home/ncteh317/Desktop/lb3d/drag_test/test/bouzidi/Oblate_2.5/Re0.1ang30/Production/velx_out_t00030000-0882622039.h5"
file has vanished: "/home/ncteh317/Desktop/lb3d/drag_test/test/bouzidi/Oblate_2.5/Re0.1ang30/Production/vely_out_t00030000-0882622039.h5"
file has vanished: "/home/ncteh317/Desktop/lb3d/drag_test/test/bouzidi/Oblate_2.5/Re0.1ang30/Production/velz_out_t00030000-0882622039.h5"
Re0.1ang45/
Re0.1ang45/angle.sh
Re0.1ang45/chk_uid-info
Re0.1ang45/init.cfg
Re0.1ang45/input-default
Re0.1ang45/input-default.md
Re0.1ang45/myjob.cmd
Re0.1ang45/slurm-2430490.out
Re0.1ang45/Production/
Re0.1ang45/Production/init.cfg
Re0.1ang45/Production/input-default
Re0.1ang45/Production/input-default.md
Re0.1ang45/Production/md-cfg-desc_out_t00000000-0882629804.txt
Re0.1ang45/Production/md-cfg_out_p00000000-0882629804.asc
Re0.1ang45/Production/md-summary_out_t00030000-0882629804.txt
file has vanished: "/home/ncteh317/Desktop/lb3d/drag_test/test/bouzidi/Oblate_2.5/Re0.1ang45/Production/od_out_t00030000-0882629804.h5"
file has vanished: "/home/ncteh317/Desktop/lb3d/drag_test/test/bouzidi/Oblate_2.5/Re0.1ang45/Production/velx_out_t00030000-0882629804.h5"
file has vanished: "/home/ncteh317/Desktop/lb3d/drag_test/test/bouzidi/Oblate_2.5/Re0.1ang45/Production/vely_out_t00030000-0882629804.h5"
file has vanished: "/home/ncteh317/Desktop/lb3d/drag_test/test/bouzidi/Oblate_2.5/Re0.1ang45/Production/velz_out_t00030000-0882629804.h5"
Re0.1ang60/
Re0.1ang60/angle.sh
Re0.1ang60/chk_uid-info
Re0.1ang60/init.cfg
Re0.1ang60/input-default
Re0.1ang60/input-default.md
Re0.1ang60/myjob.cmd
Re0.1ang60/slurm-2430491.out
Re0.1ang60/Production/
Re0.1ang60/Production/init.cfg
Re0.1ang60/Production/input-default
Re0.1ang60/Production/input-default.md
Re0.1ang60/Production/md-cfg-desc_out_t00000000-0882627357.txt
Re0.1ang60/Production/md-cfg_out_p********-0882627357.asc
Re0.1ang60/Production/md-summary_out_t00030000-0882627357.txt
file has vanished: "/home/ncteh317/Desktop/lb3d/drag_test/test/bouzidi/Oblate_2.5/Re0.1ang60/Production/od_out_t00030000-0882627357.h5"
file has vanished: "/home/ncteh317/Desktop/lb3d/drag_test/test/bouzidi/Oblate_2.5/Re0.1ang60/Production/velx_out_t00030000-0882627357.h5"
file has vanished: "/home/ncteh317/Desktop/lb3d/drag_test/test/bouzidi/Oblate_2.5/Re0.1ang60/Production/vely_out_t00030000-0882627357.h5"
file has vanished: "/home/ncteh317/Desktop/lb3d/drag_test/test/bouzidi/Oblate_2.5/Re0.1ang60/Production/velz_out_t00030000-0882627357.h5"
Re0.1ang80/
Re0.1ang80/angle.sh
Re0.1ang80/chk_uid-info
Re0.1ang80/init.cfg
Re0.1ang80/input-default
Re0.1ang80/input-default.md
Re0.1ang80/myjob.cmd
Re0.1ang80/slurm-2430492.out
Re0.1ang80/Production/
Re0.1ang80/Production/md-cfg-desc_out_t00000000-0882622795.txt
Re0.1ang80/Production/md-cfg_out_p00000000-0882622795.asc
Re0.1ang90/
Re0.1ang90/angle.sh
Re0.1ang90/chk_uid-info
Re0.1ang90/init.cfg
Re0.1ang90/input-default
Re0.1ang90/input-default.md
Re0.1ang90/myjob.cmd
Re0.1ang90/slurm-2430493.out
Re0.1ang90/Production/
Re0.1ang90/Production/init.cfg
Re0.1ang90/Production/input-default
Re0.1ang90/Production/input-default.md
Re0.1ang90/Production/md-cfg-desc_out_t00000000-0882623909.txt
Re0.1ang90/Production/md-cfg_out_p********-0882623909.asc
Re0.1ang90/Production/md-summary_out_t00030000-0882623909.txt
file has vanished: "/home/ncteh317/Desktop/lb3d/drag_test/test/bouzidi/Oblate_2.5/Re0.1ang90/Production/od_out_t00030000-0882623909.h5"
file has vanished: "/home/ncteh317/Desktop/lb3d/drag_test/test/bouzidi/Oblate_2.5/Re0.1ang90/Production/velx_out_t00030000-0882623909.h5"
file has vanished: "/home/ncteh317/Desktop/lb3d/drag_test/test/bouzidi/Oblate_2.5/Re0.1ang90/Production/vely_out_t00030000-0882623909.h5"
file has vanished: "/home/ncteh317/Desktop/lb3d/drag_test/test/bouzidi/Oblate_2.5/Re0.1ang90/Production/velz_out_t00030000-0882623909.h5"
file has vanished: "/home/ncteh317/Desktop/lb3d/drag_test/test/bouzidi/Oblate_2.5/Re0.1ang0/Production/vely_out_t00030000-0882612060.h5"
Basicamente, as linhas com file has vanished: são os arquivos grandes que eu queria fazer backup. Eu acho que eles estão indexados no início do comando rsync . Mas durante o processo de cópia, de alguma forma, os arquivos são excluídos. Eu procurei tópicos semelhantes nos fóruns. Mas, os problemas mencionados são arquivos sendo excluídos durante o processo de cópia.
O mesmo processo de cópia que experimentei com FileZilla , software de transferência de arquivos baseado em GUI, após novas simulações e o problema ainda permanece - os arquivos grandes desaparecem durante o processo de cópia no próprio supercomputador.
No meu caso, as simulações são concluídas e não há operação / modificação feita nos arquivos durante o processo de cópia. Sendo este o caso, alguém pode lançar alguma luz sobre o que está errado?