Paralelizar loop para analisar arquivos

Question

Eu tenho o seguinte script para analisar alguns pcap arquivos de rastreamento. Existem várias pastas e cada pasta contém 4% de arquivospcap que precisam ser analisados:

for d in * ; 
do cd ${d}; 
for file in firewall router2 vclient vserver ; 
do tshark -r ${file}_log.pcap -T fields -e ip.src -e ip.dst -e tcp.srcport -e tcp.dstport -e tcp.len -e ip.len -e ip.hdr_len -e tcp.hdr_len -e _ws.col.Time -t e | awk '{print $1,$2,$3,$4,$6-$7-$8,$9}' > ${file}_conv_t.txt; 
cat ${file}_conv_t.txt | awk '$4==5001{print $6,$5}' | sed 's/\([0-9]*\)\.[0-9]*//' | awk '{sum[$1]+=$2} END {for (i in sum) {print i,sum[i]} }' | sort | cut -d ' ' -f 2 > ${file}_in.csv; 
cat ${file}_conv_t.txt | awk '$3==5001{print $6,$5}' | sed 's/\([0-9]*\)\.[0-9]*//' | awk '{sum[$1]+=$2} END {for (i in sum) {print i,sum[i]} }' | sort | cut -d ' ' -f 2 > ${file}_out.csv ; 
done ; 
cd .. ; 
done

Como todos os arquivos podem ser analisados independentemente, quero paralelizar isso o máximo possível. Eu tentei colocar um & em vez de ; após o primeiro done , para que cada arquivo seja processado em paralelo, assim:

for d in * ; 
do cd ${d}; 
for file in firewall router2 vclient vserver ; 
do tshark -r ${file}_log.pcap -T fields -e ip.src -e ip.dst -e tcp.srcport -e tcp.dstport -e tcp.len -e ip.len -e ip.hdr_len -e tcp.hdr_len -e _ws.col.Time -t e | awk '{print $1,$2,$3,$4,$6-$7-$8,$9}' > ${file}_conv_t.txt; 
cat ${file}_conv_t.txt | awk '$4==5001{print $6,$5}' | sed 's/\([0-9]*\)\.[0-9]*//' | awk '{sum[$1]+=$2} END {for (i in sum) {print i,sum[i]} }' | sort | cut -d ' ' -f 2 > ${file}_in.csv; 
cat ${file}_conv_t.txt | awk '$3==5001{print $6,$5}' | sed 's/\([0-9]*\)\.[0-9]*//' | awk '{sum[$1]+=$2} END {for (i in sum) {print i,sum[i]} }' | sort | cut -d ' ' -f 2 > ${file}_out.csv ; 
done &
cd .. ; 
done

mas isso não funcionou, ainda era sequencial.

O que estou perdendo?

    

Answer 1

Use {} chaves:

for d in * ; 
do cd ${d}; 
for file in firewall router2 vclient vserver ; 
do { tshark -r ${file}_log.pcap -T fields -e ip.src -e ip.dst -e tcp.srcport -e tcp.dstport -e tcp.len -e ip.len -e ip.hdr_len -e tcp.hdr_len -e _ws.col.Time -t e | awk '{print $1,$2,$3,$4,$6-$7-$8,$9}' > ${file}_conv_t.txt; 
cat ${file}_conv_t.txt | awk '$4==5001{print $6,$5}' | sed 's/\([0-9]*\)\.[0-9]*//' | awk '{sum[$1]+=$2} END {for (i in sum) {print i,sum[i]} }' | sort | cut -d ' ' -f 2 > ${file}_in.csv; 
cat ${file}_conv_t.txt | awk '$3==5001{print $6,$5}' | sed 's/\([0-9]*\)\.[0-9]*//' | awk '{sum[$1]+=$2} END {for (i in sum) {print i,sum[i]} }' | sort | cut -d ' ' -f 2 > ${file}_out.csv ; } &
done
cd .. ; 
done