Por que meu script FTP não obtém todos os arquivos usando o comando mget?

Question

Estou tendo um script que extrai dados de diferentes servidores cujos detalhes são lidos de um arquivo externo. Ele lê os arquivos e a saída detalhada mostra todos os arquivos correspondentes, mas busca apenas um arquivo do host remoto. A seguir, meu script:

while IFS=','; read region sdp ip1 ip2 ip3 user1 pass1 user2 pass2 user3 pass3
do

    in=/var/opt/fds/statistics/
    out=/pmautomation/PM/RawFiles/Data/BHCA/$date/$region/$sdp/
    file=\*"PSC-TrafficHandler_8.1_A_"\*"_System."$date\*".stat"

    mkdir -p /pmautomation/PM/RawFiles/Data/BHCA/$date/$region/$sdp/

    ftp -in $ip1<<END_SCRIPT
    quote USER $user1
    quote PASS $pass1
    bin
    prompt off
    lcd /pmautomation/PM/RawFiles/Data/BHCA/$date/$region/$sdp/
    cd /var/opt/fds/statistics/
    binary
    mget *PSC-TrafficHandler_8.1_A_*_System.$date*.stat
    bye
    END_SCRIPT

done < /root/SDP_BHC/bin/Credentials.csv

A seguir, a saída:

IP: 10.XXX.XX.XX

Interactive mode on.
Local directory now /pmautomation/PM/RawFiles/Data/BHCA/20150802/EAST/WB_SDP49
mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_0000.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_0100.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_0200.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_0300.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_0400.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_0500.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_0600.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_0700.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_0800.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_0900.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_1000.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_1100.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_1200.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_1300.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_1400.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_1500.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_1600.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_1700.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_1800.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_1900.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_2000.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_2100.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_2200.stat? mget WDSDP49B_PSC-TrafficHandler_8.1_A_2_System.20150802_2300.stat? 

Por que meu comando mget não é capaz de obter todos os arquivos e obter apenas um arquivo de todos os arquivos correspondentes?

    

Answer 1

Eu tenho algumas dicas de Jeff. De alguma forma, o prompt off não estava funcionando e eu estava recebendo prompt para obter os arquivos correspondentes. Eu tentei colocar o 'y' abaixo do comando mget e funcionou. A seguir, o código atualizado:

while IFS=','; read region sdp ip1 ip2 ip3 user1 pass1 user2 pass2 user3 pass3
do

    in=/var/opt/fds/statistics/
    out=/pmautomation/PM/RawFiles/Data/BHCA/$date/$region/$sdp/
    file=\*"PSC-TrafficHandler_8.1_A_"\*"_System."$date\*".stat"

    mkdir -p /pmautomation/PM/RawFiles/Data/BHCA/$date/$region/$sdp/

    ftp -in $ip1<<END_SCRIPT
    quote USER $user1
    quote PASS $pass1
    bin
    prompt off
    lcd /pmautomation/PM/RawFiles/Data/BHCA/$date/$region/$sdp/
    cd /var/opt/fds/statistics/
    binary
    mget *PSC-TrafficHandler_8.1_A_*_System.$date*.stat
    y
    y
    y
    y
    bye
    END_SCRIPT

done < /root/SDP_BHC/bin/Credentials.csv

Este é um comportamento muito estranho, mas funcionou para mim. Felizmente, eu sabia que o número de arquivos no host remoto e colocando o mesmo número de 'y' fazia com que funcionasse.

    

Answer 2

Já tentou remover o separador do final do documento "aqui"? Desde que o script não está detectando o fim que pode ser seu problema.

while IFS=','; read region sdp ip1 ip2 ip3 user1 pass1 user2 pass2 user3 pass3
do

    in=/var/opt/fds/statistics/
    out=/pmautomation/PM/RawFiles/Data/BHCA/$date/$region/$sdp/
    file=\*"PSC-TrafficHandler_8.1_A_"\*"_System."$date\*".stat"

    mkdir -p /pmautomation/PM/RawFiles/Data/BHCA/$date/$region/$sdp/

    ftp -in $ip1<<END_SCRIPT
    quote USER $user1
    quote PASS $pass1
    bin
    prompt off
    lcd /pmautomation/PM/RawFiles/Data/BHCA/$date/$region/$sdp/
    cd /var/opt/fds/statistics/
    binary
    mget *PSC-TrafficHandler_8.1_A_*_System.$date*.stat
    bye
END_SCRIPT    <----------

done < /root/SDP_BHC/bin/Credentials.csv

    

Answer 3

Use wget .

while IFS=','; read region sdp ip1 ip2 ip3 user1 pass1 user2 pass2 user3 pass3
do
    in=/var/opt/fds/statistics/
    out=/pmautomation/PM/RawFiles/Data/BHCA/$date/$region/$sdp/
    file=\*"PSC-TrafficHandler_8.1_A_"\*"_System."$date\*".stat"

    mkdir -p $out
    cd $out

    wget --ftp-user=$user1 --ftp-password=$pass1 ftp://$ip1/$in/$file

done < /root/SDP_BHC/bin/Credentials.csv