Crie o CSV e o Gráfico de estatísticas a partir do próprio arquivo de log

Question

eu não sou tão bom assim, para que funcione ...

Eu escrevi um pequeno script que executa um teste de velocidade a cada 15 minutos e registra a saída em arquivos de log diários:

#!/bin/bash

DATE=$(date +%d.%m.%Y_%H:%M)
DAY=$(date +%d.%m.%Y)
WANIP=$(dig +short myip.opendns.com @resolver1.opendns.com)
echo "Timestamp: "$DATE >> /home/rene/speedtests/$DAY.log
echo "WanIP: "$WANIP >> /home/rene/speedtests/$DAY.log

/home/rene/speedtest-cli --server 3199 --share --simple >> /home/rene/speedtests/$DAY.log
echo "----------" >> /home/rene/speedtests/$DAY.log

O log se parece com o seguinte:

[rene@manjaro-home ~]$ cat speedtests/11.10.2016.log 
Timestamp: 11.10.2016_00:00
WanIP: 77.119.x.x
Ping: 63.773 ms
Download: 32.90 Mbit/s
Upload: 29.05 Mbit/s
Share results: http://www.speedtest.net/result/5703959727.png
----------
Timestamp: 11.10.2016_00:15
WanIP: 77.119.x.x
Ping: 70.06 ms
Download: 30.85 Mbit/s
Upload: 26.31 Mbit/s
Share results: http://www.speedtest.net/result/5703983637.png
----------
Timestamp: 11.10.2016_00:30
WanIP: 77.119.x.x
Ping: 65.722 ms
Download: 31.16 Mbit/s
Upload: 26.80 Mbit/s
Share results: http://www.speedtest.net/result/5704006209.png
----------

Quero apresentar uma queixa contra o meu provedor, porque eu deveria ter 150/50, e desde um mês eu tenho essa largura de banda ruim = (

agora eu preciso de um script que converta este arquivo de log assim:

Time,Ping,Download,Upload,Link
11.10.2016_00:00,63.773,32.90,29.05,http://www.speedtest.net/result/5703959727.png
11.10.2016_00:15,70.06,30.85,26.31,http://www.speedtest.net/result/5703983637.png
11.10.2016_00:30,65.722,31.16,26.80,http://www.speedtest.net/result/5704006209.png

eu já tentei algumas coisas, mas eu sou apenas para despejar para fazer isso com o awk ;-D

[rene@manjaro-home ~]$ cat daily-csv.sh 
#!/bin/bash
DAY=$(date +%d.%m.%Y)

#echo '"Timestamp","Ping","Download","Upload","Link"' > /home/rene/speedtests/$DAY.csv
for line in 'cat /home/rene/speedtests/$DAY.log'; do
            timestamp=$(echo $line | awk -F"Timestamp: " '{$0=$1}2');
echo $timestamp
done





#for F in /home/rene/speedtests/$DAY.log
#do
#    {
#        read Timestamp
#        read WanIP
#        read Ping
#        read Download
#        read Upload
#   read Link
#   read Placeholder
#    } < $F
#    echo "$Timestamp,$Ping,$Download,$Upload,$Link" >> speedtests/$DAY.csv
#    
#TIME=${Timestamp#* }
#PING=${Ping#* }
#DOWN=${Download#* }
#UP=${Upload#* }
#LINK=${Link#*: }
#
#echo $TIME,$PING,$DOWN,$UP,$LINK
#
#
#done

A parte não comentada fornece apenas o primeiro log.

Então ... alguém poderia me ajudar? :)

    

Answer 1

bem, no primeiro, eu estava trabalhando com o script que você carregou, mas não funciona no meu ambiente (Solaris 10), então esta é a resposta para sua pergunta:

    #!/bin/bash

    #######################################################
    # Script to convert with SED and AWK a logfile to CSV #
    #######################################################


    ######################
    ##    Empty files   ##
    ######################

    cat /dev/null>a.txt
    cat /dev/null>b.txt
    cat /dev/null>c.txt
    cat /dev/null>d.txt
    cat /dev/null>e.txt

    cat /dev/null>one.txt
    cat /dev/null>two.txt
    cat /dev/null>three.txt
    cat /dev/null>four.txt
    cat /dev/null>five.csv


    #########################################
    ##    Separate all aoutputs in files   ##
    #########################################


    for line in 'cat speedtests/11.10.2016.log | grep "Timestamp: "';
    do
    echo $line |awk -F"Timestamp: " '{print $1}' | sed '/^$/d';
    done>a.txt
    cat a.txt;



    for line in 'cat speedtests/11.10.2016.log | grep "Ping"';
    do
    echo $line |awk -F"Ping:" '{print $1}' | sed '/ms/d' |sed '/^$/d';
    done>b.txt;
    cat b.txt;



    for line in 'cat speedtests/11.10.2016.log | grep "Download: "';
    do
    echo $line |awk -F"Download: " '{print $1}' | sed '/Mbit/d' | sed '/^$/d';
    done>c.txt
    cat c.txt


    for line in 'cat speedtests/11.10.2016.log | grep "Upload: "';
    do
    echo $line |awk -F"Upload: " '{print $1}' | sed '/Mbit/d' | sed '/^$/d';
    done>d.txt
    cat d.txt


    for line in 'cat speedtests/11.10.2016.log | grep "http"';
    do
    echo $line |awk -F":" '{print $2}'  |sed '/^$/d' | sed  '1s/./https:\//' | sed 's/\//\\\//g' 
    done>e.txt
    cat e.txt


    #################################################################
    ##    Create outputs from previous outputs and adding a coma   ##
    #################################################################

    while read var
    do
    sed "s/$/,$var/" a.txt > one.txt
    done<b.txt
    cat one.txt;

    while read var
    do
    sed "s/$/,$var/" one.txt > two.txt
    done<c.txt
    cat two.txt;

    while read var
    do
    sed "s/$/,$var/" two.txt > three.txt
    done<d.txt
    cat three.txt;


    while read var
    do
    sed "s/$/,$var/" three.txt > four.txt
    done<e.txt
    cat four.txt;

    ##########################################
    ##    Add header only if you requided   ##
    ##########################################

    echo "------Time---------Ping--Down---Up--------Link-------------------------------------" >five.csv

    cat four.txt >>five.csv

    ###############################################
    ##    This is the final output as you whish  ##
    ###############################################

    cat five.csv;

Por favor, avise

Atenciosamente Luis