Resuma certos campos de dados do arquivo de log do Apache CLF

1

Eu quero buscar algumas informações do formato de log combinado Registros apache :

51.254.56.62 - - [01/Jun/2016:20:49:28 +0500] "GET /vendors/jquery.slimscroll.min.js HTTP/1.1" 404 - "http://networkconfig.net/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:46.0) Gecko/20100101 Firefox/46.0"
51.254.56.62 - - [01/Jun/2016:20:49:28 +0500] "GET /jquery.fullPage.js HTTP/1.1" 304 - "http://networkconfig.net/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:46.0) Gecko/20100101 Firefox/46.0"
51.254.56.62 - - [01/Jun/2016:20:49:29 +0500] "GET /js/TweenLite.min.js HTTP/1.1" 304 - "http://networkconfig.net/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:46.0) Gecko/20100101 Firefox/46.0"
51.254.56.62 - - [01/Jun/2016:20:49:29 +0500] "GET /js/EasePack.min.js HTTP/1.1" 304 - "http://networkconfig.com/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:46.0) Gecko/20100101 Firefox/46.0"
51.254.56.62 - - [01/Jun/2016:20:49:29 +0500] "GET /js/rAF.js HTTP/1.1" 304 - "http://networkconfig.com/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:46.0) Gecko/20100101 Firefox/46.0"
51.254.56.62 - - [01/Jun/2016:20:49:29 +0500] "GET /js/networkconfig.js HTTP/1.1" 304 - "http://networkconfig.com/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:46.0) Gecko/20100101 Firefox/46.0"
182.180.10.40 - - [01/Jun/2016:20:49:29 +0500] "GET /js/rAF.js HTTP/1.1" 304 - "http://networkconfig.com/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:46.0) Gecko/20100101 Firefox/46.0"
182.180.10.40 - - [01/Jun/2016:20:49:29 +0500] "GET /js/networkconfig.js HTTP/1.1" 304 - "http://networkconfig.com/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:46.0) Gecko/20100101 Firefox/46.0"
182.180.10.40 - - [01/Jun/2016:20:49:28 +0500] "GET /vendors/jquery.slimscroll.min.js HTTP/1.1" 404 - "http://networkconfig.net/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:46.0) Gecko/20100101 Firefox/46.0"
182.180.10.40 - - [01/Jun/2016:20:49:28 +0500] "GET /jquery.fullPage.js HTTP/1.1" 304 - "http://networkconfig.net/" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:46.0) Gecko/20100101 Firefox/46.0"

Isso é o que eu fiz:

  awk '{ print $1,$11}' accesslog | sort | uniq -c | sort -nr | head -n 10

  3 51.254.56.62 "http://networkconfig.net/"
  3 51.254.56.62 "http://networkconfig.com/"
  2 182.180.10.40 "http://networkconfig.net/"
  2 182.180.10.40 "http://networkconfig.com/"

O que eu quero é:

Domains                     Hits By IP

networkconfig.net           3 hits 51.254.56.62  | 2 hits 182.180.10.40 and so on
networkconfig.com           3 hits 51.254.56.62 | 2 hits 182.180.10.40 and so on
    
por blaCkninJa 01.06.2016 / 18:36

1 resposta

0

Versão revisada (3ª) de ugly.sh :

#!/bin/bash
{ echo "Domains  Hits by IP" ; \
  awk '{ print $1 gsub(/^.*:\/\/|\"|\/.*$/,"",$11) "\t" $11 }' $1 | \
      sort | \
      uniq  -c | \
      sort -k3,3 -k1,1nr | \
      while n="" read a b c; do \
          [ $a = 1 ] && p='' || p=s ; \
          if [ "$n" = "$c" ] ; then \
               echo -n "  |  $a hit$p $b" ; \
          else echo ; \
               echo -n   "$c $a hit$p $b" ; \
          fi  ; n="$c" ; \
      done ; \
      echo ; \
} | \
while read a b ; do \
    printf "%-30s   %s\n" "$a" "$b" ; \
done

Saída de ./ugly.sh accesslog :

Domains                          Hits by IP

networkconfig.com                3 hits 51.254.56.62  |  2 hits 182.180.10.40
networkconfig.net                3 hits 51.254.56.62  |  2 hits 182.180.10.40

Saída de ./ugly.sh log.txt , (URL do OP para dados: log.txt ):

Domains                          Hits by IP

-                                1 hit 180.76.15.138  |  1 hit 192.243.55.136
www.google.com.pk                3 hits 122.129.73.92
www.networkconfigorchard.com     2 hits 39.46.59.57  |  8 hits 39.46.6.0
    
por 01.06.2016 / 21:04