Como grep endereço IP e número de acesso

Question

Estou usando o seguinte comando para obter o número de ocorrências de um endereço IP:

grep "21/Jul/2013:22" /var/log/ispconfig/httpd/*/access.log | cut -d[ -f2 | cut -d] -f1 | awk -F: '{print $2":"$3}' | sort -nk1 -nk2 | uniq -c | awk '{ if ($1 > 10) print $0}'

Isso me dará o número de acessos por minuto. Um resultado de exemplo é:

181 22:00
330 22:01
253 22:02
240 22:03
202 22:04
232 22:05
195 22:06
174 22:07
251 22:08
287 22:09
281 22:10

Mas como obtenho o endereço IP e especifique o número de ocorrências maior que 100 (100 é apenas um exemplo. Posso especificar 200 ou 300)?

Exemplo eu quero os seguintes resultados:

101 22:00   192.168.1.1
80  22:00   192.168.1.2
105 22:01   192.168.1.1
115 22:01   192.168.1.2
110 22:01   192.168.1.3

atualização:

aqui está o log de amostra:

112.204.155.217 - - [21/Jul/2013:22:51:43 +0800] "GET /wp-content/uploads/2011/01/sinulog-150x150.png HTTP/1.1" 200 48759 "http://mysite.com/2013/05/14/may-2013-local-election-results-cebu-province-local/" "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1500.72 Safari/537.36"
2.90.192.222 - - [21/Jul/2013:22:51:45 +0800] "GET /wp-content/plugins/wp-minify/min/?f=wp-includes/js/jquery/jquery.js,wp-content/plugins/anti-spam/js/anti-spam.js,wp-includes/js/comment-reply.min.js&m=1370437401 HTTP/1.1" 304 0 "http://mysite.com/2013/05/10/it-takes-a-man-and-a-woman-top-20-highest-grossing-pinoy-film/" "Mozilla/5.0 (Linux; U; Android 2.3.6; en-us; GT-S5830i Build/GINGERBREAD) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1"
2.90.192.222 - - [21/Jul/2013:22:51:45 +0800] "GET /wp-content/plugins/wp-minify/min/?f=wp-content/themes/thesis_182/style.css,wp-content/themes/thesis_182/custom/layout.css,wp-content/themes/thesis_182/custom/custom.css,wp-content/plugins/async-social-sharing/assets/css/async-share.css,wp-content/plugins/contact-form-7/includes/css/styles.css&m=1373070592 HTTP/1.1" 304 0 "http://mysite.com/2013/05/10/it-takes-a-man-and-a-woman-top-20-highest-grossing-pinoy-film/" "Mozilla/5.0 (Linux; U; Android 2.3.6; en-us; GT-S5830i Build/GINGERBREAD) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1"
112.211.182.145 - - [21/Jul/2013:22:51:49 +0800] "GET /2013/07/21/aia-de-leon-imago-failed-pass-blind-audition-the-voice-ph-video/ HTTP/1.1" 200 9960 "http://www.google.com.ph/url?sa=t&rct=j&q=&esrc=s&source=web&cd=2&cad=rja&ved=0CDYQFjAB&url=http%3A%2F%2Fmysite.com%2F2013%2F07%2F21%2Faia-de-leon-imago-failed-pass-blind-audition-the-voice-ph-video%2F&ei=mvXrUb2PFeekigfmo4HABw&usg=AFQjCNGR2knrNsMpeOn6vw1a7yyt57-IZg&sig2=sk6-hLkuluCxjNx2S-X0eg&bvm=bv.49478099,d.aGc" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0"
112.211.182.145 - - [21/Jul/2013:22:51:50 +0800] "GET /wp-content/plugins/wp-minify/min/?f=wp-content/themes/thesis_182/style.css,wp-content/themes/thesis_182/custom/layout.css,wp-content/themes/thesis_182/custom/custom.css,wp-content/plugins/async-social-sharing/assets/css/async-share.css,wp-content/plugins/contact-form-7/includes/css/styles.css&m=1373070592 HTTP/1.1" 200 7171 "http://mysite.com/2013/07/21/aia-de-leon-imago-failed-pass-blind-audition-the-voice-ph-video/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0"
112.211.182.145 - - [21/Jul/2013:22:51:51 +0800] "GET /wp-content/plugins/wp-minify/min/?f=wp-includes/js/jquery/jquery.js,wp-content/plugins/anti-spam/js/anti-spam.js,wp-includes/js/comment-reply.min.js&m=1370437401 HTTP/1.1" 200 33872 "http://mysite.com/2013/07/21/aia-de-leon-imago-failed-pass-blind-audition-the-voice-ph-video/" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:22.0) Gecko/20100101 Firefox/22.0"
222.127.42.226 - - [21/Jul/2013:22:51:59 +0800] "GET /wp-content/uploads/2011/03/Marlon-Stockinger-Filipino-Swiss-Formula-Racer-270x180.jpg HTTP/1.1" 200 20345 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_3 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10B329 Safari/8536.25"
182.250.152.124 - - [21/Jul/2013:22:51:59 +0800] "GET /wp-content/uploads/2013/01/602826_10151372744414415_1480456855_n.jpg HTTP/1.1" 200 99263 "-" "rarely used"
112.205.18.203 - - [21/Jul/2013:22:52:01 +0800] "GET /2013/07/18/sona-2013-traffic-advisory-re-routing-monday-july-22/ HTTP/1.1" 200 10649 "http://www.google.com.ph/search?ei=g_XrUYTyJPCZiAfB4oDQBA&q=traffic+advisory+sona+2013&oq=traffic+advisory+sona+2013&gs_l=mobile-gws-serp.12...7703.33377.0.44233.61.38.12.11.13.3.270.5770.13j7j18.38.0....0...1c.1.21.mobile-gws-serp.Ekl3e2HRi4I" "Mozilla/5.0 (iPhone; CPU iPhone OS 6_1_4 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko) Version/6.0 Mobile/10B350 Safari/8536.25"

    

Answer 1

Minha solução para o seu caso:

awk '$4 ~ /21\/Jul\/2013:22/ {print $4, $1}' test.txt | \
awk -F: '{print $2":"$3, $4}' | sed -e 's/ [0-9]* / /g' | \
sort -nk2 | sort -t":" -nk1 -nk2 | uniq -c | awk '$1 > 0'

A saída processada pelo seu exemplo de log:

  1 22:51 112.204.155.217
  3 22:51 112.211.182.145
  1 22:51 182.250.152.124
  1 22:51 222.127.42.226
  2 22:51 2.90.192.222
  1 22:52 112.205.18.203

Substitua awk '$1 > 0' no final do comando pelo número desejado, ou seja, imprima com awk '$1 > 1'

  3 22:51 112.211.182.145
  2 22:51 2.90.192.222

    

Answer 2

Tente usar awk para cortar os dados. nawk ou gawk são mais poderosos e tornam isso mais fácil.

Algo como awk '{print $4, $1}' deve desviar os dados que o corte pode manipular facilmente.]]

EDIT: Conheça o comando man. Ele fornecerá documentação sob demanda, mais rápido que o Google ou o StackExchange. Você deve poder recortar e colar os seguintes exemplos.

Os acessos por minuto por IP estão disponíveis com:

awk '$4 ~ /21\/Jul\/2013:22/ {print $4, $1}' /var/log/ispconfig/httpd/*/access.log  \
| cut -c 11-15,20- | sort | uniq -c | awk '$1 > 10 {print}'

Se você quer hit por IP, tente isto:

awk '$4 ~ /21\/Jul\/2013:22/ {print $1}' /var/log/ispconfig/httpd/*/access.log  \
| sort | uniq -c | awk '$1 > 10 {print}'

Seus resultados originais podem ser feitos usando:

awk '$4 ~ /21\/Jul\/2013:22/ {print $4}' /var/log/ispconfig/httpd/*/access.log \
| cut -c 11-15 | sort | uniq -c | awk '$1 > 10 {print}'