script bash, imprimindo várias linhas que correspondem a um padrão específico

0

Eu tenho um arquivo de log e estou fazendo um script para fazer algumas ações. Uma ação é imprimir uma área específica do log.

Cada bloco no registro começa com um registro de data e hora específico e dentro do bloco pode ter outras datas, etc.

Eu quero pegar o bloco que dentro dele existe a palavra exceção. Tentei com sed mas como sei processo linha por linha, também tentei com awk e FS \ n mas novamente nada ....

Uma parte do arquivo de log,

06:14:27.9 starting web server
06:14:33.3 Initializing Spring framework Logs
Oct 18, 2013 6:14:33 AM org.apache.catalina.startup.Embedded start
INFO: Starting tomcat server
Oct 18, 2013 6:14:34 AM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/6.0.32
Oct 18, 2013 6:14:35 AM org.apache.catalina.startup.ContextConfig DefaultWebConfig
INFO: No default web.xml
Oct 18, 2013 6:14:38 AM org.apache.catalina.session.StandardManager doLoad
SEVERE: IOException while loading persisted sessions: java.io.WriteAbortedException: writing aborted; java.io.NotSerializableException: ads.doc.backoffice.StoreInfos
java.io.WriteAbortedException: writing aborted; java.io.NotSerializableException: ads.doc.backoffice.StoreInfos
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1354)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
......
......
......
......
    at ads.tools.AppServerMain.main(AppServerMain.java:83)
Caused by: java.io.NotSerializableException: ads.doc.backoffice.StoreInfos
INFO: Jk running ID=0 time=0/105  config=null
06:14:48.6 Starting exporter server
06:14:48.6 starting cron server

outra parte de um log

03:19:13.4 Begin summary update for ads.doc.inventory.InventoryItemSummary
03:19:33.9 CronServer:: DailyJob ads.tools.UpdateSummaries@17c5d6cf failed with exception ads.util.AppError: Cannot create UnitName from keys: Each
ads.util.AppError: Cannot create UnitName from keys: Each
at ads.db.DBObjectDefault.createFromKeys(DBObjectDefault.java:42)
at ads.db.DBTable.createFromKeys(DBTable.java:227)
at ads.db.DBValue.getValue(DBValue.java:621)
at ads.dbmanager.DBObjectsManager.initObjects(DBObjectsManager.java:400)
at ads.dbmanager.DBObjectsManager.reload(DBObjectsManager.java:447)
at ads.dbmanager.DBObjectsManager.loadFromStore(DBObjectsManager.java:497)
at ads.doc.inventory.InventoryItemSummary.refreshSince(InventoryItemSummary.java:173)
at ads.db.DBSummaryTable.refreshAll(DBSummaryTable.java:67)
at ads.tools.CronServer$DailyThread.run(CronServer.java:271)
[SOAPException: faultCode=SOAP-ENV:Client; msg=Error opening socket: java.net.ConnectException: Connection refused; targetException=java.lang.IllegalArgumentException: Error opening socket: java.net.ConnectException: Connection refused]
at org.apache.soap.transport.http.SOAPHTTPConnection.send(SOAPHTTPConnection.java:354)
at org.apache.soap.rpc.Call.invoke(Call.java:248)
at ads.support.SupportCall.call(SupportCall.java:56)
at ads.tools.SupportThread.run(SupportThread.java:101)
03:46:42.5 Periodic support request failed: ads.support.SupportException: Error opening socket: java.net.ConnectException: Connection refused
06:31:36.1 Upload failed: java.io.FileNotFoundException: c:/tmp/cygwin1.dll (No such file or directory)
08:01:08.0 connect from /172.22.3.28

Eu quero imprimir do primeiro até o segundo último (06: 14: 33.3 até 06: 14: 48.6). E dentro do log existem vários blocos como este.

Tentei isto: awk '/^[0-9][0-9]\:[0-9][0-9]\:[0-9][0-9]\.[0-9].*[e|E]xception.*[0-9][0-9]\:[0-9][0-9]\:[0-9][0-9]\.[0-9]/ {print}' FS="\n" RS="" log.txt

e também isso:

sed '/^[0-9][0-9]\:[0-9][0-9]\:[0-9][0-9]\.[0-9].*[e|E]xception.*/,/^[0-9][0-9]\:[0-9][0-9]\:[0-9][0-9]\.[0-9]/!d' log.txt

mas não consigo obter o resultado que quero.

    
por C Ts 06.12.2013 / 15:50

2 respostas

1

Como nenhum assistente awk entrou na conversa ainda, demonstrarei o que eu quis dizer sobre usar mais de uma captura de busca em perl (já que não sou assistente de awk e prefiro perl a bash - mas você pode traduzir isso em bash). Sem comentários e dados de teste, são ~ 15 linhas:

#!/usr/bin/perl
use strict;
use warnings FATAL => qw(all);

my $timestamp = qr/^\d+:\d+:\d+/;
# Alternate version, see discussion below:
#my $timestamp = qr/^([a-zA-Z]+\s+\d+,\s+\d+\s+)?\d+:\d+:\d+/;

my $block = "";
my $exception = 0;
while (<DATA>) {
# Note that perl considers an empty string false.
    if ($block) {
    # $_ is the current input line in perl.
        if ($_ =~ $timestamp) {
            print "\n\n***EXCEPTION***\n$block" if $exception;
            $block = "";
            $exception = 0;
        }
    # This isn't actually a second regexp, it's a substring search.
    # But you could use a regexp.
        $exception = 1 if (index($_,"Exception") >= 0);
        $block .= $_;
        next;
    }
    $block = $_ if ($_ =~ $timestamp);
}

# Everything after this is read in via <DATA> for testing.
__DATA__
06:14:27.9 starting web server
06:14:33.3 Initializing Spring framework Logs
Oct 18, 2013 6:14:33 AM org.apache.catalina.startup.Embedded start
INFO: Starting tomcat server
Oct 18, 2013 6:14:34 AM org.apache.catalina.core.StandardEngine start
INFO: Starting Servlet Engine: Apache Tomcat/6.0.32
Oct 18, 2013 6:14:35 AM org.apache.catalina.startup.ContextConfig DefaultWebConfig
INFO: No default web.xml
Oct 18, 2013 6:14:38 AM org.apache.catalina.session.StandardManager doLoad
SEVERE: IOException while loading persisted sessions: java.io.WriteAbortedException: writing aborted; java.io.NotSerializableException: ads.doc.backoffice.StoreInfos
java.io.WriteAbortedException: writing aborted; java.io.NotSerializableException: ads.doc.backoffice.StoreInfos
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1354)
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
    at ads.tools.AppServerMain.main(AppServerMain.java:83)
Caused by: java.io.NotSerializableException: ads.doc.backoffice.StoreInfos
INFO: Jk running ID=0 time=0/105  config=null
06:14:48.6 Starting exporter server
06:14:48.6 starting cron server
03:19:13.4 Begin summary update for ads.doc.inventory.InventoryItemSummary
03:19:33.9 CronServer:: DailyJob ads.tools.UpdateSummaries@17c5d6cf failed with exception ads.util.AppError: Cannot create UnitName from keys: Each
ads.util.AppError: Cannot create UnitName from keys: Each
at ads.db.DBObjectDefault.createFromKeys(DBObjectDefault.java:42)
at ads.db.DBTable.createFromKeys(DBTable.java:227)
at ads.db.DBValue.getValue(DBValue.java:621)
at ads.dbmanager.DBObjectsManager.initObjects(DBObjectsManager.java:400)
at ads.dbmanager.DBObjectsManager.reload(DBObjectsManager.java:447)
at ads.dbmanager.DBObjectsManager.loadFromStore(DBObjectsManager.java:497)
at ads.doc.inventory.InventoryItemSummary.refreshSince(InventoryItemSummary.java:173)
at ads.db.DBSummaryTable.refreshAll(DBSummaryTable.java:67)
at ads.tools.CronServer$DailyThread.run(CronServer.java:271)
[SOAPException: faultCode=SOAP-ENV:Client; msg=Error opening socket: java.net.ConnectException: Connection refused; targetException=java.lang.IllegalArgumentException: Error opening socket: java.net.ConnectException: Connection refused]
at org.apache.soap.transport.http.SOAPHTTPConnection.send(SOAPHTTPConnection.java:354)
at org.apache.soap.rpc.Call.invoke(Call.java:248)
at ads.support.SupportCall.call(SupportCall.java:56)
at ads.tools.SupportThread.run(SupportThread.java:101)
03:46:42.5 Periodic support request failed: ads.support.SupportException: Error opening socket: java.net.ConnectException: Connection refused
06:31:36.1 Upload failed: java.io.FileNotFoundException: c:/tmp/cygwin1.dll (No such file or directory)
08:01:08.0 connect from /172.22.3.28        

O material DATA é todos os seus exemplos de log juntos. O ponto das duas pesquisas é primeiro identificar o início de um bloco e registrá-lo e, em seguida, determinar se o bloco contém um Exception .

Como está, o script encontra os seguintes quatro blocos:

***EXCEPTION***
06:14:33.3 Initializing Spring framework Logs
[...]
INFO: Jk running ID=0 time=0/105  config=null

***EXCEPTION***
03:19:33.9 CronServer:: DailyJob ads.tools.UpdateSummaries@17c5d6cf failed with exception ads.util.AppError: Cannot create UnitName from keys: Each
[...]
at ads.tools.SupportThread.run(SupportThread.java:101)

***EXCEPTION***
03:46:42.5 Periodic support request failed: ads.support.SupportException: Error opening socket: java.net.ConnectException: Connection refused

***EXCEPTION***
06:31:36.1 Upload failed: java.io.FileNotFoundException: c:/tmp/cygwin1.dll (No such file or directory)

Isso se baseia na interpretação de cada bloco como começando com um registro de data e hora do estilo 06:31:36 ; o material com Oct 18, 2013 6:14:33 é, obviamente, o material de log do JRE passado para o log do servidor e, portanto, parte de uma entrada. Se você quiser fazer isso de forma diferente, separando as exceções individuais do JRE, use a "Versão alternativa" de $timestamp , que considera os dois estilos como iniciando uma entrada.

    
por 06.12.2013 / 18:28
0

Tudo bem, este é um dos mais próximos:

awk -F "[0-9][0-9]:[0-9][0-9]:[0-9][0-9].[0-9]" '{for (i=1; i<= NF; i++) if ( $i ~ /[e|E]xception/ ) print $i} RS=""' t.txt

Eu uso o regex de registro de data e hora como FS e pesquise a exceção nos campos. Não é perfeito, mas perto.

Depois de enviar as duas partes dos logs OP fornecidos, a saída que obtive é a seguinte:

Initializing Spring framework Logs<br>
Oct 18, 2013 6:14:33 AM org.apache.catalina.startup.Embedded start<br>
INFO: Starting tomcat server<br>
Oct 18, 2013 6:14:34 AM org.apache.catalina.core.StandardEngine start<br>
INFO: Starting Servlet Engine: Apache Tomcat/6.0.32<br>
Oct 18, 2013 6:14:35 AM org.apache.catalina.startup.ContextConfig <br>efaultWebConfig
INFO: No default web.xml<br>
Oct 18, 2013 6:14:38 AM org.apache.catalina.session.StandardManager doLoad<br>
SEVERE: IOException while loading persisted sessions: <br>java.io.WriteAbortedException: writing aborted; <br>java.io.NotSerializableException: ads.doc.backoffice.StoreInfos<br>
java.io.WriteAbortedException: writing aborted; <br>java.io.NotSerializableException: ads.doc.backoffice.StoreInfos
    at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1354)<br>
    at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1990)
    at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1915)
...... <br>
...... <br>
...... <br>
...... <br>
    at ads.tools.AppServerMain.main(AppServerMain.java:83)<br>
Caused by: java.io.NotSerializableException: ads.doc.backoffice.StoreInfos<br>
INFO: Jk running ID=0 time=0/105  config=null<br>

 CronServer:: DailyJob ads.tools.UpdateSummaries@17c5d6cf failed with exception ads.util.AppError: Cannot create UnitName from keys: Each
ads.util.AppError: Cannot create UnitName from keys: Each
at ads.db.DBObjectDefault.createFromKeys(DBObjectDefault.java:42)
at ads.db.DBTable.createFromKeys(DBTable.java:227)
at ads.db.DBValue.getValue(DBValue.java:621)
at ads.dbmanager.DBObjectsManager.initObjects(DBObjectsManager.java:400)
at ads.dbmanager.DBObjectsManager.reload(DBObjectsManager.java:447)
at ads.dbmanager.DBObjectsManager.loadFromStore(DBObjectsManager.java:497)
at ads.doc.inventory.InventoryItemSummary.refreshSince(InventoryItemSummary.java:173)
at ads.db.DBSummaryTable.refreshAll(DBSummaryTable.java:67)
at ads.tools.CronServer$DailyThread.run(CronServer.java:271)
[SOAPException: faultCode=SOAP-ENV:Client; msg=Error opening socket: java.net.ConnectException: Connection refused; targetException=java.lang.IllegalArgumentException: Error opening socket: java.net.ConnectException: Connection refused]
at org.apache.soap.transport.http.SOAPHTTPConnection.send(SOAPHTTPConnection.java:354)
at org.apache.soap.rpc.Call.invoke(Call.java:248)
at ads.support.SupportCall.call(SupportCall.java:56)
at ads.tools.SupportThread.run(SupportThread.java:101)

 Periodic support request failed: ads.support.SupportException: Error opening socket: java.net.ConnectException: Connection refused

 Upload failed: java.io.FileNotFoundException: c:/tmp/cygwin1.dll (No such file or directory)
    
por 06.12.2013 / 22:08