Como fazer o Analog analisar apenas uma semana de logs?

1

No meu servidor windows, eu uso analógico para analisar meus arquivos de log e analisá-los. Os logs são diários e nomeados como exYYMMDD.log Como posso definir no arquivo cfg analógico para analisar apenas uma semana de registros?

    
por Magnetic_dud 15.06.2009 / 12:46

3 respostas

1

Eu comecei a usar analógicos e de reportagem ...

Eu tirei isso do howto nos documentos:

You can also include the date in the LOGFILE name, by using the following codes.

%D date of month
%m month name, in English
%M month number
%y two-digit year
%Y four-digit year
%H hour
%n minute
%w day of week, in English

So for example,
LOGFILE access_log%Y%M.log

will look for the logfile access_log200109.log, if it's September 2001. The date used is actually the TO date if one was specified, and otherwise the time of the start of the program. So for example, you can look at all of last month's logfiles with the commands

O que deve lhe dar uma pequena ajuda para configurar exatamente o que arquivos de log que você deseja analisar. Além disso, há essa entrada correspondente, que detalha as datas "PARA" e "DE":

There is also one other pair of commands which belongs in this category, namely the FROM > and TO commands. These specify a time period to restrict the analysis to. The simplest usage of these commands is FROM yyMMdd or FROM yyMMdd:hhmm, where yy represents the last > two digits of the year (analog assumes that the year is between 1970 and 2069), MM represents the month, dd is the date, hh the hour, and mm the minute. So, for example, to > analyse only requests from 1st July 1999 to 1pm on 15th June 2000 I would use the configuration

FROM 990701
TO 000615:1300

Alternatively, each of the components can be preceded by + or - to represent time relative to the time at which the program was invoked. In this case, the date can have more than 2 digits. This allows constructions like

FROM -01-00+01 # from tomorrow last year
TO -00-0131 # to the end of last month (OK even if last month didn't have 31 days)
FROM -00-00-112
TO -00-00-01 # statistics for the last 16 weeks
FROM -00-00-00:-06+01 # statistics for the last 6 hours

There are command line abbreviations +F and +T for the FROM and TO commands; for example, +T-00-00-01:1800 looks at statistics until 6pm yesterday. -F and -T turn off the from and to, as do FROM OFF and TO OFF.

Com esses, você provavelmente pode obter o que precisa. Eu arriscaria um palpite de que sua resposta está dentro das entradas "DE" e "TO" .cfg.

Como mais uma alternativa, você sempre pode pular as opções acima e simplesmente escrever um script em lote que moverá / arquivará ou excluirá a semana anterior de logfiles -out_ do local padrão "\ LogFiles \ W3SVC1" para dizer algo como " D: \ IIS-Logfiles-Archive ", e então executam o analisador analógico do dia. Alguns pseudocódigo que podem ajudar você a começar e seguir na direção certa seriam:

(e novamente, este é o código PSEUDO. Rude, feio e se encaixa como um suéter de lã molhada. Provavelmente não é necessário excluir o diretório, mas eu estou cribbing de outro script que eu uso , mmmkay?:)

net stop all_your_webservices_here
cd C:\WINDOWS\system32\LogFiles\W3SVC1
del /f /q .
cd c:\
rd /s /q C:\WINDOWS\system32\LogFiles\W3SVC1
mkdir C:\WINDOWS\system32\LogFiles\W3SVC1
net start all_your_webservices_here
"C:\Program Files\analog 6.0\analog.exe" && rmagic

Desculpe, esse último pouquinho lá está se você tiver o reportmagic instalado e funcionando também. Se você não fizer isso, eu recomendo. Torna analógico bonito, impressiona o gerenciamento. Mmmm, bom. Como sopa ou algo assim. :)

Dito isto, vou supor que você esteja querendo restringir a análise a uma semana, porque está demorando para que o analógico analise seus registros. (Este é um palpite totalmente aleatório, e colorido pela minha opinião, porque eu meio que / vi a mesma coisa quando inicialmente configurá-lo:)

Esteja avisado, uma vez que você analisar um conjunto de arquivos de log (digamos, alguns dias ou o valor de uma semana), pode demorar um pouco, porém re-running analógico para examinar qualquer arquivo de log seguinte é extremamente rápido. Para resumir; criar uma tarefa agendada para analógico para executar uma vez por dia e enquanto a execução inicial levará algum tempo, cada execução sucessiva será muito mais rápida que a primeira.

HTH, companheiro ...

    
por 15.06.2009 / 15:28
1

Cara, eu só pensei nisso depois de postar. Talvez isso ajude um pouco mais, já que a maioria dos links para: "Veja como Bob fez sua configuração, e aqui está como Jerry fez o seu aqui, o que é legal e tudo etc. " não são mais válidos.

Na esperança de que isso possa ajudá-lo um pouco mais, aqui está meu arquivo analog.cfg:

"#" The below WAS production, and became a test.
"#"
"#" LOGFILE C:\WINDOWS\system32\LogFiles\W3SVC1\ex%y%M%D.*
"#"
"#" It worked, but only gave us one logfile at a time.
"#"
"#" The below is a test.
"#"
"#" LOGFILE C:\WINDOWS\system32\LogFiles\W3SVC1*.log
"#"
"#" The above worked, because we want to parse ALL the logfiles,
"#" sequentially. Sticking with this for now.
"#"

"#"
"#" This line records how long it took to generate reports.
"#"

RUNTIME ON

"#"
"#" This line re-sets the logfile to localtime. Took a while to
"#" figure out as well.
"#"

LOGTIMEOFFSET -360
LOGFILE C:\WINDOWS\system32\LogFiles\W3SVC1*.log

"#"
"#" This line tells "analog" to dump it's output into "computer readable"
"#" format, for reportmagic to use. If we didn't use this, we would
"#" tell it to output it in html format for human, or browser-readable
"#" format.
"#"

OUTPUT COMPUTER

"#"
"#" This line tells analog specifically WHERE to dump it's output.
"#"

OUTFILE D:\inetpub\wwwroot\stats\report.dat

"#"
"#" These lines tell analog where the charts go.
"#"

LOCALCHARTDIR "D:\inetpub\wwwroot\stats\charts\%Y.%M."
CHARTDIR "charts/%Y.%M."

"#"
"#" These lines set the pretty text in the html
"#"

HOSTNAME "[My company's intranet]"
HOSTURL http://mycompany_intranet

"#"
"#" These lines tell analog how to resolve it's hostnames. We
"#" need to work on these some, because the "dnscache.txt" file
"#" simply continues to grow as this runs and results in us having
"#" to exert some maintenance. I was unable to get it to resolve
"#" DIRECTLY from the network, however I'm sure that it's possible
. "#"

DNS WRITE
DNSFILE dnscache.txt
DNSLOCKFILE dnslock.txt
DNSGOODHOURS 8

"#"
"#" Here's some pre-set, pre-module-load options that we need to set in the very beginning.
"#" They're kinda like the ones above, but you can consider them meta-tags or meta-variables.
"#"

REQLINKINCLUDE pages
REFLINKINCLUDE *
REDIRREFLINKINCLUDE *
FAILREFLINKINCLUDE *
SUBBROW /
SUBTYPE .gz,.Z

"#" Add whichever of these types of pages you have on your server, or others.

PAGEINCLUDE *.shtml
PAGEINCLUDE *.html
PAGEINCLUDE *.asp
PAGEINCLUDE *.jsp
PAGEINCLUDE *.js
PAGEINCLUDE *.cfm
PAGEINCLUDE *.css
PAGEINCLUDE *.pl
PAGEINCLUDE *.php
PAGEINCLUDE *.swf

TYPEALIAS .html ".html [Hypertext Markup Language]"
TYPEALIAS .htm ".htm [Hypertext Markup Language]"
TYPEALIAS .shtml ".shtml [Server-parsed HTML]"
TYPEALIAS .ps ".ps [PostScript]"
TYPEALIAS .gz ".gz [Gzip compressed files]"
TYPEALIAS .tar.gz ".tar.gz [Compressed archives]"
TYPEALIAS .jpg ".jpg [JPEG graphics]"
TYPEALIAS .jpeg ".jpeg [JPEG graphics]"
TYPEALIAS .gif ".gif [GIF graphics]"
TYPEALIAS .png ".png [PNG graphics]"
TYPEALIAS .txt ".txt [Plain text]"
TYPEALIAS .cgi ".cgi [CGI scripts]"
TYPEALIAS .pl ".pl [Perl scripts]"
TYPEALIAS .css ".css [Cascading Style Sheets]"
TYPEALIAS .class ".class [Java class files]"
TYPEALIAS .pdf ".pdf [Adobe Portable Document Format]"
TYPEALIAS .zip ".zip [Zip archives]"
TYPEALIAS .hqx ".hqx [Macintosh BinHex files]"
TYPEALIAS .exe ".exe [Executables]"
TYPEALIAS .wav ".wav [WAV sound files]"
TYPEALIAS .avi ".avi [AVI movies]"
TYPEALIAS .arc ".arc [Compressed archives]"
TYPEALIAS .mid ".mid [MIDI sound files]"
TYPEALIAS .mp3 ".mp3 [MP3 sound files]"
TYPEALIAS .doc ".doc [Microsoft Word document]"
TYPEALIAS .rtf ".rtf [Rich Text Format]"
TYPEALIAS .mov ".mov [Quick Time movie]"
TYPEALIAS .mpg ".mpg [MPEG movie]"
TYPEALIAS .mpeg ".mpeg [MPEG movie]"
TYPEALIAS .asp ".asp [Active Server Pages]"
TYPEALIAS .jsp ".jsp [Java Server Pages]"
TYPEALIAS .cfm ".cfm [Cold Fusion]"
TYPEALIAS .php ".php [PHP]"
TYPEALIAS .js ".js [JavaScript code]"

"#" x=General
"#" 1=Yearly Q=Quarterly m=Monthly W=Weekly D=Daily d=DailySum
"#" H=Hourly h=HourlySum w=HoursOfWeek 4=15minly 6=15minlySum 5=5minly
"#" 7=5minlySum
"#" o=Domain Z=Organisat. S=Host s=RefSite f=ReferrerURL
"#" n=SearchWord N=SearchQuery B=Browser b=BrowserSum p=OS
"#" i=Dir. r=File t=FileType z=FileSize c=Status
"#" I=Failure L=FailHost K=FailRef E=Redir l=RedirHost k=RedirRef
"#" v=VHost R=RedirVHost M=FailVHost u=User j=RedirUser J=FailUser
"#" y=IntSearchWord Y=IntSearchQuery P=ProcTime

WEEKBEGINSON MONDAY
MARKCHAR +
REPSEPCHAR none
COMPSEP " "
RAWBYTES OFF
BYTESDP 2
NOROBOTS ON
SEARCHCHARCONVERT ON
BARSTYLE i
PNGIMAGES ON
HTMLPAGEWIDTH 65
ASCIIPAGEWIDTH 76
MINGRAPHWIDTH 15
ALLCHART ON
ALLGRAPH P


"#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#"
"#" SERVICE REPORTS - HOW WELL WE ARE DELIVERING THE PAGES "#"
"#" "#"
"#" Status - whether the pages were delivered okay, or failed "#"
"#" Proctime - time to dispatch files "#"
"#" Failure - people requests these pages, and it failed "#"
"#" Failhost, Failref - failures arose from links on these hosts, on these URLs "#"
"#" Redir - people requested these files but were redirected "#"
"#" Redirhost, redirref - redirections arose from links on these hosts and URLs "#"
"#" "#"
"#" COLS, CHART, SORTBY, FLOOR, SUB, ARGS, LINKINCLUDE - as above "#"
"#" 304ISSUCCESS - 304 Not Modified means user saw a cached copy: i.e. success! "#"
"#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#""#"

REPORTORDER x1QmWDdHwh4657oZSlLujJkKfsNnBbpvRMcPztiEIYyr

DOMAIN ON
DOMCHART ON
DOMCOLS PQ
DOMSORTBY PAGES
DOMFLOOR -500p
SUBDOMAIN ...
SUBDOMSORTBY PAGES
SUBDOMFLOOR 1.5%p

DIRECTORY ON
DIRCHART ON
DIRCOLS PQc
"#"
"#" Excluding all "local" and "images" directories, as per web team
"#" request.
"#"
DIREXCLUDE */local
DIREXCLUDE /images
"#"
DIRSORTBY PAGES7
SUBDIRSORTBY PAGES7
DIRFLOOR 0q
SUBDIRFLOOR 0q
SUBDIR /
////

FILETYPE ON
TYPECHART ON
TYPECOLS Rb
TYPESORTBY BYTES
SUBTYPE .gz,.Z

SIZE ON
SIZECHART ON
SIZECOLS ScR

ORGANISATION ON
ORGCHART ON
ORGCOLS Pb
ORGSORTBY PAGES
ORGFLOOR -500p
SUBORGSORTBY PAGES
SUBORG ....com
SUBORGFLOOR 25.0%p

HOST ON
HOSTCHART ON
HOSTCOLS Pb
HOSTSORTBY PAGES
HOSTFLOOR -500p

BROWSERREP ON
BROWREPCHART ON
BROWREPCOLS Rb
BROWREPSORTBY REQUESTS
BROWREPFLOOR -400r

BROWSERSUM ON
BROWSUMCHART ON
BROWSUMCOLS PQq
BROWSUMSORTBY PAGES7
BROWSUMFLOOR -200q
SUBBROW ///
SUBBROWSORTBY PAGES7
SUBBROWFLOOR -400q

STATUS ON
STATUSCHART ON
STATUSCOLS R
STATUSSORTBY ALPHABETICAL
STATUSFLOOR -200r
304ISSUCCESS ON

OSREP ON
OSCHART ON
OSCHARTEXPAND Windows
OSCOLS PQq
OSSORTBY PAGES7
OSFLOOR -200q
SUBOSSORTBY PAGES7
SUBOSFLOOR -500q

PROCTIME ON
PROCTIMECOLS R
PROCTIMECHART ON

USER ON
USERCOLS Rb
USERCHART ON
USERSORTBY REQUESTS
USERFLOOR -500r

REQUEST ON
REQCHART ON
REQCOLS RSc
REQSORTBY REQUESTS7
REQFLOOR 0.5%s
REQARGSSORTBY REQUESTS7
REQARGSFLOOR 0.5%s
REQEXCLUDE *
REQINCLUDE *.html
REQINCLUDE *.htm
REQINCLUDE *.zip
REQINCLUDE *.gz
REQINCLUDE *.Z
REQINCLUDE .pdf
REQLINKEXCLUDE /cgi-bin/

REQLINKEXCLUDE /scripts/*
REQLINKINCLUDE *
REQLINKINCLUDE *.zip
REQLINKINCLUDE *.pdf

FAILURE ON
FAILCHART ON
FAILCOLS RS
FAILSORTBY REQUESTS7
FAILFLOOR -300s
FAILARGSSORTBY REQUESTS7
FAILARGSFLOOR 10s
FAILLINKEXCLUDE *

FAILHOST ON
FAILHOSTCHART ON
FAILHOSTCOLS RS
FAILHOSTSORTBY REQUESTS7
FAILHOSTFLOOR -500s

FAILUSER ON
FAILUSERCOLS R
FAILUSERCHART ON
FAILUSERSORTBY REQUESTS
FAILUSERFLOOR -400r

FAILREF ON
FAILREFCHART ON
FAILREFCOLS RS
FAILREFSORTBY REQUESTS7
FAILREFFLOOR -300s
FAILREFARGSSORTBY REQUESTS7
FAILREFARGSFLOOR 10s
FAILREFLINKINCLUDE *

REDIR ON
REDIRCHART ON
REDIRCOLS SR
REDIRSORTBY REQUESTS7
REDIRFLOOR -300s
REDIRARGSSORTBY REQUESTS7
REDIRARGSFLOOR 500s

REDIRHOST ON
REDIRHOSTCHART ON
REDIRHOSTCOLS SR
REDIRHOSTSORTBY REQUESTS7
REDIRHOSTFLOOR -500s

REDIRREF ON
REDIRREFCHART ON
REDIRREFCOLS RS
REDIRREFSORTBY REQUESTS7
REDIRREFFLOOR -300s
REDIRREFARGSSORTBY REQUESTS7
REDIRREFARGSFLOOR 500s
REDIRREFLINKINCLUDE *

REFSITE ON
REFSITECHART ON
REFSITECOLS PQ
REFSITESORTBY PAGES7
REFSITEFLOOR -500q
REFDIRSORTBY PAGES7
REFDIRFLOOR 50.5%q

REFERRER ON
REFCHART ON
REFCOLS PQ
REFSORTBY PAGES7
REFLINKINCLUDE *
REFFLOOR -500q
REFARGSSORTBY PAGES7
REFARGSFLOOR 50.5%q

VHOST ON
VHOSTCHART ON
VHOSTCOLS Rb
VHOSTFLOOR -200b
VHOSTSORTBY BYTES

REDIRVHOST ON
REDIRVHOSTCHART ON
REDIRVHOSTCOLS R
REDIRVHOSTFLOOR -200r
REDIRVHOSTSORTBY REQUESTS

REDIRUSER ON
REDIRUSERCOLS R
REDIRUSERCHART ON
REDIRUSERSORTBY REQUESTS
REDIRUSERFLOOR -200r

SEARCHWORD ON
SEARCHWORDCHART ON
SEARCHWORDCOLS PQq
SEARCHWORDSORTBY PAGES7
SEARCHWORDFLOOR 1.5%q

SEARCHQUERY ON
SEARCHQUERYCHART ON
SEARCHQUERYCOLS Q
SEARCHQUERYSORTBY PAGES7
SEARCHQUERYFLOOR -500q

INTSEARCHQUERY ON
INTSEARCHQUERYCOLS R
INTSEARCHQUERYCHART ON
INTSEARCHQUERYFLOOR -300r
INTSEARCHQUERYSORTBY REQUESTS

INTSEARCHWORD ON
INTSEARCHWORDCOLS R
INTSEARCHWORDCHART ON
INTSEARCHWORDFLOOR -400r
INTSEARCHWORDSORTBY REQUESTS

Mais uma vez, espero que isso ajude também e boa sorte, companheiro!

    
por 15.06.2009 / 15:46
0

Para análise personalizada de arquivos de log, há um bom script publicado no link . Ele é escrito para logs do servidor da web, mas você pode usá-lo como ponto de partida para escrever seu próprio analisador de logs para logs de qualquer tipo.

Patrick

    
por 28.06.2009 / 18:00