Logar URLs de solicitação do Firefox para stdout ou um arquivo?

2

Eu preciso obter um log de todos os arquivos recuperados da Internet quando eu carregar uma página no Firefox; Eu realmente não posso usar o Firefox Web Developer (Ferramentas) / Network Monitor porque algumas páginas que usam Flash, tendem a travar o Firefox quando eu tenho o Network Monitor aberto (mesmo que elas pareçam carregar corretamente sem o Network monitor). Por isso, preciso de obter uma lista de URLs que "sobrevivam" a tais falhas.

Estou ciente de que posso obter um log de tráfego de rede do Firefox usando variáveis de ambiente, por exemplo, no Linux (isso funciona no Firefox 43, mas não consegui fazê-lo funcionar com MOZ_LOG_MODULES / MOZ_LOG_FILE ):

NSPR_LOG_MODULES=timestamp,nsHttp:3,sync NSPR_LOG_FILE=/tmp/ff.log /path/to/firefox/firefox

No entanto, isso imprime informações muito detalhadas com solicitações / respostas HTTP completas, como:

2017-01-07 19:18:49.799922 UTC - -1371059392[b726e9c0]: Http2Session::ALPNCallback version=303
2017-01-07 19:18:49.981528 UTC - -1220458752[b726e0c0]: http request [
2017-01-07 19:18:49.981551 UTC - -1220458752[b726e0c0]:   POST / HTTP/1.1
2017-01-07 19:18:49.981560 UTC - -1220458752[b726e0c0]:   Host: ocsp.int-x3.letsencrypt.org
2017-01-07 19:18:49.981567 UTC - -1220458752[b726e0c0]:   User-Agent: Mozilla/5.0 (X11; Linux i686; rv:4
3.0) Gecko/20100101 Firefox/43.0
2017-01-07 19:18:49.981574 UTC - -1220458752[b726e0c0]:   Accept: text/html,application/xhtml+xml,applic
ation/xml;q=0.9,*/*;q=0.8
2017-01-07 19:18:49.981580 UTC - -1220458752[b726e0c0]:   Accept-Language: en-US,en;q=0.5
2017-01-07 19:18:49.981589 UTC - -1220458752[b726e0c0]:   Accept-Encoding: gzip, deflate
2017-01-07 19:18:49.981595 UTC - -1220458752[b726e0c0]:   DNT: 1
2017-01-07 19:18:49.981604 UTC - -1220458752[b726e0c0]:   Content-Length: 85
2017-01-07 19:18:49.981611 UTC - -1220458752[b726e0c0]:   Content-Type: application/ocsp-request
2017-01-07 19:18:49.981617 UTC - -1220458752[b726e0c0]:   Connection: keep-alive
2017-01-07 19:18:49.981639 UTC - -1220458752[b726e0c0]: ]
2017-01-07 19:18:50.071135 UTC - -1371059392[b726e9c0]: http response [
2017-01-07 19:18:50.071174 UTC - -1371059392[b726e9c0]:   HTTP/1.1 200 OK
2017-01-07 19:18:50.071195 UTC - -1371059392[b726e9c0]:   Server: nginx
2017-01-07 19:18:50.071203 UTC - -1371059392[b726e9c0]:   Content-Type: application/ocsp-response
2017-01-07 19:18:50.071210 UTC - -1371059392[b726e9c0]:   Content-Length: 527
2017-01-07 19:18:50.071218 UTC - -1371059392[b726e9c0]:   Etag: "3FFFC13100849446EA3D8FA68B0077AF4792776825416386C5CF76104592E0CE"
2017-01-07 19:18:50.071225 UTC - -1371059392[b726e9c0]:   Last-Modified: Wed, 04 Jan 2017 00:00:00 UTC
2017-01-07 19:18:50.071233 UTC - -1371059392[b726e9c0]:   Cache-Control: public, no-transform, must-revalidate, max-age=24762
2017-01-07 19:18:50.071240 UTC - -1371059392[b726e9c0]:   Expires: Sun, 08 Jan 2017 02:11:32 GMT
2017-01-07 19:18:50.071247 UTC - -1371059392[b726e9c0]:   Date: Sat, 07 Jan 2017 19:18:50 GMT
2017-01-07 19:18:50.071254 UTC - -1371059392[b726e9c0]:   Connection: keep-alive
2017-01-07 19:18:50.071259 UTC - -1371059392[b726e9c0]: ]

... e eu realmente só preciso dos URLs completos, algo como as listas mostradas pelo Network Monitor do Firefox:

Então, existe uma maneira de persuadir o Firefox a despejar apenas as URLs acessadas em tempo real para stdout ou para um arquivo, de modo que, se o Firefox travar, eu ainda tenha um log de URLs acessados?

    
por user683118 07.01.2017 / 20:57

1 resposta

0

Eu preciso obter uma lista de URLs que "sobreviverão" a essas falhas.

O HTTP Request Logger add-on do Firefox parece atender suas necessidades exatas:

Logs HTTP requests (referrer, HTTP method and URL) to a text file for security and auditing reasons.

Fonte Logger de solicitações de HTTP

Sometimes it might be useful to log each and every URL that is accessed through your Mozilla Firefox web browser. It might be for security, auditing or other reasons that you want to log accessed URLs. I developed the HTTP Request Logger add-on in order to track and discover a malicious web page that triggered the CVE-2010-4452 vulnerability in Oracle Java through a specially crafted applet tag and a respective Java applet class. You can download and install the add-on from here: HTTP Request Logger. You can also view the project on GitHub: prekageo/http-request-logger.

The add-on implements a JavaScript XPCOM component. This component is registered in chrome.manifest. The component registers an observer for http-on-modify-request topics. The observer’s callback function is called when a new HTTP request is ready to be sent. The function appends into a simple text file located on the user’s desktop a line consisting of the referrer, HTTP method and URL of the HTTP request.

Fonte Add-on HTTP Request Logger para Mozilla Firefox

Não sei se a saída sobreviverá a uma falha do Firefox.

A saída é gravada em http-request-log.txt na área de trabalho (nome completo C:\Users\username\Desktop\http-request-log.txt

Exemplo de saída (a partir do recarregamento de link ):

(none) GET https://superuser.com/questions
(none) POST http://clients1.google.com/ocsp
https://superuser.com/questions GET https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js
https://superuser.com/questions GET https://cdn.sstatic.net/Js/stub.en.js?v=5d747a3197db
https://superuser.com/questions GET https://cdn.sstatic.net/Sites/superuser/all.css?v=39a3353d7dea
https://superuser.com/questions GET https://www.gravatar.com/avatar/1c46f449aee8300e9bcf2e6462e0127c?s=32&d=identicon&r=PG
https://chat.stackexchange.com/rooms/114/ask-a-super-user-moderator POST https://chat.stackexchange.com/events
https://superuser.com/questions GET https://i.stack.imgur.com/Ohyu2.png?s=48&g=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/adab46b83f2276ff0a2da05c3aea1a1e?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/49427fda382a6ebf890c6401eb1dc117?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/8c7bfdd599703d0f59b8adb35c33384d?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/ce3dd556aaf605869d870670f4addad2?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/b79191086ccc83d2efbf0346a9498552?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/2b4de46accb1b608fc45963de8c171d8?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/b61ea906fab08361132d0f98ffb89906?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://i.stack.imgur.com/pw4Ug.gif?s=32&g=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/93ec7cff90c748bc4f4da0c540e68bfc?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/5df3eeb6bd2ca404e3b21a08721ad45d?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/c6bac24248317129b390d60b7c088e40?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/1bcd453599a20c9eb444902d81c827c0?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/2e6a415fd9d93e089c5ac7138b306c8d?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://i.stack.imgur.com/VyvYN.png?s=32&g=1
https://superuser.com/questions GET https://lh5.googleusercontent.com/-snn4nMpy9Pw/AAAAAAAAAAI/AAAAAAAAATY/f_fJpek-wHo/photo.jpg?sz=32
https://superuser.com/questions GET https://i.stack.imgur.com/xObp1.png?s=32&g=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/994136ff5c5a426f77ded906bc1323b0?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://i.stack.imgur.com/98CKO.png?s=32&g=1
https://superuser.com/questions GET https://graph.facebook.com/867261659995682/picture?type=large
https://superuser.com/questions GET https://i.stack.imgur.com/oGmjT.png?s=32&g=1
https://superuser.com/questions GET https://i.stack.imgur.com/gaAXg.jpg?s=32&g=1
https://superuser.com/questions GET https://lh6.googleusercontent.com/-EwJ5_a_MCkc/AAAAAAAAAAI/AAAAAAAAAD8/Hra648Whrto/photo.jpg?sz=32
https://superuser.com/questions GET https://www.gravatar.com/avatar/bb3b38b5e3dac0b2df433d5c78cf9fc5?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/763da2b58630f974d468d88c26ac2e4e?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/c273f9992292a61959df77bca08665a3?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/41e856251e65f04dd94d72eed93e48e6?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/bc9f58c57918b259c487410155bd9e23?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/7db005472dc422ca5cbb143a5fd36125?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/b08fc2b0be91884d666ef12eac8d0d8f?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/6d1f8e5a125a589884b3784caa11736b?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://i.stack.imgur.com/ORos2.png?s=32&g=1
https://superuser.com/questions GET https://lh3.googleusercontent.com/-2DRzHlFuo4A/AAAAAAAAAAI/AAAAAAAAAJ4/UoePoxsDseg/photo.jpg?sz=32
https://superuser.com/questions GET https://i.stack.imgur.com/cYO05.jpg?s=32&g=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/bb7cd698db4bb8e9fa866d5154a05355?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/f9c738932874403fd70f2c617951f9e2?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://i.stack.imgur.com/Dqrx8.png?s=32&g=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/fc34739487a5e7e115acdbadf95c8fb3?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://lh4.googleusercontent.com/-Bah9vRlCwX8/AAAAAAAAAAI/AAAAAAAAAF4/sHjIDOHJT-c/photo.jpg?sz=32
https://superuser.com/questions GET https://www.gravatar.com/avatar/01cc31c9b124d090aeb3e06f94bb8ed8?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/417ffb01cabc10df5dc2ffad281e6a65?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/52eda3f9c34ef88ca164606301366366?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/a7da99412e7177afd0951d7511bdfec8?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/b3ac28f02a59435e921d244c0d82fbbf?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/e880c06c3667fe7c282f0dcb22e48525?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/f7a735cfc6d5fdf3ef44bdbb94fe3beb?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/7ebac3a51e6820a973e14c5b8b28e452?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://www.gravatar.com/avatar/1859855084a2f798e294422cf6ba2d31?s=32&d=identicon&r=PG&f=1
https://superuser.com/questions GET https://www.gravatar.com/avatar/ee06ba399c5eaf3f5a337a96207c2768?s=32&d=identicon&r=PG
https://superuser.com/questions GET https://graph.facebook.com/100000616410966/picture?type=large
https://superuser.com/questions GET https://engine.adzerk.net/ados.js
https://superuser.com/questions GET https://www.google-analytics.com/analytics.js
https://superuser.com/questions GET https://secure.quantserve.com/quant.js
https://superuser.com/questions GET https://sb.scorecardresearch.com/beacon.js
https://cdn.sstatic.net/Sites/superuser/all.css?v=39a3353d7dea GET https://cdn.sstatic.net/img/share-sprite-new.svg?v=78be252218f3
https://cdn.sstatic.net/Sites/superuser/all.css?v=39a3353d7dea GET https://cdn.sstatic.net/Sites/superuser/img/sprites.svg?v=6f5b3bbd35ac
https://cdn.sstatic.net/Sites/superuser/all.css?v=39a3353d7dea GET https://cdn.sstatic.net/img/favicons-sprite16.png?v=41af8148f8e0
(none) GET http://i.stack.imgur.com/05rE1.png
https://superuser.com/questions GET https://cdn.rawgit.com/ofirdagan/cross-domain-local-storage/d779a81a6383475a1bf88595a98b10a8bd5bb4ae/dist/scripts/xdLocalStorage.min.js
https://superuser.com/questions GET https://cdn.sstatic.net/Js/full.en.js?v=53cad7e6923d
https://superuser.com/questions GET https://scontent.xx.fbcdn.net/v/t1.0-1/p200x200/15621850_1188627624525749_7199393790071419683_n.jpg?oh=e4fa46a17dc0d6b646a664641891f18b&oe=58D7DB42
https://superuser.com/questions GET https://scontent.xx.fbcdn.net/v/t1.0-1/p200x200/14141922_1247416555288880_9079557968728006677_n.jpg?oh=05088f1d9fc7ee9e8c50a2795c018cb2&oe=59132CC4
https://superuser.com/questions GET https://maxcdn.bootstrapcdn.com/font-awesome/4.5.0/css/font-awesome.min.css
(none) GET https://i.stack.imgur.com/05rE1.png
https://superuser.com/questions GET https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20json%20where%20url%3D%22http%3A%2F%2Fstackexchange.com%2Fhot-questions-for-mobile%22&format=json
https://superuser.com/questions GET https://api.stackexchange.com/2.2/users/607889;194121;683117;682924;662836;618039;683114;683118;593210;3365;683110;529857;683096;683101;462324;683103;247768;661200;197638;683086;683095;658520;683078;654677;683081;683080;200839;487419;407559;683068;34923;551116;683059;387824;683053;683054;419223;297715;683050;292907;293642;683035;683044;683042;683045;124122;683041;682501;61846;581216?site=superuser&key=lL1S1jr2m*DRwOvXMPp26g((&access_token=eIDDpZG(uvLciqdnktsQmw))
https://superuser.com/questions GET https://api.stackexchange.com/2.2/questions?pagesize=5&order=desc&sort=activity&site=meta.superuser
https://superuser.com/questions GET https://rawgit.com/shu8/SE-Answers_scripts/master/dupeClosedMigratedCSS.css
https://superuser.com/questions GET https://superuser.com/questions/1164534
https://superuser.com/questions GET https://superuser.com/questions/1164524
https://superuser.com/questions GET https://superuser.com/questions/1164522
https://superuser.com/questions GET https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20html%20where%20url%3D%22http%3A%2F%2Fsuperuser.com%2Fquestions%2F1164516%22&diagnostics=true
https://superuser.com/questions GET https://query.yahooapis.com/v1/public/yql?q=select%20*%20from%20html%20where%20url%3D%22http%3A%2F%2Fsuperuser.com%2Fquestions%2F1164496%22&diagnostics=true
https://superuser.com/questions GET https://cdn-prom.sstatic.net/WinterBash/js/qa-wb.js?v=9
(none) POST http://ocsp.digicert.com/
(none) GET https://qa.sockets.stackexchange.com/
https://superuser.com/questions GET https://superuser.com/questions/1164534/what-happens-if-i-sleep-windows-10-while-programs-are-running
https://superuser.com/questions GET https://superuser.com/questions/1164524/select-the-driver-to-be-installed-windows-7-home-premium-x86
https://superuser.com/questions GET https://superuser.com/questions/1164522/how-can-i-build-a-sound-display-to-show-the-alsa-master-volume-for-the-fn-keys
(none) POST http://ocsp.digicert.com/
(none) POST http://ocsp.digicert.com/
https://superuser.com/questions GET https://i.stack.imgur.com/Kmqx8.png
https://superuser.com/questions GET https://cdn-prom.sstatic.net/WinterBash/css/wb-include.css?22
https://superuser.com/questions GET https://cdn-prom.sstatic.net/WinterBash/js/theactualhats.js?v=8
https://superuser.com/questions GET https://metasmoke.erwaysoftware.com/xdom_storage.html
https://superuser.com/questions GET https://cdn-prom.sstatic.net/WinterBash/js/hat-78404333.js?v=8
https://superuser.com/questions GET https://winterbash2016.stackexchange.com/api/current-hats?callback=winterBashCurrentHatsNaN&userids=337631%3B607889%3BNaN&host=superuser.com&_=1483822563839
https://metasmoke.erwaysoftware.com/xdom_storage.html GET https://cdn.rawgit.com/ofirdagan/cross-domain-local-storage/d779a81a6383475a1bf88595a98b10a8bd5bb4ae/dist/scripts/xdLocalStoragePostMessageApi.min.js
https://superuser.com/questions GET https://cdn-prom.sstatic.net/WinterBash/js/hat-78278539.js?v=8
https://superuser.com/questions/1164582/logging-firefox-request-urls-to-stdout-or-a-file GET https://winterbash2016.stackexchange.com/api/current-hats?callback=winterBashCurrentHatsNaN&userids=NaN&host=superuser.com&_=1483822104590
https://superuser.com/questions/1164582/logging-firefox-request-urls-to-stdout-or-a-file POST https://superuser.com/posts/1164582/editor-heartbeat/answer
https://0.docs.google.com/document/d/1AFlWPYR7xnrQRsrVTS2Q_-sLhK7Ew1hLDz5oNKlu_n0/xdbcm.html?internal=false&gcp=%7B%22qlc%22%3Anull%2C%22elh%22%3Afalse%2C%22eoo%22%3Atrue%7D&rt=j&xpc=%7B%22cn%22%3A%22docs%22%2C%22tp%22%3A1%2C%22osh%22%3Atrue%2C%22ppu%22%3A%22https%3A%2F%2Fdocs.google.com%2Frobots.txt%22%2C%22lpu%22%3A%22https%3A%2F%2F0.docs.google.com%2Frobots.txt%22%7D GET https://0.docs.google.com/document/d/1AFlWPYR7xnrQRsrVTS2Q_-sLhK7Ew1hLDz5oNKlu_n0/bind?id=1AFlWPYR7xnrQRsrVTS2Q_-sLhK7Ew1hLDz5oNKlu_n0&sid=1cc64febbb900314&VER=8&token=AC4w5VilbfesZ4ZFdH5AoL0ZwlR4uNlZ2g%3A1483822100782&lsq=1482517927748&u=18029933486027320144&c=1&w=1&gsi=0&smv=2&RID=rpc&SID=EA75E4E583E22ADB&CI=1&AID=48&TYPE=xmlhttp&zx=r45bxqz5hyo4&t=1
https://superuser.com/questions/1164582/logging-firefox-request-urls-to-stdout-or-a-file GET https://winterbash2016.stackexchange.com/api/current-hats?callback=winterBashCurrentHatsNaN&userids=NaN&host=superuser.com&_=1483822104591
https://chat.stackexchange.com/rooms/114/ask-a-super-user-moderator POST https://chat.stackexchange.com/events
    
por 07.01.2017 / 21:58