Removendo datas de um arquivo CSV

Question

Eu tenho uma lista enorme de tweets, mas eu quero remover muitos deles antes de uma determinada data. Eu acho que eu preciso usar sed, awk ou grep para remover esses caras, mas não tenho certeza sobre a sintaxe. O formato para isso está na segunda coluna na forma de "2017-9-18 XX: XX: XX" e diz que eu queria remover os tweets antes de 2017-9-15 ".

Obrigado uma tonelada pessoal!

    

Answer 1

Você pode usar dategrep . De perldoc /usr/local/bin/dategrep :

NAME
    dategrep - print lines matching a date range

SYNOPSIS
      dategrep --start "12:00" --end "12:15" --format "%b %d %H:%M:%S" syslog
      dategrep --end "12:15" --format "%b %d %H:%M:%S" syslog
      dategrep --last-minutes 5 --format "%b %d %H:%M:%S" syslog
      dategrep --last-minutes 5 --format rsyslog syslog
      cat syslog | dategrep --end "12:15"

DESCRIPTION
    Do you even remember how often in your life you needed to find lines in a
    log file falling in a date range? And how often you build brittle regexs
    in grep to match entries spanning over a hour change?

    dategrep hopes to solve this problem once and for all.

...

INSTALLATION
    It is possible to install this script via perl normal install routines.

      perl Makefile.PL && make && make install

    Or via CPAN:

      cpan App::dategrep

    You can also install one of the two prebuild versions, which already
    include all or some of dategrep's dependencies. Which to choose mainly
    depends on how hard it is for you to install Date::Manip. The small
    version is just 22.3KB big and includes all libraries except Date::Manip.
    The big one packs everything in a nice, neat package for you, but will
    cost you almost 10MB of disk space. Both are always included in the latest
    release <https://github.com/mdom/dategrep/releases/latest>.

    So, to install the big version you could just type:

      wget -O /usr/local/bin/dategrep https://github.com/mdom/dategrep/releases/download/v0.58/dategrep-standalone-big
      chmod +x /usr/local/bin/dategrep

    And for the small one (with the apt-get for Debian):

      apt-get install libdate-manip-perl
      wget -O /usr/local/bin/dategrep https://github.com/mdom/dategrep/releases/download/v0.58/dategrep-standalone-small
      chmod +x /usr/local/bin/dategrep

    

Answer 2

Expressões regulares tornam isso um problema simples. A primeira versão produz datas a partir de 2017-9-01.

grep -E "2017-([9]|[0-1][0-9])" file > output_file

Este segundo exemplo filtra ainda mais a saída para excluir datas antes de 2017-9-15. Mas somente se o dia do mês for zero preenchido.

grep -E "2017-([9]|[0-1][0-9])-([0-9]|[0-9][0-9])" file | grep -Ev "2017-9-(0[0-9]|[0-1][0-5])" > output_file

Cada par de colchetes representa um único dígito. O caractere | significa ou em regex. Consulte o Guia do Bash para Iniciantes Capítulo 4. Expressões regulares para obter mais detalhes.