overlay animação transparente sobre vídeo com ffmpeg

Question

A tarefa geral é sobrepor infográficos animados em vídeo.

O que eu gostaria, como resultado, mostrado neste vídeo link

Bela linha ciano semitransparente animada em 00: 08-00: 11

Descobri como fazer isso sobrepondo imagens uma por uma com "convert" do ImageMagick. Funciona bem, mas é lento.

Tentou criar e sobrepor vídeo com transparência de acordo com as respostas neste tópico stackoverflow.com/questions/644684/turn-image-sequence-into-video-with-transparency

e algumas improvisações, assim como:

ffmpeg -loop 1 -i ./frames/0253.png -r 30 -t 3 -pix_fmt rgba -vcodec png frame.mov

crie um vídeo com canal alfa de fotos animadas

ffmpeg -pattern_type glob -i 'angle/*.png' -r 30 -pix_fmt rgba -vcodec png z.mov

E vídeos sobrepostos

ffmpeg -i frame.mov -i z.mov -filter_complex "[0:0][1:0]overlay[out]" -shortest -map [out] -pix_fmt rgba -vcodec png test.mov

Mas tudo resulta em uma imagem pixelizada aproximada

Gostariadereceberalgumasdicassobreoqueestouperdendoaquieoquemaiseupoderiafazer...

Saídaffmpegcompleta:

alexandrov@ThinkPad-Edge:~/Documents/training20150109/edit$ffmpeg-loop1-i./frames/0253.png-r30-t3-pix_fmtrgba-vcodecpngframe.mov>>log.txtffmpegversion2.5.gitCopyright(c)2000-2015theFFmpegdevelopersbuiltonJan11201519:12:34withgcc4.8(Ubuntu4.8.2-19ubuntu1)configuration:--prefix=/home/alexandrov/ffmpeg_build--extra-cflags=-I/home/alexandrov/ffmpeg_build/include--extra-ldflags=-L/home/alexandrov/ffmpeg_build/lib--bindir=/home/alexandrov/bin--enable-gpl--enable-libass--enable-libfdk-aac--enable-libfreetype--enable-libmp3lame--enable-libopus--enable-libtheora--enable-libvorbis--enable-libvpx--enable-libx264--enable-nonfree--enable-x11grablibavutil54.16.100/54.16.100libavcodec56.20.100/56.20.100libavformat56.18.100/56.18.100libavdevice56.3.100/56.3.100libavfilter5.7.100/5.7.100libswscale3.1.101/3.1.101libswresample1.1.100/1.1.100libpostproc53.3.100/53.3.100Input#0,png_pipe,from'./frames/0253.png':Duration:N/A,bitrate:N/AStream#0:0:Video:png,rgb24,1280x720[SAR1:1DAR16:9],25fps,25tbr,25tbn,25tbcFile'frame.mov'alreadyexists.Overwrite?[y/N]yOutput#0,mov,to'frame.mov':Metadata:encoder:Lavf56.18.100Stream#0:0:Video:png(png/0x20676E70),rgba,1280x720[SAR1:1DAR16:9],q=2-31,200kb/s,30fps,15360tbn,30tbcMetadata:encoder:Lavc56.20.100pngStreammapping:Stream#0:0->#0:0(png(native)->png(native))Press[q]tostop,[?]forhelpframe=5fps=0.0q=0.0size=5321kBtime=00:00:00.10bitrate=435924frame=7fps=6.6q=0.0size=8869kBtime=00:00:00.16bitrate=435922frame=11fps=5.9q=0.0size=15964kBtime=00:00:00.30bitrate=435922frame=13fps=5.4q=0.0size=19512kBtime=00:00:00.36bitrate=435922frame=17fps=5.5q=0.0size=26607kBtime=00:00:00.50bitrate=435922frame=19fps=5.1q=0.0size=30154kBtime=00:00:00.56bitrate=435921frame=23fps=5.3q=0.0size=37249kBtime=00:00:00.70bitrate=435922frame=25fps=5.0q=0.0size=40797kBtime=00:00:00.76bitrate=435921frame=29fps=5.1q=0.0size=47892kBtime=00:00:00.90bitrate=435921frame=31fps=4.9q=0.0size=51439kBtime=00:00:00.96bitrate=435921frame=35fps=4.9q=0.0size=58534kBtime=00:00:01.10bitrate=435921frame=37fps=4.9q=0.0size=62082kBtime=00:00:01.16bitrate=435921frame=41fps=4.9q=0.0size=69177kBtime=00:00:01.30bitrate=435921frame=43fps=4.8q=0.0size=72725kBtime=00:00:01.36bitrate=435921frame=47fps=4.9q=0.0size=79820kBtime=00:00:01.50bitrate=435921frame=49fps=4.7q=0.0size=83367kBtime=00:00:01.56bitrate=435921frame=53fps=4.9q=0.0size=90462kBtime=00:00:01.70bitrate=435921frame=55fps=4.7q=0.0size=94010kBtime=00:00:01.76bitrate=435921frame=60fps=4.8q=0.0size=102879kBtime=00:00:01.93bitrate=435921frame=63fps=4.7q=0.0size=108200kBtime=00:00:02.03bitrate=435921frame=66fps=4.7q=0.0size=113521kBtime=00:00:02.13bitrate=435921frame=69fps=4.7q=0.0size=118843kBtime=00:00:02.23bitrate=435921frame=73fps=4.7q=0.0size=125938kBtime=00:00:02.36bitrate=435921frame=78fps=4.7q=0.0size=134807kBtime=00:00:02.53bitrate=435921frame=81fps=4.7q=0.0size=140128kBtime=00:00:02.63bitrate=435921frame=84fps=4.7q=0.0size=145449kBtime=00:00:02.73bitrate=435921frame=87fps=4.7q=0.0size=150770kBtime=00:00:02.83bitrate=435921frame=90fps=4.7q=0.0size=156092kBtime=00:00:02.93bitrate=435921frame=90fps=4.6q=0.0Lsize=159640kBtime=00:00:03.00bitrate=435924.5kbits/sdup=15drop=0video:159639kBaudio:0kBsubtitle:0kBotherstreams:0kBglobalheaders:0kBmuxingoverhead:0.000662%alexandrov@ThinkPad-Edge:~/Documents/training20150109/edit$ffmpeg-pattern_typeglob-i'angle/*.png'-r30-pix_fmtrgba-vcodecpngz.mov>>log.txtffmpegversion2.5.gitCopyright(c)2000-2015theFFmpegdevelopersbuiltonJan11201519:12:34withgcc4.8(Ubuntu4.8.2-19ubuntu1)configuration:--prefix=/home/alexandrov/ffmpeg_build--extra-cflags=-I/home/alexandrov/ffmpeg_build/include--extra-ldflags=-L/home/alexandrov/ffmpeg_build/lib--bindir=/home/alexandrov/bin--enable-gpl--enable-libass--enable-libfdk-aac--enable-libfreetype--enable-libmp3lame--enable-libopus--enable-libtheora--enable-libvorbis--enable-libvpx--enable-libx264--enable-nonfree--enable-x11grablibavutil54.16.100/54.16.100libavcodec56.20.100/56.20.100libavformat56.18.100/56.18.100libavdevice56.3.100/56.3.100libavfilter5.7.100/5.7.100libswscale3.1.101/3.1.101libswresample1.1.100/1.1.100libpostproc53.3.100/53.3.100Input#0,image2,from'angle/*.png':Duration:00:00:03.24,start:0.000000,bitrate:N/AStream#0:0:Video:png,rgba64be,1280x720[SAR72:72DAR16:9],25fps,25tbr,25tbn,25tbcFile'z.mov'alreadyexists.Overwrite?[y/N]yOutput#0,mov,to'z.mov':Metadata:encoder:Lavf56.18.100Stream#0:0:Video:png(png/0x20676E70),rgba,1280x720[SAR1:1DAR16:9],q=2-31,200kb/s,30fps,15360tbn,30tbcMetadata:encoder:Lavc56.20.100pngStreammapping:Stream#0:0->#0:0(png(native)->png(native))Press[q]tostop,[?]forhelpframe=6fps=0.0q=0.0size=363kBtime=00:00:00.13bitrate=22310.frame=12fps=11q=0.0size=1105kBtime=00:00:00.33bitrate=27156.frame=18fps=11q=0.0size=1803kBtime=00:00:00.53bitrate=27698.frame=24fps=10q=0.0size=2576kBtime=00:00:00.73bitrate=28777.frame=30fps=10q=0.0size=3282kBtime=00:00:00.93bitrate=28809.frame=36fps=10q=0.0size=4100kBtime=00:00:01.13bitrate=29634.frame=42fps=10q=0.0size=4829kBtime=00:00:01.33bitrate=29672.frame=48fps=10q=0.0size=5586kBtime=00:00:01.53bitrate=29844.frame=54fps=10q=0.0size=6441kBtime=00:00:01.73bitrate=30443.frame=59fps=10q=0.0size=7154kBtime=00:00:01.90bitrate=30846.frame=65fps=10q=0.0size=8010kBtime=00:00:02.10bitrate=31244.frame=69fps=10q=0.0size=8580kBtime=00:00:02.23bitrate=31471.frame=75fps=10q=0.0size=9435kBtime=00:00:02.43bitrate=31763.frame=81fps=10q=0.0size=10290kBtime=00:00:02.63bitrate=32012.frame=87fps=10q=0.0size=11146kBtime=00:00:02.83bitrate=32225.frame=93fps=10q=0.0size=12001kBtime=00:00:03.03bitrate=32410.frame=97fps=10q=0.0Lsize=12857kBtime=00:00:03.23bitrate=32575.6kbits/sdup=16drop=0video:12856kBaudio:0kBsubtitle:0kBotherstreams:0kBglobalheaders:0kBmuxingoverhead:0.009313%alexandrov@ThinkPad-Edge:~/Documents/training20150109/edit$ffmpeg-iframe.mov-iz.mov-filter_complex"[0:0][1:0]overlay[out]" -shortest -map [out] -pix_fmt rgba -vcodec png test.mov >> log.txt
ffmpeg version 2.5.git Copyright (c) 2000-2015 the FFmpeg developers
  built on Jan 11 2015 19:12:34 with gcc 4.8 (Ubuntu 4.8.2-19ubuntu1)
  configuration: --prefix=/home/alexandrov/ffmpeg_build --extra-cflags=-I/home/alexandrov/ffmpeg_build/include --extra-ldflags=-L/home/alexandrov/ffmpeg_build/lib --bindir=/home/alexandrov/bin --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-nonfree --enable-x11grab
  libavutil      54. 16.100 / 54. 16.100
  libavcodec     56. 20.100 / 56. 20.100
  libavformat    56. 18.100 / 56. 18.100
  libavdevice    56.  3.100 / 56.  3.100
  libavfilter     5.  7.100 /  5.  7.100
  libswscale      3.  1.101 /  3.  1.101
  libswresample   1.  1.100 /  1.  1.100
  libpostproc    53.  3.100 / 53.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'frame.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf56.18.100
  Duration: 00:00:03.00, start: 0.000000, bitrate: 435924 kb/s
    Stream #0:0(eng): Video: png (png  / 0x20676E70), rgba, 1280x720 [SAR 1:1 DAR 16:9], 435921 kb/s, 30 fps, 30 tbr, 15360 tbn, 15360 tbc (default)
    Metadata:
      handler_name    : DataHandler
      encoder         : Lavc56.20.100 png
Input #1, mov,mp4,m4a,3gp,3g2,mj2, from 'z.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf56.18.100
  Duration: 00:00:03.23, start: 0.000000, bitrate: 32568 kb/s
    Stream #1:0(eng): Video: png (png  / 0x20676E70), rgba, 1280x720 [SAR 1:1 DAR 16:9], 32572 kb/s, 30 fps, 30 tbr, 15360 tbn, 15360 tbc (default)
    Metadata:
      handler_name    : DataHandler
      encoder         : Lavc56.20.100 png
Output #0, mov, to 'test.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf56.18.100
    Stream #0:0: Video: png (png  / 0x20676E70), rgba, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 30 fps, 15360 tbn, 30 tbc (default)
    Metadata:
      encoder         : Lavc56.20.100 png
Stream mapping:
  Stream #0:0 (png) -> overlay:main
  Stream #1:0 (png) -> overlay:overlay
  overlay -> Stream #0:0 (png)
Press [q] to stop, [?] for help
frame=    3 fps=0.0 q=0.0 size=    1717kB time=00:00:00.03 bitrate=421888
frame=    5 fps=3.8 q=0.0 size=    5212kB time=00:00:00.10 bitrate=427001
frame=    7 fps=3.7 q=0.0 size=    8720kB time=00:00:00.16 bitrate=428613
frame=    8 fps=3.3 q=0.0 size=   10475kB time=00:00:00.20 bitrate=429061
frame=   11 fps=3.7 q=0.0 size=   15746kB time=00:00:00.30 bitrate=429972
frame=   13 fps=3.7 q=0.0 size=   19264kB time=00:00:00.36 bitrate=430381
frame=   13 fps=3.2 q=0.0 size=   19264kB time=00:00:00.36 bitrate=430381
frame=   15 fps=3.1 q=0.0 size=   22781kB time=00:00:00.43 bitrate=430672
frame=   17 fps=3.1 q=0.0 size=   26297kB time=00:00:00.50 bitrate=430857
frame=   19 fps=3.0 q=0.0 size=   29814kB time=00:00:00.56 bitrate=431000
frame=   21 fps=3.1 q=0.0 size=   33331kB time=00:00:00.63 bitrate=431122
frame=   23 fps=3.1 q=0.0 size=   36848kB time=00:00:00.70 bitrate=431228
frame=   25 fps=3.1 q=0.0 size=   40367kB time=00:00:00.76 bitrate=431328
frame=   27 fps=3.2 q=0.0 size=   43884kB time=00:00:00.83 bitrate=431399
frame=   29 fps=3.2 q=0.0 size=   47402kB time=00:00:00.90 bitrate=431466
frame=   31 fps=3.2 q=0.0 size=   50923kB time=00:00:00.96 bitrate=431548
frame=   33 fps=3.2 q=0.0 size=   54449kB time=00:00:01.03 bitrate=431655
frame=   36 fps=3.3 q=0.0 size=   59741kB time=00:00:01.13 bitrate=431822
frame=   38 fps=3.3 q=0.0 size=   63267kB time=00:00:01.20 bitrate=431903
frame=   40 fps=3.3 q=0.0 size=   66794kB time=00:00:01.26 bitrate=431981
frame=   43 fps=3.3 q=0.0 size=   72080kB time=00:00:01.36 bitrate=432055
frame=   46 fps=3.4 q=0.0 size=   77365kB time=00:00:01.46 bitrate=432117
frame=   48 fps=3.4 q=0.0 size=   80889kB time=00:00:01.53 bitrate=432159
frame=   50 fps=3.4 q=0.0 size=   84414kB time=00:00:01.60 bitrate=432197
frame=   53 fps=3.5 q=0.0 size=   89700kB time=00:00:01.70 bitrate=432248
frame=   55 fps=3.4 q=0.0 size=   93224kB time=00:00:01.76 bitrate=432279
frame=   57 fps=3.4 q=0.0 size=   96749kB time=00:00:01.83 bitrate=432308
frame=   58 fps=3.3 q=0.0 size=   98511kB time=00:00:01.86 bitrate=432322
frame=   60 fps=3.4 q=0.0 size=  102035kB time=00:00:01.93 bitrate=432348
frame=   63 fps=3.4 q=0.0 size=  107322kB time=00:00:02.03 bitrate=432383
frame=   64 fps=3.4 q=0.0 size=  109084kB time=00:00:02.06 bitrate=432394
frame=   66 fps=3.4 q=0.0 size=  112608kB time=00:00:02.13 bitrate=432415
frame=   69 fps=3.4 q=0.0 size=  117895kB time=00:00:02.23 bitrate=432445
frame=   71 fps=3.4 q=0.0 size=  121419kB time=00:00:02.30 bitrate=432463
frame=   73 fps=3.4 q=0.0 size=  124943kB time=00:00:02.36 bitrate=432480
frame=   75 fps=3.4 q=0.0 size=  128468kB time=00:00:02.43 bitrate=432496
frame=   77 fps=3.4 q=0.0 size=  131992kB time=00:00:02.50 bitrate=432511
frame=   79 fps=3.4 q=0.0 size=  135516kB time=00:00:02.56 bitrate=432526
frame=   81 fps=3.4 q=0.0 size=  139041kB time=00:00:02.63 bitrate=432540
frame=   83 fps=3.4 q=0.0 size=  142565kB time=00:00:02.70 bitrate=432553
frame=   85 fps=3.4 q=0.0 size=  146089kB time=00:00:02.76 bitrate=432565
frame=   87 fps=3.4 q=0.0 size=  149614kB time=00:00:02.83 bitrate=432577
frame=   89 fps=3.4 q=0.0 size=  153138kB time=00:00:02.90 bitrate=432588
frame=   91 fps=3.4 q=0.0 size=  156663kB time=00:00:02.96 bitrate=432599
frame=   91 fps=3.3 q=0.0 Lsize=  160188kB time=00:00:03.03 bitrate=432613.9kbits/s dup=0 drop=6    
video:160187kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.000884%
alexandrov@ThinkPad-Edge:~/Documents/training 20150109/edit$ 

    

Answer 1

O problema surge devido a 64bit png .
Use isso para resolver o problema no método 1 :

convert -size 1280x720 xc:transparent -background transparent \
    -channel RGBA -fill '#0FF8' \
    -draw 'polygon 200, 600, 200, 20, 600, 50, 600, 50' \
    -fill '#0008' -draw 'polygon 200, 660, 200, 40, 660, 70, 660, 70' \
    -fill '#fFF8' -draw 'polygon 200, 500, 200, 00, 500, 30, 500, 30' \
    -channel RGBA -depth 8 -blur '10x5' test.png

Aqui adicionei -depth 8 para restringir o png a 32 bits em vez de 64.

Se você verificar os pngs criados, o tamanho de test.png será de 318.8kb e o de cyan.png será de 17.3kb. Além disso, testando com o ffmpeg:

ffmpeg -i cyan.png
...
Input #0, png_pipe, from 'cyan.png':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: png, rgba, 1280x720 [SAR 72:72 DAR 16:9], 25 tbr, 25 tbn, 25 tbc

Mas para o png de sobreposição múltipla, é:

ffmpeg -i test.png
...
Input #0, png_pipe, from 'test.png':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: png, rgba64be, 1280x720 [SAR 72:72 DAR 16:9], 25 tbr, 25 tbn, 25 tbc

Você vê que os codecs são diferentes: um é rgba64be e o outro é rgba . O 64bit png pode ser um problema para o ffmpeg por enquanto.

Então seu fluxo se torna:

Etapa 1 - criar o png de sobreposição.

convert -size 1280x720 xc:transparent -background transparent \
    -channel RGBA -fill '#0FF8' \
    -draw 'polygon 200, 600, 200, 20, 600, 50, 600, 50' -fill '#0008' \
    -draw 'polygon 200, 660, 200, 40, 660, 70, 660, 70' -fill '#fFF8' \
    -draw 'polygon 200, 500, 200, 00, 500, 30, 500, 30' -channel RGBA \
    -depth 8 -blur '10x5' test.png

Etapa 2: crie a imagem de plano de fundo.

convert -size 1280x720 xc:yellow -background yellow -channel RGBA gnd.png

Etapa 3: crie o filme de sobreposição.

ffmpeg -loop 1 -i test.png -t 1 -pix_fmt argb -vcodec qtrle z.mov

Etapa 4: crie o filme de fundo.

ffmpeg -loop 1 -i gnd.png -t 1 -pix_fmt argb -vcodec qtrle gnd.mov

Etapa 5 - sobreponha o filme de sobreposição.

ffmpeg -vcodec qtrle -i gnd.mov -vcodec qtrle -i z.mov \
    -filter_complex "[0:0][1:0]overlay=format=rgb[out]" -shortest \
    -map [out] -vcodec qtrle test.mov

O filme deve estar bem agora.

No entanto, eu também sugiro que seja possível pular a criação de um filme de sobreposição. Apenas use:

ffmpeg -i gnd.mov -i test.png \
    -filter_complex "[0:0][1:0]overlay=format=rgb[out]" \
    -map [out] -vcodec qtrle test.mov

    

Answer 2

Parece que o ffmpeg não consegue lidar muito bem com sobreposições de primitivas semi-transparentes desfocadas de cores diferentes.

Solução alternativa:

• Crie vários vídeos. Um para cada cor que você precisa.
• Sobreponha-os sequencialmente em segundo plano em vídeo.

Para ilustrar isso, vamos criar uma imagem com 3 triângulos sobrepostos de cores diferentes. E uma imagem com fundo vermelho sólido.

convert -size 1280x720 xc:transparent -background transparent -channel RGBA -fill '#0FF8' -draw 'polygon 200, 600, 200, 20, 600, 50, 600, 50' -fill '#0008' -draw 'polygon 200, 660, 200, 40, 660, 70, 660, 70' -fill '#fFF8' -draw 'polygon 200, 500, 200, 00, 500, 30, 500, 30' -channel RGBA -blur '10x5' test.png
convert -size 1280x720 xc:yellow -background yellow -channel RGBA gnd.png

Agora criamos dois filmes de 1 segundo de cada um e os sobrepomos

ffmpeg -loop 1 -i test.png -t 1 -pix_fmt argb -vcodec qtrle z.mov
ffmpeg -loop 1 -i gnd.png -t 1 -pix_fmt argb -vcodec qtrle gnd.mov
ffmpeg -vcodec qtrle -i gnd.mov -vcodec qtrle -i z.mov -filter_complex "[0:0][1:0]overlay[out]" -shortest -map [out] -vcodec qtrle -pix_fmt argb test.mov

aqui está um quadro de sobreposição de baixa qualidade que temos:

Permitecriartrêsvídeosdiferentesparacadacor

convert-size1280x720xc:transparent-backgroundtransparent-channelRGBA-fill'#0FF8'-draw'polygon200,600,200,20,600,50,600,50'-channelRGBA-blur'10x5'cyan.pngconvert-size1280x720xc:transparent-backgroundtransparent-channelRGBA-fill'#0008'-draw'polygon200,660,200,40,660,70,660,70'-channelRGBA-blur'10x5'black.pngconvert-size1280x720xc:transparent-backgroundtransparent-channelRGBA-fill'#fFF8'-draw'polygon200,500,200,00,500,30,500,30'-channelRGBA-blur'10x5'white.pngconvert-size1280x720xc:redgnd.pngffmpeg-loop1-icyan.png-t1-pix_fmtargb-vcodecqtrlecyan.movffmpeg-loop1-iwhite.png-t1-pix_fmtargb-vcodecqtrlewhite.movffmpeg-loop1-iblack.png-t1-pix_fmtargb-vcodecqtrleblack.movffmpeg-loop1-ignd.png-t1-pix_fmtargb-vcodecqtrlegnd.mov

Esobreponha-osumaumsobreonossovídeodefundo

ffmpeg-ignd.mov-icyan.mov-iwhite.mov-iblack.mov-filter_complex"[0:0][2:0]overlay[tmp1]; [tmp1][3:0]overlay[tmp2]; [tmp2][1:0]overlay[out]" -shortest -map [out] -vcodec qtrle -pix_fmt argb good.mov

O que temos é uma foto bem melhor.

Nota importante é que a sequência de sobreposição é muito importante aqui!