Estou usando o comando ffmpeg a seguir para mesclar duas entradas MKV com o parâmetro overlay.
O resultado deve ser uma saída com input1 em cima de input2.
No processo, a saída deve ser webm.
Ambas as entradas são do mesmo comprimento (com uma margem de um segundo).
Uma ilustração;
-----------------
| |
| |
| input1.mkv |
| |
|---------------|
| |
| |
| input2.mkv |
| |
-----------------
Saída de comando e aparada:
ffmpeg -i input1.mkv -i input2.mkv -y -filter_complex \
[0:v]select=1, setpts=PTS-STARTPTS, scale=400:300, pad=400:600 [top]; \
[1:v]select=1, setpts=PTS-STARTPTS, scale=400:300 [bottom]; \
[top][bottom] overlay=0:300 [out]; \
[0:a:0][1:a:0] amerge=inputs=2 [a]; \
[a] asetpts=PTS-STARTPTS [a] \
-map [a] -c:v libvpx -crf 10 -b:v 360K -q:v 7 -c:a libvorbis -b:a 32k \
-map [out] output.webm
ffmpeg version 2.4.git Copyright (c) 2000-2014 the FFmpeg developers
built on Oct 30 2014 14:00:21 with gcc 4.6 (Ubuntu/Linaro 4.6.3-1ubuntu5)
configuration: --prefix=/home/bla/ffmpeg_build --extra-cflags=-I/home/bla/ffmpeg_build/include --extra-ldflags=-L/home/bla/ffmpeg_build/lib --bindir=/home/bla/bin --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-nonfree --enable-openssl
libavutil 54. 11.100 / 54. 11.100
libavcodec 56. 10.100 / 56. 10.100
libavformat 56. 11.100 / 56. 11.100
libavdevice 56. 2.100 / 56. 2.100
libavfilter 5. 2.100 / 5. 2.100
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
Guessed Channel Layout for Input Stream #0.1 : mono
Input #0, matroska,webm, from '/tmp/input1.mkv':
Metadata:
ENCODER : Lavf54.20.4
Duration: 00:02:50.45, start: 0.000000, bitrate: 174 kb/s
Stream #0:0: Video: vp8, yuv420p, 640x480, SAR 1:1 DAR 4:3, 30 fps, 30 tbr, 1k tbn, 1k tbc (default)
Stream #0:1: Audio: pcm_mulaw ([7][0][0][0] / 0x0007), 8000 Hz, 1 channels, s16, 64 kb/s (default)
Guessed Channel Layout for Input Stream #1.1 : mono
Input #1, matroska,webm, from '/tmp/input2.mkv':
Metadata:
ENCODER : Lavf54.20.4
Duration: 00:02:50.46, start: 0.013000, bitrate: 1901 kb/s
Stream #1:0: Video: vp8, yuv420p, 640x480, SAR 1:1 DAR 4:3, 30 fps, 30 tbr, 1k tbn, 1k tbc (default)
Stream #1:1: Audio: pcm_mulaw ([7][0][0][0] / 0x0007), 8000 Hz, 1 channels, s16, 64 kb/s (default)
[Parsed_amerge_8 @ 0x325ada0] No channel layout for input 1
[Parsed_amerge_8 @ 0x325ada0] Input channel layouts overlap: output layout will be determined by the number of distinct input channels
[libvpx @ 0x3268aa0] v1.3.0
Output #0, webm, to '/tmp/output.webm':
Metadata:
encoder : Lavf56.11.100
Stream #0:0: Audio: vorbis (libvorbis), 8000 Hz, stereo, fltp, 32 kb/s (default)
Metadata:
encoder : Lavc56.10.100 libvorbis
Stream #0:1: Video: vp8 (libvpx), yuv420p, 400x600 [SAR 1:1 DAR 2:3], q=-1--1, 360 kb/s, 30 fps, 1k tbn, 30 tbc (default)
Metadata:
encoder : Lavc56.10.100 libvpx
Stream mapping:
Stream #0:0 (vp8) -> select
Stream #0:1 (pcm_mulaw) -> amerge:in0
Stream #1:0 (vp8) -> select
Stream #1:1 (pcm_mulaw) -> amerge:in1
asetpts -> Stream #0:0 (libvorbis)
overlay -> Stream #0:1 (libvpx)
Press [q] to stop, [?] for help
[vp8 @ 0x322af20] Discarding interframe without a prior keyframe!
Error while decoding stream #0:0: Invalid data found when processing input
[vp8 @ 0x322af20] Discarding interframe without a prior keyframe!
Error while decoding stream #0:0: Invalid data found when processing input
frame= 316 fps= 17 q=0.0 size= 753kB time=00:00:13.53 bitrate= 456.0kbits/s dup=0 drop=146
[vp8 @ 0x322af20] Upscaling is not implemented. Update your FFmpeg version to the newest one from Git. If the problem still occurs, it means that your file has a feature which has not been implemented.
[vp8 @ 0x322af20] If you want to help, upload a sample of this file to ftp://upload.ffmpeg.org/incoming/ and contact the ffmpeg-devel mailing list. ([email protected])
Input stream #0:0 frame changed from size:320x240 fmt:yuv420p to size:384x288 fmt:yuv420p
Input stream #0:0 frame changed from size:384x288 fmt:yuv420p to size:320x240 fmt:yuv420p
Input stream #0:0 frame changed from size:320x240 fmt:yuv420p to size:384x288 fmt:yuv420p
Input stream #0:0 frame changed from size:384x288 fmt:yuv420p to size:512x384 fmt:yuv420p
Input stream #0:0 frame changed from size:512x384 fmt:yuv420p to size:640x480 fmt:yuv420p
[Parsed_amerge_8 @ 0x33462c0] No channel layout for input 1
[Parsed_amerge_8 @ 0x33462c0] Input channel layouts overlap: output layout will be determined by the number of distinct input channels
[libvorbis @ 0x3266fc0] Queue input is backward in time
[webm @ 0x3266200] Non-monotonous DTS in output stream 0:0; previous: 13880, current: 3912; changing to 13880. This may result in incorrect timestamps in the output file.
frame= 2730 fps= 21 q=0.0 size= 6030kB time=00:01:39.33 bitrate= 497.3kbits/s dup=0 drop=1036
Error while decoding stream #0:1: Cannot allocate memory
Last message repeated 65 times
frame= 2738 fps= 21 q=0.0 size= 6048kB time=00:01:39.66 bitrate= 497.1kbits/s dup=0 drop=1036
Error while decoding stream #0:1: Cannot allocate memory
Last message repeated 170 times
frame= 2784 fps= 21 q=0.0 size= 6230kB time=00:01:53.17 bitrate= 450.9kbits/s dup=0 drop=1403
Error while decoding stream #1:1: Cannot allocate memory
Last message repeated 133 times
[webm @ 0x3266200] Non-monotonous DTS in output stream 0:0; previous: 113164, current: 3896; changing to 113164. This may result in incorrect timestamps in the output file.
[webm @ 0x3266200] Non-monotonous DTS in output stream 0:0; previous: 113164, current: 3928; changing to 113164. This may result in incorrect timestamps in the output file.
[webm @ 0x3266200] Non-monotonous DTS in output stream 0:0; previous: 113164, current: 3960; changing to 113164. This may result in incorrect timestamps in the output file.
[webm @ 0x3266200] Non-monotonous DTS in output stream 0:0; previous: 113164, current: 3992; changing to 113164. This may result in incorrect timestamps in the output file.
frame= 2784 fps= 21 q=0.0 Lsize= 6295kB time=00:01:53.17 bitrate= 455.6kbits/s dup=0 drop=1456
video:5595kB audio:643kB subtitle:0kB other streams:0kB global headers:3kB muxing overhead: 0.898592%
Este comando faz o que deve fazer.
No entanto, os dois vídeos não estão totalmente sincronizados.
A entrada1 na parte superior é reproduzida decentemente enquanto a entrada2 na parte inferior tem quadros pretos, diminui ou acelera e faz com que o áudio e o vídeo fiquem fora de sincronia.
Para descartar a qualidade individual das entradas, trocamos a posição dos vídeos e o vídeo principal sempre é exibido com decência.
Como podemos corrigir isso?
- ATUALIZAÇÃO 1 -
A execução do FFMPEG através do módulo fluent-ffmpeg do Node.js concebeu todos os avisos e erros.
Eu executei o comando FFMPEG no console e a quantidade de saída é gigantesca.
Aqui está um pastebin não aparado com o log - > link
- ATUALIZAÇÃO 2-
Um possível lead: Os arquivos mkv de entrada são fluxos WebRTC sendo afundados. Por favor me corrija se eu estiver errado;
Em transmissões ao vivo, a qualidade muda de acordo com a conexão. Se isso significa que os quadros estão sendo enviados com tamanhos diferentes, isso poderia explicar o problema do ffmpeg reclamar sobre a alteração dos tamanhos dos quadros.
Então, para reformular a questão;
Como é possível combinar vídeos de entrada de 2 mkv (fluxo WebRTC de origem) sem eliminar quadros que mudam de tamanho?