graças a @pfnuesel eu encontrei a resposta nos logs do i3:
[..] window 0x014013e1 wants to be at -1670x-1466 with 6220x4644
e conseguiu corrigi-lo com:
floating_minimum_size 500 x 300
floating_maximum_size 2000 x 1500
for_window [class="(?i)firefox" instance="^(?!Navigator$)"] floating enable