Na sua captura, os dois servidores estão configurando "Não fragmentar bits". Isso significa que ambas as extremidades estão tentando fazer o Path MTU Discovery.
Parece que há um firewall que bloqueia ICMP Fragmentation Needed
de seu servidor Linux em relação ao servidor remoto. Como solução alternativa, habilite a fixação MSS com:
iptables -A OUTPUT -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
Você também pode tentar desabilitar o P MTU Discovery no Linux:
echo 1 |sudo tee /proc/sys/net/ipv4/ip_no_pmtu_disc
Na página iptables
man:
TCPMSS This target allows to alter the MSS value of TCP SYN packets, to control the maximum size for that connection (usually limiting it to your outgoing interface's MTU minus 40 for IPv4 or 60 for IPv6, respectively). Of course, it can only be used in conjunction with -p tcp.
This target is used to overcome criminally braindead ISPs or servers which block "ICMP Fragmentation Needed" or "ICMPv6 Packet Too
Big" packets. The symptoms of this problem are that everything works fine from your Linux firewall/router, but machines behind it
can never exchange large packets:
1) Web browsers connect, then hang with no data received.
2) Small mail works fine, but large emails hang.
3) ssh works fine, but scp hangs after initial handshaking.
Workaround: activate this option and add a rule to your firewall configuration like:
iptables -t mangle -A FORWARD -p tcp --tcp-flags SYN,RST SYN
-j TCPMSS --clamp-mss-to-pmtu
--set-mss value
Explicitly sets MSS option to specified value. If the MSS of the packet is already lower than value, it will not be
increased (from Linux 2.6.25 onwards) to avoid more problems with hosts relying on a proper MSS.
--clamp-mss-to-pmtu
Automatically clamp MSS value to (path_MTU - 40 for IPv4; -60 for IPv6). This may not function as desired where asymmetric
routes with differing path MTU exist — the kernel uses the path MTU which it would use to send packets from itself to the
source and destination IP addresses. Prior to Linux 2.6.25, only the path MTU to the destination IP address was considered
by this option; subsequent kernels also consider the path MTU to the source IP address.
These options are mutually exclusive.
Veja: link
Editar: Depois de dar uma olhada mais de perto nas capturas, descobri que há um firewall quebrado ao longo do caminho que está filtrando todos os pacotes IP que usam a opção TCP Timestamp. Basta executar na caixa do Linux: echo 0 | sudo tee /proc/sys/net/ipv4/tcp_timestamps