Versões mais antigas do FreeBSD / pf eram conhecidas por terem problemas com o driver Xen e o TCP Segmentation Offloading.
Tente: sysctl net.inet.tcp.tso = 0
Eu tenho um novo setup fbsd 9.1 com pf, que por si só não sofre lentidão ao baixar um iso debian do meu repositório debian local (ftp.se.debian.org). Qualquer máquina por trás dele, pela qual as rotas de firewall freebsd e nats, experimentam cerca de 10 a 12 segundos antes de recuperar qualquer dado depois que o handshake tcp inicial é formado.
Velocidade após o atraso inicial é bom, cerca de 10-12 MB / s constantemente. Eu suspeito que estou fazendo algo errado, veja regras e tcpdump abaixo. Pode valer a pena acrescentar que o freebsd está sendo executado em uma VM XenServer (6.0), com os dispositivos xenhvm compilados em um kernel personalizado.
# pf.conf
wanif = "xn0"
dmzif = "xn2"
dmznet = "10.64.1.0/24"
scrub on $wanif reassemble tcp no-df random-id
nat on $wanif from $dmzif:network to any -> ($wanif)
block log
block in all
pass quick on lo0 all
pass out all keep state
pass in quick on $dmzif inet from $dmznet to ! $intnets keep state
pass out on $wanif proto tcp all modulate state flags S/SA
pass out on $wanif all keep state
# tcpdump -ni xn0
12:09:04.389635 IP 71.72.73.74.51953 > 194.71.11.69.80: Flags [S], seq 2077316563, win 5840, options [mss 1460,sackOK,TS val 478788359 ecr 0,nop,wscale 4], length 0
12:09:04.401362 IP 194.71.11.69.80 > 71.72.73.74.51953: Flags [S.], seq 93082952, ack 2077316564, win 5792, options [mss 1460,sackOK,TS val 2817201177 ecr 478788359,nop,wscale 7], length 0
12:09:04.401851 IP 71.72.73.74.51953 > 194.71.11.69.80: Flags [.], ack 1, win 365, options [nop,nop,TS val 478788362 ecr 2817201177], length 0
12:09:04.402126 IP 71.72.73.74.51953 > 194.71.11.69.80: Flags [P.], seq 1:195, ack 1, win 365, options [nop,nop,TS val 478788362 ecr 2817201177], length 194
12:09:04.611851 IP 71.72.73.74.51953 > 194.71.11.69.80: Flags [P.], seq 1:195, ack 1, win 365, options [nop,nop,TS val 478788415 ecr 2817201177], length 194
12:09:05.035855 IP 71.72.73.74.51953 > 194.71.11.69.80: Flags [P.], seq 1:195, ack 1, win 365, options [nop,nop,TS val 478788521 ecr 2817201177], length 194
12:09:05.884041 IP 71.72.73.74.51953 > 194.71.11.69.80: Flags [P.], seq 1:195, ack 1, win 365, options [nop,nop,TS val 478788733 ecr 2817201177], length 194
12:09:07.580009 IP 71.72.73.74.51953 > 194.71.11.69.80: Flags [P.], seq 1:195, ack 1, win 365, options [nop,nop,TS val 478789157 ecr 2817201177], length 194
12:09:07.944140 IP 194.71.11.69.80 > 71.72.73.74.51953: Flags [S.], seq 93082952, ack 2077316564, win 5792, options [mss 1460,sackOK,TS val 2817204720 ecr 478788362,nop,wscale 7], length 0
12:09:07.944908 IP 71.72.73.74.51953 > 194.71.11.69.80: Flags [.], ack 1, win 365, options [nop,nop,TS val 478789248 ecr 2817204720,nop,nop,sack 1 {0:1}], length 0
12:09:10.972026 IP 71.72.73.74.51953 > 194.71.11.69.80: Flags [P.], seq 1:195, ack 1, win 365, options [nop,nop,TS val 478790005 ecr 2817204720], length 194
12:09:17.756060 IP 71.72.73.74.51953 > 194.71.11.69.80: Flags [P.], seq 1:195, ack 1, win 365, options [nop,nop,TS val 478791701 ecr 2817204720], length 194
12:09:17.767744 IP 194.71.11.69.80 > 71.72.73.74.51953: Flags [.], ack 195, win 54, options [nop,nop,TS val 2817214544 ecr 478791701], length 0
12:09:17.895263 IP 194.71.11.69.80 > 71.72.73.74.51953: Flags [.], seq 1:1449, ack 195, win 54, options [nop,nop,TS val 2817214672 ecr 478791701], length 1448
12:09:17.895326 IP 194.71.11.69.80 > 71.72.73.74.51953: Flags [.], seq 1449:2897, ack 195, win 54, options [nop,nop,TS val 2817214672 ecr 478791701], length 1448
# tcpdump -ni xn2 port 80
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on xn2, link-type EN10MB (Ethernet), capture size 65535 bytes
12:03:18.248115 IP 10.64.1.2.34774 > 194.71.11.69.80: Flags [P.], seq 827084948:827085121, ack 856345816, win 365, options [nop,nop,TS val 4294916027 ecr 3651988161], length 173
12:03:18.269060 IP 194.71.11.69.80 > 10.64.1.2.34774: Flags [.], ack 173, win 122, options [nop,nop,TS val 3651991631 ecr 4294916027], length 0
12:03:18.269309 IP 194.71.11.69.80 > 10.64.1.2.34774: Flags [.], seq 1:1449, ack 173, win 122, options [nop,nop,TS val 3651991631 ecr 4294916027], length 1448
12:03:18.269364 IP 194.71.11.69.80 > 10.64.1.2.34774: Flags [.], seq 1449:2897, ack 173, win 122, options [nop,nop,TS val 3651991631 ecr 4294916027], length 1448
12:03:18.269397 IP 194.71.11.69.80 > 10.64.1.2.34774: Flags [.], seq 2897:4345, ack 173, win 122, options [nop,nop,TS val 3651991631 ecr 4294916027], length 1448
12:03:18.269427 IP 194.71.11.69.80 > 10.64.1.2.34774: Flags [P.], seq 4345:5793, ack 173, win 122, options [nop,nop,TS val 3651991631 ecr 4294916027], length 1448
12:03:18.269744 IP 10.64.1.2.34774 > 194.71.11.69.80: Flags [.], ack 1449, win 546, options [nop,nop,TS val 4294916032 ecr 3651991631], length 0
12:03:18.269797 IP 10.64.1.2.34774 > 194.71.11.69.80: Flags [.], ack 2897, win 727, options [nop,nop,TS val 4294916032 ecr 3651991631], length 0
12:03:18.269818 IP 10.64.1.2.34774 > 194.71.11.69.80: Flags [.], ack 4345, win 908, options [nop,nop,TS val 4294916032 ecr 3651991631], length 0
12:03:18.269837 IP 10.64.1.2.34774 > 194.71.11.69.80: Flags [.], ack 5793, win 1089, options [nop,nop,TS val 4294916032 ecr 3651991631], length 0
12:03:18.269861 IP 10.64.1.2.53888 > 130.239.18.173.80: Flags [F.], seq 1457436047, ack 1959872378, win 452, options [nop,nop,TS val 4294916032 ecr 1986976335], length 0
12:03:18.290194 IP 130.239.18.173.80 > 10.64.1.2.53888: Flags [.], ack 1, win 122, options [nop,nop,TS val 1986977333 ecr 4294916032], length 0
12:03:18.290227 IP 194.71.11.69.80 > 10.64.1.2.34774: Flags [.], seq 5793:7241, ack 173, win 122, options [nop,nop,TS val 3651991637 ecr 4294916032], length 1448
12:03:18.290247 IP 194.71.11.69.80 > 10.64.1.2.34774: Flags [.], seq 7241:8689, ack 173, win 122, options [nop,nop,TS val 3651991637 ecr 4294916032], length 1448
12:03:18.290266 IP 194.71.11.69.80 > 10.64.1.2.34774: Flags [.], seq 8689:10137, ack 173, win 122, options [nop,nop,TS val 3651991637 ecr 4294916032], length 1448
12:03:18.290292 IP 194.71.11.69.80 > 10.64.1.2.34774: Flags [P.], seq 10137:11585, ack 173, win 122, options [nop,nop,TS val 3651991637 ecr 4294916032], length 1448
12:03:18.290312 IP 194.71.11.69.80 > 10.64.1.2.34774: Flags [.], seq 11585:13033, ack 173, win 122, options [nop,nop,TS val 3651991637 ecr 4294916032], length 1448
12:03:18.290332 IP 194.71.11.69.80 > 10.64.1.2.34774: Flags [.], seq 13033:14481, ack 173, win 122, options [nop,nop,TS val 3651991637 ecr 4294916032], length 1448
12:03:18.290357 IP 194.71.11.69.80 > 10.64.1.2.34774: Flags [.], seq 14481:15929, ack 173, win 122, options [nop,nop,TS val 3651991637 ecr 4294916032], length 1448
12:03:18.290382 IP 194.71.11.69.80 > 10.64.1.2.34774: Flags [P.], seq 15929:17377, ack 173, win 122, options [nop,nop,TS val 3651991637 ecr 4294916032], length 1448
12:03:18.290420 IP 194.71.11.69.80 > 10.64.1.2.34774: Flags [.], seq 17377:18825, ack 173, win 122, options [nop,nop,TS val 3651991637 ecr 4294916032], length 1448
12:03:18.290444 IP 194.71.11.69.80 > 10.64.1.2.34774: Flags [.], seq 18825:20273, ack 173, win 122, options [nop,nop,TS val 3651991637 ecr 4294916032], length 1448
12:03:18.290469 IP 194.71.11.69.80 > 10.64.1.2.34774: Flags [.], seq 20273:21721, ack 173, win 122, options [nop,nop,TS val 3651991637 ecr 4294916032], length 1448
12:03:18.290553 IP 10.64.1.2.34774 > 194.71.11.69.80: Flags [.], ack 7241, win 1270, options [nop,nop,TS val 4294916037 ecr 3651991637], length 0
12:03:18.290599 IP 10.64.1.2.34774 > 194.71.11.69.80: Flags [.], ack 8689, win 1451, options [nop,nop,TS val 4294916037 ecr 3651991637], length 0
12:03:18.290621 IP 10.64.1.2.34774 > 194.71.11.69.80: Flags [.], ack 10137, win 1632, options [nop,nop,TS val 4294916037 ecr 3651991637], length 0
12:03:18.290640 IP 10.64.1.2.34774 > 194.71.11.69.80: Flags [.], ack 11585, win 1813, options [nop,nop,TS val 4294916037 ecr 3651991637], length 0
12:03:18.290665 IP 10.64.1.2.34774 > 194.71.11.69.80: Flags [.], ack 13033, win 1994, options [nop,nop,TS val 4294916037 ecr 3651991637], length 0
12:03:18.290684 IP 10.64.1.2.34774 > 194.71.11.69.80: Flags [.], ack 14481, win 2175, options [nop,nop,TS val 4294916037 ecr 3651991637], length 0
12:03:18.290705 IP 10.64.1.2.34774 > 194.71.11.69.80: Flags [.], ack 15929, win 2356, options [nop,nop,TS val 4294916037 ecr 3651991637], length 0
12:03:18.290729 IP 10.64.1.2.34774 > 194.71.11.69.80: Flags [.], ack 17377, win 2537, options [nop,nop,TS val 4294916037 ecr 3651991637], length 0
12:03:18.290755 IP 10.64.1.2.34774 > 194.71.11.69.80: Flags [.], ack 18825, win 2718, options [nop,nop,TS val 4294916037 ecr 3651991637], length 0
12:03:18.290774 IP 10.64.1.2.34774 > 194.71.11.69.80: Flags [.], ack 20273, win 2899, options [nop,nop,TS val 4294916037 ecr 3651991637], length 0
12:03:18.290798 IP 10.64.1.2.34774 > 194.71.11.69.80: Flags [.], ack 21721, win 3080, options [nop,nop,TS val 4294916037 ecr 3651991637], length 0
12:03:18.311156 IP 194.71.11.69.80 > 10.64.1.2.34774: Flags [.], seq 21721:23169, ack 173, win 122, options [nop,nop,TS val 3651991642 ecr 4294916037], length 1448
12:03:18.311190 IP 194.71.11.69.80 > 10.64.1.2.34774: Flags [.], seq 23169:24617, ack 173, win 122, options [nop,nop,TS val 3651991642 ecr 4294916037], length 1448
12:03:18.311208 IP 194.71.11.69.80 > 10.64.1.2.34774: Flags [.], seq 24617:26065, ack 173, win 122, options [nop,nop,TS val 3651991642 ecr 4294916037], length 1448
12:03:18.311228 IP 194.71.11.69.80 > 10.64.1.2.34774: Flags [.], seq 26065:27513, ack 173, win 122, options [nop,nop,TS val 3651991642 ecr 4294916037], length 1448
12:03:18.311247 IP 194.71.11.69.80 > 10.64.1.2.34774: Flags [.], seq 27513:28961, ack 173, win 122, options [nop,nop,TS val 3651991642 ecr 4294916037], length 1448
12:03:18.311266 IP 194.71.11.69.80 > 10.64.1.2.34774: Flags [P.], seq 28961:30409, ack 173, win 122, options [nop,nop,TS val 3651991642 ecr 4294916037], length 1448
Versões mais antigas do FreeBSD / pf eram conhecidas por terem problemas com o driver Xen e o TCP Segmentation Offloading.
Tente: sysctl net.inet.tcp.tso = 0