Eu configurei um haproxy (1.6.3) no Ubuntu 16.04 para balancear a carga de dois servidores web. Dos meus testes anteriores, os servidores da web podem manipular mais de 20 mil solicitações / s. Os servidores da web foram testados contra wrk2 , e eu verifiquei o número de solicitações no log. No entanto, com haproxy na frente dos servidores da web, parece que a solicitação por segundo é limitada a cerca de 6k request / s. Há algo de errado na configuração do haproxy?
haproxy.cnf
global
log /dev/log local0
log /dev/log local1 notice
chroot /var/lib/haproxy
stats socket /run/haproxy/admin.sock mode 660 level admin
stats timeout 30s
maxconn 102400
user haproxy
group haproxy
daemon
# Default SSL material locations
ca-base /etc/ssl/certs
crt-base /etc/ssl/private
# Default ciphers to use on SSL-enabled listening sockets.
# For more information, see ciphers(1SSL). This list is from:
# https://hynek.me/articles/hardening-your-web-servers-ssl-ciphers/
ssl-default-bind-ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS
ssl-default-bind-options no-sslv3
defaults
log global
mode http
option httplog
option dontlognull
# https://serverfault.com/questions/504308/by-what-criteria-do-you-tune-timeouts-in-ha-proxy-config
timeout connect 5000
timeout check 5000
timeout client 30000
timeout server 30000
timeout tunnel 3600s
errorfile 400 /etc/haproxy/errors/400.http
errorfile 403 /etc/haproxy/errors/403.http
errorfile 408 /etc/haproxy/errors/408.http
errorfile 500 /etc/haproxy/errors/500.http
errorfile 502 /etc/haproxy/errors/502.http
errorfile 503 /etc/haproxy/errors/503.http
errorfile 504 /etc/haproxy/errors/504.http
listen web-test
maxconn 40000 # the default is 2000
mode http
bind *:80
balance roundrobin
option forwardfor
option http-keep-alive # connections will no longer be closed after each request
server test1 SERVER1:80 check maxconn 20000
server test2 SERVER2:80 check maxconn 20000
Se runnign wrk com 3 instâncias, recebo aproximadamente o mesmo resultado:
./wrk -t4 -c100 -d30s -R4000 http://HAPROXY/
Running 30s test @ http://HAPROXY/
4 threads and 100 connections
Thread calibration: mean lat.: 1577.987ms, rate sampling interval: 7139ms
Thread calibration: mean lat.: 1583.182ms, rate sampling interval: 7180ms
Thread calibration: mean lat.: 1587.795ms, rate sampling interval: 7167ms
Thread calibration: mean lat.: 1583.128ms, rate sampling interval: 7147ms
Thread Stats Avg Stdev Max +/- Stdev
Latency 8.98s 2.67s 13.93s 58.43%
Req/Sec 516.75 11.28 529.00 87.50%
64916 requests in 30.00s, 51.69MB read
Requests/sec: 2163.75 # Requests/sec decrease slightly
Transfer/sec: 1.72MB
Stats de haproxy:
Seestiverexecutandoowrkcom1instânciaparaumdosservidoreswebsemohaproxy:
./wrk-t4-c100-d30s-R4000http://SERVER1Running30stest@http://SERVER14threadsand100connectionsThreadcalibration:meanlat.:1.282ms,ratesamplinginterval:10msThreadcalibration:meanlat.:1.363ms,ratesamplinginterval:10msThreadcalibration:meanlat.:1.380ms,ratesamplinginterval:10msThreadcalibration:meanlat.:1.351ms,ratesamplinginterval:10msThreadStatsAvgStdevMax+/-StdevLatency1.41ms0.97ms22.42ms96.48%Req/Sec1.05k174.272.89k86.01%119809requestsin30.00s,98.15MBreadRequests/sec:3993.36#Requests/secisabout4kTransfer/sec:3.27MB
haproxia-vv VersãoHA-Proxy1.6.32015/12/25 Copyright2000-2015WillyTarreau
Buildoptions:TARGET=linux2628CPU=genericCC=gccCFLAGS=-g-O2-fstack-protector-strong-Wformat-Werror=format-security-Wdate-time-D_FORTIFY_SOURCE=2OPTIONS=USE_ZLIB=1USE_REGPARM=1USE_OPENSSL=1USE_LUA=1USE_PCRE=1Defaultsettings:maxconn=2000,bufsize=16384,maxrewrite=1024,maxpollevents=200Encryptedpasswordsupportviacrypt(3):yesBuiltwithzlibversion:1.2.8Compressionalgorithmssupported:identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with OpenSSL version : OpenSSL 1.0.2g-fips 1 Mar 2016
Running on OpenSSL version : OpenSSL 1.0.2g 1 Mar 2016
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports prefer-server-ciphers : yes
Built with PCRE version : 8.38 2015-11-23
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built with Lua version : Lua 5.3.1
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
HA-Proxy version 1.6.3 2015/12/25
Copyright 2000-2015 Willy Tarreau <[email protected]>
Build options :
TARGET = linux2628
CPU = generic
CC = gcc
CFLAGS = -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2
OPTIONS = USE_ZLIB=1 USE_REGPARM=1 USE_OPENSSL=1 USE_LUA=1 USE_PCRE=1
Default settings :
maxconn = 2000, bufsize = 16384, maxrewrite = 1024, maxpollevents = 200
Encrypted password support via crypt(3): yes
Built with zlib version : 1.2.8
Compression algorithms supported : identity("identity"), deflate("deflate"), raw-deflate("deflate"), gzip("gzip")
Built with OpenSSL version : OpenSSL 1.0.2g-fips 1 Mar 2016
Running on OpenSSL version : OpenSSL 1.0.2g 1 Mar 2016
OpenSSL library supports TLS extensions : yes
OpenSSL library supports SNI : yes
OpenSSL library supports prefer-server-ciphers : yes
Built with PCRE version : 8.38 2015-11-23
PCRE library supports JIT : no (USE_PCRE_JIT not set)
Built with Lua version : Lua 5.3.1
Built with transparent proxy support using: IP_TRANSPARENT IPV6_TRANSPARENT IP_FREEBIND
Available polling systems :
epoll : pref=300, test result OK
poll : pref=200, test result OK
select : pref=150, test result OK
Total: 3 (3 usable), will use epoll.
Eu sei que ab não é uma maneira muito precisa de testar isso, mas eu pensei que o haproxy deveria dar um resultado melhor do que um único nó. No entanto, os resultados mostram o contrário.
teste ab HAPROXY
ab -n 10000 -c 10 http://HAPROXY/
Requests per second: 4276.18 [#/sec] (mean)
teste AB SERVER1
ab -n 10000 -c 10 http://SERVER1/
Requests per second: 9392.66 [#/sec] (mean)
teste AB SERVER2
ab -n 10000 -c 10 http://SERVER2/
Requests per second: 8513.28 [#/sec] (mean)
A VM é single core, portanto, não há necessidade de usar nbproc . Além disso, monitorei a CPU, o uso da memória, todas as VMs usam menos de 30% de CPU e 20% de memória. Deve haver algo errado sobre as configurações do haproxy ou as configurações do meu sistema.
Agora tenho quase o mesmo desempenho do haproxy e do servidor único, e o problema é que há um maxconn 2000 padrão na seção listen que perdi. No entanto, espero que o desempenho seja melhor ao ter mais servidores de back-end, e ainda não consigo alcançar isso.
Com as mesmas configurações, agora atualizo para o haproxy 1.8.3, mas isso não faz muita diferença.