Como configurar o upstart para a instância do OpenVPN que está usando o plugin down-root

1
  • Ubuntu 14.04
  • OpenVPN 2.3.2
  • Daemon de inicialização: upstart

arquivo de configuração do servidor:

topology net30
mode server
tls-server
push "route-gateway 192.168.0.1"
ifconfig 192.168.0.1 192.168.0.2
ifconfig-pool 192.168.0.5 192.168.0.9

port 1196
proto udp
dev tun
keepalive 10 120
comp-lzo
user nobody
group nogroup
persist-key
persist-tun
status /var/log/openvpn/devops.log 1
syslog vpn-devops
verb 15
writepid /var/run/openvpn/devops.pid
ca /etc/openvpn/devops/ca.crt
cert /etc/openvpn/devops/server.crt
key /etc/openvpn/devops/server.key
dh /etc/openvpn/dh.pem
ifconfig-pool-persist /var/lib/openvpn/devops.ipp
client-config-dir /etc/openvpn/devops/ccd
crl-verify /etc/openvpn/devops/crl.pem
client-to-client

Problema: sempre que reinicio o serviço, vejo isso no arquivo de log:

May 17 14:32:08 t-xhyve vpn-devops[21013]: TCP/UDP: Closing socket
May 17 14:32:08 t-xhyve vpn-devops[21013]: Closing TUN/TAP interface
May 17 14:32:08 t-xhyve vpn-devops[21013]: /sbin/ip addr del dev tun1 local 192.168.0.1 peer 192.168.0.2
May 17 14:32:08 t-xhyve vpn-devops[21013]: Linux ip addr del failed: external program exited with error status: 2

O motivo é que o OpenVPN está sendo executado como nobody user que não tem permissão para excluir a interface do túnel. Então, eu gostaria de usar o down-root plugin para chamar um script com privilégios de root.

plugin /usr/lib/openvpn/openvpn-plugin-down-root.so /usr/share/openvpn/down.sh

/usr/share/openvpn/down.sh

#!/bin/sh

/sbin/ip addr del dev $1 local $4 peer $5

tente reiniciar, mas entendi:

May 17 14:49:47 t-xhyve vpn-devops[23001]: /sbin/ip addr del dev tun1 local 192.168.0.1 peer 192.168.0.2
May 17 14:49:47 t-xhyve vpn-devops[23228]: PKCS#11: __pkcs11h_forkFixup entry pid=23228, activate_slotevent=1
May 17 14:49:47 t-xhyve vpn-devops[23228]: PKCS#11: __pkcs11h_forkFixup return
May 17 14:49:47 t-xhyve vpn-devops[23001]: Linux ip addr del failed: external program exited with error status: 2
May 17 14:49:47 t-xhyve vpn-devops[23001]: PLUGIN_CALL: PRE type=PLUGIN_DOWN
May 17 14:49:47 t-xhyve vpn-devops[23001]: ARGV[0] = '/usr/lib/openvpn/openvpn-plugin-down-root.so'
May 17 14:49:47 t-xhyve vpn-devops[23001]: ARGV[1] = 'tun1'
May 17 14:49:47 t-xhyve vpn-devops[23001]: ARGV[2] = '1500'
May 17 14:49:47 t-xhyve vpn-devops[23001]: ARGV[3] = '1542'
May 17 14:49:47 t-xhyve vpn-devops[23001]: ARGV[4] = '192.168.0.1'
May 17 14:49:47 t-xhyve vpn-devops[23001]: ARGV[5] = '192.168.0.2'
May 17 14:49:47 t-xhyve vpn-devops[23001]: ARGV[6] = 'init'
May 17 14:49:47 t-xhyve vpn-devops[23001]: ENVP[0] = 'dev=tun1'
May 17 14:49:47 t-xhyve vpn-devops[23001]: ENVP[1] = 'link_mtu=1542'
May 17 14:49:47 t-xhyve vpn-devops[23001]: ENVP[2] = 'tun_mtu=1500'
May 17 14:49:47 t-xhyve vpn-devops[23001]: ENVP[3] = 'script_context=init'
May 17 14:49:47 t-xhyve vpn-devops[23001]: ENVP[4] = 'signal=sigterm'
May 17 14:49:47 t-xhyve vpn-devops[23001]: ENVP[5] = 'redirect_gateway=0'
May 17 14:49:47 t-xhyve vpn-devops[23001]: ENVP[6] = 'dev_type=tun'
May 17 14:49:47 t-xhyve vpn-devops[23001]: ENVP[7] = 'ifconfig_remote=192.168.0.2'
May 17 14:49:47 t-xhyve vpn-devops[23001]: ENVP[8] = 'ifconfig_local=192.168.0.1'
May 17 14:49:47 t-xhyve vpn-devops[23001]: ENVP[9] = 'config=/etc/openvpn/devops/config'
May 17 14:49:47 t-xhyve vpn-devops[23001]: ENVP[10] = 'verb=15'
May 17 14:49:47 t-xhyve vpn-devops[23001]: ENVP[11] = 'daemon=0'
May 17 14:49:47 t-xhyve vpn-devops[23001]: ENVP[12] = 'daemon_log_redirect=0'
May 17 14:49:47 t-xhyve vpn-devops[23001]: ENVP[13] = 'daemon_start_time=1463471367'
May 17 14:49:47 t-xhyve vpn-devops[23001]: ENVP[14] = 'daemon_pid=23001'
May 17 14:49:47 t-xhyve vpn-devops[23001]: ENVP[15] = 'proto_1=udp'
May 17 14:49:47 t-xhyve vpn-devops[23001]: ENVP[16] = 'local_port_1=1196'
May 17 14:49:47 t-xhyve vpn-devops[23001]: ENVP[17] = 'remote_port_1=1196'
May 17 14:49:47 t-xhyve vpn-devops[23001]: PLUGIN_CALL: POST /usr/lib/openvpn/openvpn-plugin-down-root.so/PLUGIN_DOWN status=1
May 17 14:49:47 t-xhyve vpn-devops[23001]: PLUGIN_CALL: plugin function PLUGIN_DOWN failed with status 1: /usr/lib/openvpn/openvpn-plugin-down-root.so
May 17 14:49:47 t-xhyve vpn-devops[23001]: ERROR: up/down plugin call failed
May 17 14:49:47 t-xhyve vpn-devops[23001]: Exiting due to fatal error
May 17 14:49:50 t-xhyve openvpn-devops-upstart: RTNETLINK answers: Operation not permitted
May 17 14:49:50 t-xhyve openvpn-devops-upstart: DOWN-ROOT: Error sending script execution signal to background process

Eu tenho algumas perguntas:

  1. Por que o OpenVPN ainda chama isso?

    /sbin/ip addr del dev tun1 local 192.168.0.1 peer 192.168.0.2
    
  2. Por que eu tenho isso?

    May 17 14:49:47 t-xhyve vpn-devops[23001]: ERROR: up/down plugin call failed
    May 17 14:49:47 t-xhyve vpn-devops[23001]: Exiting due to fatal error
    May 17 14:49:50 t-xhyve openvpn-devops-upstart: RTNETLINK answers: Operation not permitted
    May 17 14:49:50 t-xhyve openvpn-devops-upstart: DOWN-ROOT: Error sending script execution signal to background process
    

Como resolver isso?

Are there any app-armor profiles in place?

Não.

Try modifying the shell script to do cat /proc/self/status >/tmp/temp.txt and printing out the result here.

/usr/share/openvpn/down.sh

#!/bin/sh

cat /proc/self/status >/tmp/temp.txt
#/sbin/ip addr del dev $1 local $4 peer $5

ao reiniciar:

May 18 16:51:00 t-xhyve vpn-devops[17930]: TCP/UDP: Closing socket
May 18 16:51:00 t-xhyve vpn-devops[17930]: Closing TUN/TAP interface
May 18 16:51:00 t-xhyve vpn-devops[17930]: /sbin/ip addr del dev tun1 local 192.168.0.1 peer 192.168.0.2
May 18 16:51:00 t-xhyve vpn-devops[18520]: PKCS#11: __pkcs11h_forkFixup entry pid=18520, activate_slotevent=1
May 18 16:51:00 t-xhyve vpn-devops[18520]: PKCS#11: __pkcs11h_forkFixup return
May 18 16:51:00 t-xhyve vpn-devops[17930]: Linux ip addr del failed: external program exited with error status: 2
May 18 16:51:00 t-xhyve vpn-devops[17930]: PLUGIN_CALL: PRE type=PLUGIN_DOWN
May 18 16:51:00 t-xhyve vpn-devops[17930]: ARGV[0] = '/usr/lib/openvpn/openvpn-plugin-down-root.so'
May 18 16:51:00 t-xhyve vpn-devops[17930]: ARGV[1] = 'tun1'
May 18 16:51:00 t-xhyve vpn-devops[17930]: ARGV[2] = '1500'
May 18 16:51:00 t-xhyve vpn-devops[17930]: ARGV[3] = '1542'
May 18 16:51:00 t-xhyve vpn-devops[17930]: ARGV[4] = '192.168.0.1'
May 18 16:51:00 t-xhyve vpn-devops[17930]: ARGV[5] = '192.168.0.2'
May 18 16:51:00 t-xhyve vpn-devops[17930]: ARGV[6] = 'init'
May 18 16:51:00 t-xhyve vpn-devops[17930]: ENVP[0] = 'dev=tun1'
May 18 16:51:00 t-xhyve vpn-devops[17930]: ENVP[1] = 'link_mtu=1542'
May 18 16:51:00 t-xhyve vpn-devops[17930]: ENVP[2] = 'tun_mtu=1500'
May 18 16:51:00 t-xhyve vpn-devops[17930]: ENVP[3] = 'script_context=init'
May 18 16:51:00 t-xhyve vpn-devops[17930]: ENVP[4] = 'signal=sigterm'
May 18 16:51:00 t-xhyve vpn-devops[17930]: ENVP[5] = 'redirect_gateway=0'
May 18 16:51:00 t-xhyve vpn-devops[17930]: ENVP[6] = 'dev_type=tun'
May 18 16:51:00 t-xhyve vpn-devops[17930]: ENVP[7] = 'ifconfig_remote=192.168.0.2'
May 18 16:51:00 t-xhyve vpn-devops[17930]: ENVP[8] = 'ifconfig_local=192.168.0.1'
May 18 16:51:00 t-xhyve vpn-devops[17930]: ENVP[9] = 'config=/etc/openvpn/devops/config'
May 18 16:51:00 t-xhyve vpn-devops[17930]: ENVP[10] = 'verb=15'
May 18 16:51:00 t-xhyve vpn-devops[17930]: ENVP[11] = 'daemon=0'
May 18 16:51:00 t-xhyve vpn-devops[17930]: ENVP[12] = 'daemon_log_redirect=0'
May 18 16:51:00 t-xhyve vpn-devops[17930]: ENVP[13] = 'daemon_start_time=1463564660'
May 18 16:51:00 t-xhyve vpn-devops[17930]: ENVP[14] = 'daemon_pid=17930'
May 18 16:51:00 t-xhyve vpn-devops[17930]: ENVP[15] = 'proto_1=udp'
May 18 16:51:00 t-xhyve vpn-devops[17930]: ENVP[16] = 'local_port_1=1196'
May 18 16:51:00 t-xhyve vpn-devops[17930]: ENVP[17] = 'remote_port_1=1196'
May 18 16:51:00 t-xhyve vpn-devops[17930]: PLUGIN_CALL: POST /usr/lib/openvpn/openvpn-plugin-down-root.so/PLUGIN_DOWN status=1
May 18 16:51:00 t-xhyve vpn-devops[17930]: PLUGIN_CALL: plugin function PLUGIN_DOWN failed with status 1: /usr/lib/openvpn/openvpn-plugin-down-root.so
May 18 16:51:00 t-xhyve vpn-devops[17930]: ERROR: up/down plugin call failed
May 18 16:51:00 t-xhyve vpn-devops[17930]: Exiting due to fatal error
May 18 16:51:00 t-xhyve [2016-05-18 16:51:00,738][WARN ]May 18 16:51:01 t-xhyve openvpn-devops-upstart: RTNETLINK answers: Operation not permitted
May 18 16:51:01 t-xhyve openvpn-devops-upstart: DOWN-ROOT: Error sending script execution signal to background process

O processo foi reiniciado, mas o arquivo de log ( /tmp/temp.txt ) não foi criado.

Claro, executar esse script manualmente funcionou bem:

bash -x /usr/share/openvpn/down.sh
 + cat /proc/self/status
 + 

cat /tmp/temp.txt
Name:   cat
State:  R (running)
Tgid:   19246
Ngid:   0
Pid:    19246
PPid:   19245
...

A especificação do caminho completo não ajudou:

May 19 03:52:11 t-xhyve openvpn-devops-upstart: RTNETLINK answers: Operation not permitted
May 19 03:52:11 t-xhyve openvpn-devops-upstart: DOWN-ROOT: Error sending script execution signal to background process
May 19 03:52:11 t-xhyve openvpn-devops-upstart: DOWN-ROOT: BACKGROUND: INIT command='/bin/cat /proc/self/status > /tmp/temp.txt'

How are you starting/stopping the service?

@ rda: aqui está o arquivo inicial:

start on (net-device-up
          and local-filesystems
          and runlevel [2345]
          and started rsyslog)
stop on runlevel [!2345]

respawn
respawn limit 6 60

pre-start script
    if [ ! -e /var/run/openvpn ]; then
        mkdir -m 0770 /var/run/openvpn
        chown nobody:nogroup /var/run/openvpn
    fi
end script

exec /usr/sbin/openvpn --config /etc/openvpn/devops/config
post-stop exec sleep 5

An alternative script without using start-stop-daemon, which sends SIGTERM to the main process in an pre-stop script stanza, could look as follows:

Adicionei um script pre-stop como você sugeriu:

start on (net-device-up
          and local-filesystems
          and runlevel [2345]
          and started rsyslog)
stop on runlevel [!2345]

respawn
respawn limit 6 60

env PIDFILE="/var/run/openvpn/devops.pid"

pre-start script
    if [ ! -e /var/run/openvpn ]; then
        mkdir -m 0770 /var/run/openvpn
        chown nobody:nogroup /var/run/openvpn
    fi
end script

exec /usr/sbin/openvpn --config /etc/openvpn/devops/config

pre-stop script
  PID='cat $PIDFILE'
  kill -15 $PID
  sleep 1
  if [ "$?" -eq 0 ]; then
    rm -f $PIDFILE
  else
    echo "Unable to stop VPN '$NAME'"
  fi
end script

post-stop exec sleep 5

tentei reiniciar usando restart openvpn-devops e ainda tenho isto:

May 25 02:41:54 t-xhyve openvpn-devops-upstart: RTNETLINK answers: Operation not permitted
May 25 02:41:54 t-xhyve openvpn-devops-upstart: DOWN-ROOT: Error sending script execution signal to background process
May 25 02:41:54 t-xhyve openvpn-devops-upstart: DOWN-ROOT: BACKGROUND: INIT command='/bin/cat /proc/self/status > /tmp/temp.txt'
    
por quanta 18.05.2016 / 15:00

1 resposta

1

Eu tento responder as perguntas na ordem inversa:

2. Por que eu recebo isso?

DOWN-ROOT: Error sending script execution signal to background process

Quando o plug-in down-root for usado, o processo principal será fork() antes de descartar os privilégios de root. O módulo permanecerá em um estado de espera até receber uma mensagem do processo principal via pipe para executar o script descendente (consulte readme ). O processo principal enviará esta mensagem para o processo do módulo down-root quando receber SIGTERM .

Como mencionado neste bug do OpenVPN com o arquivo de configuração do serviço systemd , Esse erro provavelmente é causado em conjunto com upstart (o daemon init padrão usado pelo Ubuntu 14.04), que enviará SIGTERM para ambos os processos. O processo do módulo down-root terminará, antes que ele tenha a chance de fazer seu trabalho, chamando o script para baixo.

Use o /etc/init.d/openvpn enviado com o Ubuntu para iniciar e parar o serviço ou corrigi-lo no script upstart . Um script de inicialização muito simples poderia parecer o seguinte. Note que usa start-stop-daemon para enviar os sinais corretos.

Crie um arquivo /etc/init/openvpn.conf

start on (net-device-up
          and local-filesystems
          and runlevel [2345]
          and started rsyslog)
stop on runlevel [!2345]

respawn
respawn limit 6 60

env PIDFILE="/var/run/openvpn/devops.pid"

pre-start script
    if [ ! -e /var/run/openvpn ]; then
        mkdir -m 0770 /var/run/openvpn
        chown nobody:nogroup /var/run/openvpn
    fi
end script

exec start-stop-daemon --start --quiet --oknodo \
  --pidfile /var/run/openvpn/devops.pid \
  --exec /usr/sbin/openvpn -- --config /etc/openvpn/devops/config < /dev/null

pre-stop script
  exec start-stop-daemon --stop --quiet --oknodo \
    --pidfile $PIDFILE --exec /usr/sbin/openvpn --retry 5
  if [ "$?" -eq 0 ]; then
    rm -f $PIDFILE
  else
    echo "Unable to stop VPN"
  fi
end script

post-stop exec sleep 5

Um script alternativo sem usar start-stop-daemon , que envia SIGTERM para o processo principal em uma sub-rotina de script pre-stop , pode ter a seguinte aparência:

start on (net-device-up
          and local-filesystems
          and runlevel [2345]
          and started rsyslog)
stop on runlevel [!2345]

respawn
respawn limit 6 60

env PIDFILE="/var/run/openvpn/devops.pid"

pre-start script
    if [ ! -e /var/run/openvpn ]; then
        mkdir -m 0770 /var/run/openvpn
        chown nobody:nogroup /var/run/openvpn
    fi
end script

exec /usr/sbin/openvpn --config /etc/openvpn/devops/config

pre-stop script
  PID='cat $PIDFILE'
  kill -15 $PID
  sleep 1
  if [ "$?" -eq 0 ]; then
    rm -f $PIDFILE
  else
    echo "Unable to stop VPN"
  fi
end script

post-stop exec sleep 5

Controlando o trabalho upstart :

initctl <command> openvpn
  • <command> : start , stop , restart

Isso deve remover o erro acima e executar o script down.sh corretamente, mas o seguinte erro ainda será exibido:

Linux ip addr del failed: external program exited with error status: 2

Este erro ocorre em primeiro lugar, porque ip addr del é executado depois que a interface TUN / TAP já foi fechada.

Mais sobre isso abaixo.

1. Por que o OpenVPN ainda chama isso?

/sbin/ip addr del dev tun1 local 192.168.0.1 peer 192.168.0.2

Isso é causado pela diretiva ifconfig , que adicionará / del o endereço IP da interface TUN / TAP ao iniciar / parar o serviço. (Veja manpage )

Para contornar esse comportamento, modifique a configuração da seguinte forma.

arquivo de configuração do servidor OpenVPN :

topology net30
mode server
tls-server
push "route-gateway 192.168.0.1"
ifconfig 192.168.0.1 192.168.0.2
ifconfig-pool 192.168.0.5 192.168.0.9
ifconfig-noexec
up /usr/share/openvpn/up.sh
down-pre
plugin /usr/lib/openvpn/openvpn-plugin-down-root.so /usr/share/openvpn/down.sh
script-security 2
port 1196
proto udp
dev tun
keepalive 10 120
comp-lzo
user nobody
group nogroup
persist-key
persist-tun
status /var/log/openvpn/devops.log 1
syslog vpn-devops
verb 15
writepid /var/run/openvpn/devops.pid
ca /etc/openvpn/devops/ca.crt
cert /etc/openvpn/devops/server.crt
key /etc/openvpn/devops/server.key
dh /etc/openvpn/dh.pem
ifconfig-pool-persist /var/lib/openvpn/devops.ipp
client-config-dir /etc/openvpn/devops/ccd
crl-verify /etc/openvpn/devops/crl.pem
client-to-client
  • ifconfig-noexec : não execute comandos ifconfig, passe --ifconfig parameters para scripts.
  • up : é necessário executar um script para configurar um endereço IP devido a ifconfig-noexec
  • down-pre : chame o script --down antes, em vez de depois, TUN / TAP fechar.
  • script-security 2 : permite a chamada de executáveis internos e scripts definidos pelo usuário.

script UP /usr/share/openvpn/up.sh :

#!/bin/sh

/sbin/ip addr add dev $1 local $4 peer $5
/sbin/ip link set dev $1 up

Script DOWN /usr/share/openvpn/down.sh :

#!/bin/sh

/sbin/ip addr del dev $1 local $4 peer $5
/sbin/ip link set dev $1 down

Os scripts UP / DOWN também trazem a interface para cima e para baixo (embora não seja obrigatório).

Verifique se eles são executáveis:

chmod +x /usr/share/openvpn/up.sh
chmod +x /usr/share/openvpn/down.sh

Como alternativa, use uma interface TUN persistente

arquivo de configuração do servidor OpenVPN (edite / adicione as linhas):

dev tun1
# down-pre
  • dev tun1 : usaremos uma interface TUN persistente
  • # down-pre : não é necessário usar a interface persistente

Por fim, crie uma interface TUN persistente :

openvpn --mktun --dev tun1

Isto constrói um túnel persistente tun1 que vive através de múltiplas instanciações do OpenVPN e morre somente quando é deletado com openvpn --rmtun --dev tun1 ou a máquina é reinicializada.

Resolução de problemas

Aqui estão alguns passos adicionais de resolução de problemas, se a configuração estiver correta, conforme mostrado acima, e o seguinte erro ainda será exibido:

DOWN-ROOT: Error sending script execution signal to background process

Inicie a instância do OpenVPN, por exemplo, com upstart :

initctl start openvpn

Verifique o que PID dos processos têm:

ps -ef | grep "[o]penvpn"

# or more specific for our devops server example
ps -ef | grep "[o]penvpn/devops"

# this should show 2 openvpn processes
UID        PID  PPID  C STIME TTY          TIME CMD
nobody    3590     1  0 21:49 ?        00:00:00 /usr/sbin/openvpn --config /etc/openvpn/devops/config
root      3593  3590  0 21:49 ?        00:00:00 /usr/sbin/openvpn --config /etc/openvpn/devops/config

Enquanto segue os logs, mata o processo principal openvpn (executando como usuário nobody ) manualmente com SIGTERM usando seu PID

kill -15 3590

Ou alternativamente em uma única linha:

kill -15 'ps -ef | grep "^[n]obody .* /usr/sbin/openvpn --config /etc/openvpn/devops/config$" | awk '{print $2}''

A saída do log condensado deve ficar assim (sem o DOWN-ROOT: Error ... ):

vpn-devops[3590]: TCP/UDP: Closing socket
vpn-devops[3590]: PLUGIN_CALL: PRE type=PLUGIN_DOWN
upstart-devops  : DOWN-ROOT: BACKGROUND: received command code: 0
vpn-devops[3590]: PLUGIN_CALL: POST /usr/lib/openvpn/openvpn-plugin-down-root.so/PLUGIN_DOWN status=0
vpn-devops[3590]: Closing TUN/TAP interface
vpn-devops[3590]: PLUGIN_CLOSE: /usr/lib/openvpn/openvpn-plugin-down-root.so
vpn-devops[3590]: PID packet_id_free
vpn-devops[3590]: SIGTERM[hard,] received, process exiting
upstart-devops  : DOWN-ROOT: close
upstart-devops  : DOWN-ROOT: BACKGROUND: received command code: 1
upstart-devops  : DOWN-ROOT: BACKGROUND: EXIT
kernel: init    : ovpn-devops main process ended, respawning
    
por 23.05.2016 / 17:18