Estou enfrentando algum problema com o DPDK no Ubuntu 16.04 com uma placa de rede Mellanox 40G.
Não consigo atribuir o NIC a um ovs (tentando usar o driver vfio-pci).
Quando tento atribuir a NIC ao OVS com o seguinte comando:
ovs-vsctl add-port br0 ens1 -- set Interface ens1 type=dpdk options:dpdk-devargs=0000:07:00.0
Eu recebo a seguinte mensagem de log:
Apr 17 15:26:43 olnmalp026Bn001 ovs-vsctl: ovs|00001|vsctl|INFO|Called as ovs-vsctl add-port br0 ens1 -- set Interface ens1 type=dpdk options:dpdk-devargs=0000:07:00.0
Apr 17 15:26:43 olnmalp026Bn001 ovs-vswitchd[56850]: ovs|00184|dpdk|ERR|EAL: Driver cannot attach the device (0000:07:00.0)
Apr 17 15:26:43 olnmalp026Bn001 ovs-vswitchd[56850]: ovs|00185|dpdk|ERR|EAL: No port found for device (0000:07:00.0)
A coisa mais estranha que notei é que, se eu executar o dpdk-procinfo, recebo a seguinte saída:
root@olnmalp026Bn001:/home/sysadmin# dpdk-procinfo
EAL: Detected 56 lcore(s)
EAL: Probing VFIO support...
EAL: cannot connect to primary process!
EAL: VFIO support could not be initialized
EAL: PCI device 0000:02:00.0 on NUMA socket 0
EAL: probe driver: 8086:1521 net_e1000_igb
EAL: PCI device 0000:02:00.1 on NUMA socket 0
EAL: probe driver: 8086:1521 net_e1000_igb
dpdk-procinfo [EAL options] -- -p PORTMASK
-m to display DPDK memory zones, segments and TAILQ information
-p PORTMASK: hexadecimal bitmask of ports to retrieve stats for
--stats: to display port statistics, enabled by default
--xstats: to display extended port statistics, disabled by default
--metrics: to display derived metrics of the ports, disabled by default
--xstats-name NAME: to display single xstat id by NAME
--xstats-ids IDLIST: to display xstat values by id. The argument is comma-separated list of xstat ids to print out.
--stats-reset: to reset port statistics
--xstats-reset: to reset port extended statistics
--collectd-format: to print statistics to STDOUT in expected by collectd format
--host-id STRING: host id used to identify the system process is running on
EAL: Error - exiting with code: 1
Cause: No Ethernet ports - bye
A placa Mellanox, não aparece nos dispositivos PCI listados nesse comando!
root@olnmalp026Bn001:/home/sysadmin# lspci | grep -i mellan
07:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx]
Mas posso atribuir o endereço PCI ao driver vfio-pci:
root@olnmalp026Bn001:/home/sysadmin# dpdk-devbind -s
Network devices using DPDK-compatible driver
============================================
0000:07:00.0 'MT27710 Family [ConnectX-4 Lx] 1015' drv=vfio-pci unused=uio_pci_generic
Network devices using kernel driver
===================================
0000:02:00.0 'I350 Gigabit Network Connection 1521' if=enp2s0f0 drv=igb unused=vfio-pci,uio_pci_generic
0000:02:00.1 'I350 Gigabit Network Connection 1521' if=enp2s0f1 drv=igb unused=vfio-pci,uio_pci_generic
...
Acho que o driver Mellanox está instalado corretamente na máquina:
root@olnmalp026Bn001:/home/sysadmin# mst start
Starting MST (Mellanox Software Tools) driver set
Loading MST PCI module - Success
[warn] mst_pciconf is already loaded, skipping
Create devices
Unloading MST PCI module (unused) - Success
root@olnmalp026Bn001:/home/sysadmin# lsmod | grep mst_
mst_pciconf 102400 0
Alguns dados sobre o sistema, caso isso ajude:
root@olnmalp026Bn001:/home/sysadmin# uname -a
Linux olnmalp026Bn001 4.4.0-87-generic #110-Ubuntu SMP Tue Jul 18 12:55:35 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
root@olnmalp026Bn001:/home/sysadmin# ovs-vsctl --version
ovs-vsctl (Open vSwitch) 2.9.0
DB Schema 7.15.1
root@olnmalp026Bn001:/home/sysadmin# dpkg -l | grep -i dpdk
ii dpdk 17.11-5 amd64 Data Plane Development Kit (runtime)
ii mlnx-ofed-dpdk 4.3-1.0.1.0 all MLNX_OFED dpdk installer package (with DKMS support)
ii openvswitch-switch-dpdk 2.9.0-2 amd64 DPDK enabled Open vSwitch switch implementation