Binding VF to VFIO inside QEMU

Bnding the VF to vfio-pci in the guest VM , there are two option

1.   Enabling vIOMMU inside QEMU/VM

2.   Using no-iommu mode of VFIO drivers

 

NO_IOMMU_MODE 

On Guest VM

 1.modprobe vfio-pci

2.echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode

3. usertools/dpdk-devbind.py -b vfio-pci 07:00.0

 

VIOMMU MODE

 

On HOST Machine

================

1.load modules

   modprobe qede

  modprobe vfio-pci

 

2. Check PF B:D:F ( bus:device:function)

   #lspci | grep QL

  04:00.0 Ethernet controller: QLogic Corp. FastLinQ QL41000 Series 10/25/40/50GbE Controller (rev 02)

  04:00.1 Ethernet controller: QLogic Corp. FastLinQ QL41000 Series 10/25/40/50GbE Controller (rev 02)

 

3.Create VF on a PF

echo 1 > /sys/devices/pci0000:00/0000:00:03.0/0000:04:00.1/sriov_numvfs

 

4. Unbind the qede driver from VF so that it can imported to VM.

  echo -n "0000:04:0e.0" > /sys/bus/pci/drivers/qede/unbind

 

5.Get Vendor ID of VF device

  # lspci -nn | grep QL | grep IOV

  04:0e.0 Ethernet controller [0200]: QLogic Corp. FastLinQ QL41000 Series Gigabit Ethernet Controller (SR-IOV VF) [1077:8090] (rev 0

  * Value in square Bracket

 

5.Bind the device to the vfio-pci driver

  sudo echo "1077 8090" > /sys/bus/pci/drivers/vfio-pci/new_id

 

6.Start QEMU with Quest VM.

  /usr/bin/qemu-system-x86_64  -machine q35,kernel-irqchip=split,accel=kvm -smp 4 -m 2G \

  -device intel-iommu,intremap=on,caching-mode=on -nographic /home/fastlinq/centos-7.8.qcow2 \

  -device vfio-pci,host=04:0e.0

 

 

Guest VM

=======

1.edit  /etc/default/grub

and add these "iommu=pt intel_iommu=on"  at end of GRUB_CMDLINE_LINUX

 

e.g

GRUB_CMDLINE_LINUX="console=tty0 rd_NO_PLYMOUTH crashkernel=auto resume=UUID=40ff

14688-2619-4046-a9eb-b7333fff1b84 console=ttyS0,115200 iommu=pt intel_iommu=on"

 

 

2.update the grub using

a)grub2-mkconfig -o /boot/grub2/grub.cfg (For Redhat/Centos)

b) update-grub (For Ubuntu)

 

3.reboot

 

4.Check BDF for VF inside VM

# lspci | grep QL

00:03.0 Ethernet controller: QLogic Corp. FastLinQ QL41000 Series Gigabit Ethernet Controller (SR-IOV VF) (rev 02)

 

5. modprobe vfio-pci

 

6.Bind the VF to VFIO using below

  usertools/dpdk-devbind.py -b vfio-pci 00:03.0

 

7.Check status

 

[root@centos-8 dpdk-stable-19.11.5]#  usertools/dpdk-devbind.py --status-dev net

Network devices using DPDK-compatible driver

============================================

0000:00:03.0 'FastLinQ QL41000 Series Gigabit Ethernet Controller (SR-IOV VF) 8090' drv=vfio-pci unused=qede

 

Network devices using kernel driver

===================================

0000:00:02.0 '82574L Gigabit Network Connection 10d3' if=enp0s2 drv=e1000e unused=vfio-pci *Active*