O VirtualBox alega oferecer suporte a esse recurso, de acordo com o manual :
Essentially this feature allows to directly use physical PCI devices on the host by the guest even if host doesn't have drivers for this particular device. Both, regular PCI and some PCI Express cards, are supported. AGP and certain PCI Express cards are not supported at the moment if they rely on GART (Graphics Address Remapping Table) unit programming for texture management as it does rather nontrivial operations with pages remapping interfering with IOMMU. This limitation may be lifted in future releases.
[...]
Intel's solution for IOMMU is marketed as "Intel Virtualization Technology for Directed I/O" (VT-d), and AMD's one is called AMD-Vi. So please check if your motherboard datasheet has appropriate technology. Even if your hardware doesn't have a IOMMU, certain PCI cards may work (such as serial PCI adapters), but the guest will show a warning on boot and the VM execution will terminate if the guest driver will attempt to enable card bus mastering.
Quanto ao VMware, não consegui encontrar nada conclusivo, mas encontrei esta questão relacionada em Super User com uma resposta:
But that doesn't really matter for you since VT-d, is not supported under Vmware Workstation [...]
O VT-d pode trazer alguma melhoria de desempenho, já que seu objetivo é permitir que máquinas virtuais usem dispositivos de hardware sem manipulação extra do host.