É para isso que serve o make localmodconfig
.
Trecho de /usr/src/linux/README
:
"make localmodconfig" Create a config based on current config and loaded modules (lsmod). Disables any module option that is not needed for the loaded modules.