Não há um inerentemente embutido no Windows, então você precisará de um utilitário de terceiros.
Você pode usar o DisplayFusion para ativá-lo:
DisplayFusion's Alt+Tab Handler has additional modes over and above what Windows offers. You can have the Alt+Tab window shown on all monitors, the current monitor, or just the Primary monitor. It can show all open windows across all monitors, or just the windows on the current monitor. You can also choose whether you'd like the window titles shown below the entries in Alt+Tab, and whether you'd like the "Desktop" entry shown or not.
