Em man 5 ssh_config
:
Since the first obtained value for each parameter is used, more host-specific declarations should be given near the beginning of the file, and general defaults at the end.
Sua seção Host *
é mais geral. Deve ser no final.