Tente usar o argumento -i
.
Do homem:
-i identity_file
Selects a file from which the identity (private key) for RSA or
DSA authentication is read. The default is ~/.ssh/identity for
protocol version 1, and ~/.ssh/id_rsa and ~/.ssh/id_dsa for pro-
tocol version 2. Identity files may also be specified on a per-
host basis in the configuration file. It is possible to have
multiple -i options (and multiple identities specified in config-
uration files).
Em seguida, você pode especificar a chave a ser usada para cada host.