Use ssh-agent e ssh-add todas as chaves que você precisa.
Exemplo:
# start the agent and capture its environment in the current shell
eval 'ssh-agent'
# add keys needed to connect to the different accounts
ssh-add /path/to/first/ssh/key
ssh-add /path/to/second/ssh/key
# do the copying
scp [email protected]:file1 [email protected]:file2