Veja man-page do sshd-config:
ListenAddress Specifies the local addresses sshd(8) should listen on. The fol- lowing forms may be used:
ListenAddress host|IPv4_addr|IPv6_addr ListenAddress host|IPv4_addr:port ListenAddress [host|IPv6_addr]:port If port is not specified, sshd will listen on the address and all prior Port options specified. The default is to listen on all local addresses. **Multiple ListenAddress options are permitted.** Additionally, any Port options must precede this option for non- port qualified addresses.
Editar
Port
Specifies the port number that sshd(8) listens on. The default is 22. Multiple options of this type are permitted. See also ListenAddress.