Ограничить доступ юзера к SSH по ключу только определенным IP

man 8 sshd

from="pattern-list"
Specifies that in addition to public key authentication, the canonical name of the
remote host must be present in the comma-separated list of patterns (‘*’ and ‘?’ serve
as wildcards). The list may also contain patterns negated by prefixing them with ‘!’;
if the canonical host name matches a negated pattern, the key is not accepted. The pur-
pose of this option is to optionally increase security: public key authentication by
itself does not trust the network or name servers or anything (but the key); however, if
somebody somehow steals the key, the key permits an intruder to log in from anywhere in
the world. This additional option makes using a stolen key more difficult (name servers
and/or routers would have to be compromised in addition to just the key).