Securing ssh server so only one user has external access with certain key

I wanted to secure my ssh server so that external access is only for one user, who is only able to use one key (with a passphrase) externally. The user is configured with another key that is used internally (has no passphrase). Other users are able to connect internally using their keys.

My user that requires local and external access in this example is "me" and the other user that has ssh access, but only from the internal network is pi.

This was done, after a bit of googling with this method.

Create a group, for users who are allowed ssh access. Add users to it, eg.

> sudo groupadd sshusers
> sudo adduser me sshusers
> sudo adduser pi sshusers

Then configure ssh server, edit /etc/ssh/sshd_config

> sudo -e /etc/ssh/ssd_config

Ensure password authentication is off by changing the following

PasswordAuthentication no

Allow the sshusers group access by adding the line

AllowGroups sshusers

Deny any users access to ssh, except via the local network (my local network addresses are 192.168.1.*). Add the following line for each user, changing the username and address as required.

DenyUsers pi@!192.168.1.*,

Save the configuration.

The DenyUsers line, will deny the given user pi access to all addresses except those in the wildcard 192.168.1.*.

Now, create the key pair for the internal access only users, eg for pi. (use sudo -u user -s to switch user)

> cd ~
> mkdir .ssh
> cd .ssh
> ssh-keygen -f id_rsa
> cat id_rsa.pub >> authorized_keys
> chmod go-rwx authorized_keys

Now that is done, the id_rsa file can be used on the client to access this user.

To configure the user who has internal and external access.

> cd ~
> mkdir .ssh
> cd .ssh
> ssh-keygen -f id_rsa_external      (Enter a passphrase)
> ssh-keygen -f id_rsa_internal      (no passphrase or one)
> echo -n "FROM=\"192.168.1.*\" " > authorized_keys
> cat id_rsa_internal.pub >> authorized_keys
> cat id_rsa_external.pub >> authorized_keys
> chmod go-rwx authorized_keys