Update (2011-12-10): I updated this article to make it a bit more clear and confirmed it works exactly the same on Linode.
Update (2014-09-21): Just a note that I've used this successfully on several times on Digital Ocean.
This is a step by step I used to set up SSH and lock down access on my Rackspace Cloud VPS running Ubuntu Server 10.04. There's a good chance this will work for other server providers and possibly other versions of Ubuntu. (But no guarantees.)
Disclaimer: This process worked for me, but it might not work for you. Be sure you understand what you're doing. You can lock yourself out of your server. Use at your own risk.
Rule #1: We don't want to log in as root ever! (OK, just this once.)
Create your own ssh public/private key pair (if you haven't already). The tutorial on Github is actually pretty good (http://help.github.com/mac-set-up-git/). There Windows and Linux versions too. You can just create your key and ignore the git stuff.
First, create an ordinary user account that can use the sudo command to 'pretend' to be root.
1) log in as root:
You should be successfully logged in. Obviously you should use the hostname or IP of your server.
2) Add an ordinary non-root user:
useradd -m -s /bin/bash -G sudo cleaver passwd cleaver
In the first command you should use your name instead of "cleaver". In the second command you'll set your password. (WARNING: Don't lose this... it will be your only way to get root access after this!)
3) Now log out (Ctrl+D)
4) Now set up the account for SSH key access. Copy the key up to the server, then log in.
scp ~/.ssh/id_rsa.pub [email protected]: ssh [email protected] # you should be prompted for the password of your non-root user
Now make a .ssh directory if it's not already there and copy your public key.
mkdir .ssh cat id_rsa.pub >> .ssh/authorized_keys rm id_rsa.pub
5) log out (Ctrl+D)
6) Now make sure you really can act as root.
Log in again:
(You should be prompted for your ssh passphrase and not have to enter your password for the user account.)
7) Next, we pretend to be root!
sudo su -
This time enter the password for your user account. Fortunately, you don't have to type this every time--just after a period of inactivity. Note: the hyphen at the end sets up the environment as root--omit it and you inherit the original user's environment (including the PATH).
8) Now check that you really are root.
The answer should be "root". WARNING: You really want to be sure you get this right--next step is to disable remote logins from root.
9) Log out from root, but not from your user account. (Ctrl+D)
10) You're now logged in as an ordinary user and you'll use sudo to execute privileged commands and disable remote logins from the root account.
sudo nano /etc/ssh/sshd_config
(This uses the nano editor, but you can substitute vi or anything else.)
11) Look for this line:
And change it to:
Ctrl-X to exit - be sure to save the file on the way out.
(Make really, really sure you got everything right. This is where you can get locked out.)
sudo /etc/init.d/ssh reload
The command above tells the ssh daemon to reload the configuration file you just changed.
12) Log out.
13) Try to login in as root.
14) Try again as your non-root user: