SSH Keys vs Passwords: Setting Up Access Correctly

Every cloud provider nudges you towards SSH keys and away from passwords, and
unusually for a security recommendation, this one involves no trade-off against
convenience. Keys are both more secure and easier to use once set up.

Why passwords fail on an internet-facing server

A server with SSH open to the internet receives automated login attempts
continuously, from the moment it exists. These are not targeted at you; they are
broad sweeps trying common usernames and passwords against every reachable
address.

A password has to be guessed, and guessing is something computers are
patient about. A private key cannot be guessed in any practical sense. That is
the whole of the argument, and it is sufficient.

How the pieces fit together

A key pair is two related files. The public key is not
secret and goes on every server you want to reach. The private key
stays on your machine and never goes anywhere else, ever.

When you connect, the server issues a challenge that can only be answered by
someone holding the private key. The private key itself is never transmitted.
This is why a compromised server cannot leak your key: it never had it.

The private key should itself be protected with a passphrase. That passphrase
encrypts the key file, so that someone who copies the file still cannot use it.
An SSH agent holds the decrypted key in memory for your session, which means you
type the passphrase once rather than on every connection.

Generating a key

Use a modern algorithm. Ed25519 is the current default recommendation: it is
fast, keys are short, and it has no awkward key-size decisions. RSA at 4096 bits
remains acceptable where something old does not support Ed25519.

Generate the key on the machine you will connect from. Set a passphrase when
prompted. Then add the public key to the provider’s console so it can be
installed on new instances automatically at creation time, which is tidier than
copying it onto each machine afterwards.

Lock the door behind you

Adding a key does not remove password access. Both remain enabled until you
disable passwords explicitly in the SSH server configuration, and until you do,
the password attack surface is exactly as large as before.

Disable password authentication and root login, reload the service, and then
open a second session to confirm before closing the one you are working in. This
ordering matters: if the configuration is wrong, the open session is how you fix
it.

The mistakes that undermine it

• Copying the private key between machines. Generate a
  separate key on each device instead. Then losing one device means removing one
  public key rather than rotating everything.
• No passphrase. A passphraseless key is a file that grants
  server access to anyone who obtains it. Use an agent so the passphrase costs you
  nothing in daily use.
• Sharing one key across a team. Each person should have
  their own. Shared keys make it impossible to say who did something and mean
  every departure requires rotating access for everybody.
• Checking a key into version control. This happens more than
  anyone would like, usually through a dotfiles repository. Private keys belong
  nowhere near a repository, public or otherwise.
• Never removing old keys. The authorised keys file
  accumulates. Review it occasionally and delete entries you cannot account
  for.

Beyond the basics

Two additions are worth the small effort once you are comfortable.

Move SSH behind the provider’s network firewall, restricted to the addresses
you actually connect from. This removes the machine from broad internet scanning
entirely, which is more effective than anything you can configure on the host.

For a team, consider a bastion host: one hardened machine that accepts SSH
from outside, with every other server reachable only from it. This gives a
single place to manage access and a single place where connections are logged.

What about the provider’s console?

Every provider offers browser-based console access that works even when SSH
is broken. It is slow and awkward, which is fine, because it is a recovery tool
rather than a daily one. Find out where it is in your provider’s interface
before you need it. The day you need it, you will be locked out and unable to
read the documentation comfortably.