SSH Key Rotation: The Security Task I Kept Putting Off

I have been using the same SSH key for three years. Yes, I know. It was on my to-do list, right next to "rotate AWS credentials" and "fix the DNS TTL". I finally did it last weekend and it was way less painful than I expected. Here is what I learned ( and what I got wrong ).

Why Rotate SSH Keys

Your SSH private key is the one thing standing between the internet and every server you manage. If it leaks ( and you would be surprised how often that happens ), every server that trusts that key is compromised. Period.

Key rotation means: generate a new keypair, copy the public key to your servers, remove the old one. That is it. The hard part is doing it without locking yourself out.

Generate a New Key Pair

Do not overthink the algorithm choice. Ed25519 is the default for a reason. It is faster, smaller, and more secure than RSA. Use it.

ssh-keygen -t ed25519 -C "davide@2026" -f ~/.ssh/id_ed25519_2026
# Generating public/private ed25519 key pair.
# Your identification has been saved in /home/dave/.ssh/id_ed25519_2026

The -C flag is for comment. I put the year in there so I know which key is which when I look at authorized_keys on a server six months from now. You should do the same.

If you are stuck on a system that does not support Ed25519 ( looking at you, old Cisco switches ), RSA with at least 4096 bits:

ssh-keygen -t rsa -b 4096 -C "davide@2026-rsa" -f ~/.ssh/id_rsa_2026

Copy the New Key to Your Servers

ssh-copy-id is the quickest way to get the new public key onto a server. It appends to authorized_keys without overwriting anything.

ssh-copy-id -i ~/.ssh/id_ed25519_2026.pub user@server1
ssh-copy-id -i ~/.ssh/id_ed25519_2026.pub user@server2
ssh-copy-id -i ~/.ssh/id_ed25519_2026.pub user@server3

If you have more than three servers, automate it. I use a simple bash loop from my hosts file:

#!/bin/bash
KEY=~/.ssh/id_ed25519_2026.pub
while read -r host; do
  echo "Deploying to $host..."
  ssh-copy-id -i "$KEY" "root@$host" 2>/dev/null && echo "  OK" || echo "  FAILED"
done < ~/.serverlist

Verify Before You Remove the Old Key

Critical step. Do not skip this. Before removing the old key, test that the new one actually works:

ssh -i ~/.ssh/id_ed25519_2026 user@server1 'echo "New key works on $(hostname)"'
# New key works on server1

Test every server. If even one fails, figure out why before you remove the old key. Common mistake: the server has an old sshd that does not support Ed25519. You will get a cryptic "no mutual signature algorithm" error.

Keep both keys in your ssh-agent during the transition period:

ssh-add ~/.ssh/id_ed25519_2026
ssh-add -l
# 256 SHA256:xxx... davide@2026 (ED25519)
# 256 SHA256:yyy... davide@2025 (ED25519)

Remove the Old Key from Servers

Once the new key works everywhere, remove the old public key from authorized_keys. This is the part most people skip, and it is the whole point of rotation.

# Remove old key by comment
OLD_KEY_COMMENT="davide@2025"
while read -r host; do
  echo "Cleaning $host..."
  ssh "root@$host" "sed -i '/$OLD_KEY_COMMENT/d' ~/.ssh/authorized_keys"
done < ~/.serverlist

Or if you want to be paranoid ( and you should ), remove the old key one server at a time and verify you can still SSH in with the new key after each removal.

Update Your SSH Config

Point your ~/.ssh/config at the new key. If you are using IdentitiesOnly ( and you should be ), update the IdentityFile line:

Host github.com
    IdentityFile ~/.ssh/id_ed25519_2026
    IdentitiesOnly yes

Host *.davideandreazzini.co.uk
    IdentityFile ~/.ssh/id_ed25519_2026
    IdentitiesOnly yes
    User root

Remove the old key file from your .ssh directory when you are confident everything works. And by remove I mean move it somewhere safe first, then delete it a week later when you have not needed it.

Automate It for Next Time

I wrote a script that does the whole rotation in one shot. It generates, deploys, verifies, and cleans up. I run it once a year now.

#!/bin/bash
# rotate-ssh-keys.sh
set -euo pipefail

YEAR=$(date +%Y)
NEW_KEY=~/.ssh/id_ed25519_${YEAR}
OLD_COMMENT=$(ssh-add -l | grep -v "${YEAR}" | head -1 | awk '{print $NF}' | tr -d '()')

# 1. Generate new key
ssh-keygen -t ed25519 -C "davide@${YEAR}" -f "$NEW_KEY" -N ""
ssh-add "$NEW_KEY"

# 2. Deploy to all servers
while read -r host; do
  ssh-copy-id -i "${NEW_KEY}.pub" "root@$host"
done < ~/.serverlist

# 3. Verify
echo "Verifying new key works on all servers..."
FAIL=0
while read -r host; do
  ssh -o BatchMode=yes -i "$NEW_KEY" "root@$host" 'echo OK' || FAIL=$((FAIL+1))
done < ~/.serverlist

if [ "$FAIL" -gt 0 ]; then
  echo "WARNING: $FAIL servers failed verification. Old key NOT removed."
  exit 1
fi

# 4. Remove old key
echo "Removing old key ($OLD_COMMENT) from servers..."
while read -r host; do
  ssh "root@$host" "sed -i '/${OLD_COMMENT}/d' ~/.ssh/authorized_keys"
done < ~/.serverlist

echo "Rotation complete. New key: $NEW_KEY"

Put it in a cron job if you want. I just run it manually once a year with a calendar reminder. Good enough.

Conclusion

SSH key rotation is not exciting, but it is important. The whole process takes maybe 15 minutes if you have fewer than 20 servers. Automate the deploy and verify steps, and you can do it in 5. Do not be like me and wait three years :)