How SSH Really Works

SSH (Secure Shell) is the protocol you use to log into a remote machine and run commands over an untrusted network. Its predecessor, Telnet, sent everything — including your password — as plaintext, so anyone on the path could read it. SSH exists to give you a connection with three guarantees: confidentiality (eavesdroppers see only ciphertext), integrity (tampered data is detected), and authentication (both sides prove who they are, so you aren't fooled by an imposter server).

The elegant part is how it achieves this, because it uses two different families of cryptography for two different jobs, and understanding the split is understanding SSH:

• Asymmetric (public-key) cryptography — a key pair where one key encrypts what only the other can decrypt. Slow, but it lets two strangers who share no secret establish trust. SSH uses it to set up the connection and authenticate the parties.
• Symmetric cryptography — a single shared secret key that both encrypts and decrypts. Fast, but both sides must already hold the same key. SSH uses it to encrypt the actual session data once set up.

The whole handshake is really a scheme to use slow asymmetric crypto briefly to agree on a fast symmetric key, then use that symmetric key for everything after.

Before the details, here is the whole sequence, so each later section has a place to hang. Everything from step 3 onward happens over an encrypted channel; the earlier steps are in the clear but designed to leak nothing useful.

The five conceptual phases: establish a TCP connection, agree on protocol versions, perform key exchange to derive a shared secret, verify the server's identity, then authenticate the client's identity. Only after all five does a shell appear.

SSH runs on top of TCP (the reliable, ordered transport from earlier in the course), so the very first thing is an ordinary TCP three-way handshake to port 22, the default SSH port. Nothing cryptographic has happened yet; this just gives both sides a reliable byte pipe.

Immediately after, each side sends a plaintext version banner — a single line like SSH-2.0-OpenSSH_9.6. This tells the other side which protocol version and implementation it's talking to, so they can agree on a compatible dialect (essentially everything today is SSH-2; the older SSH-1 had cryptographic weaknesses and is dead).

# What the version banner looks like on the wire:
$ nc example.com 22
SSH-2.0-OpenSSH_9.6

In the same early phase the two sides negotiate the algorithms they'll use — which key-exchange method, which symmetric cipher, which integrity (MAC) algorithm. Each sends a list in preference order and they pick the best mutually-supported option. This negotiation is why SSH can evolve: old algorithms are retired and new ones added without changing the protocol's shape.

Now the central magic. The client and server need a shared symmetric key to encrypt the session — but they're communicating over a network anyone can read, and they share no prior secret. Diffie-Hellman key exchange solves exactly this: it lets two parties derive an identical secret while an eavesdropper who sees every message they exchange still cannot compute it.

The intuition without the number theory: each side generates a private random value it never sends, and from it computes a public value it does send. Each then combines its own private value with the other's public value, and — by the math of Diffie-Hellman — both arrive at the same result. The eavesdropper sees only the two public values, from which deriving the shared secret is computationally infeasible. That shared result is hashed down into the session keys used for symmetric encryption and integrity for the rest of the connection.

Two important properties fall out of doing it this way:

• Forward secrecy. Because the DH private values are freshly generated per connection and thrown away after, recording the encrypted traffic and later stealing the server's long-term key still won't decrypt that past session — the session key was never transmitted and no longer exists.
• The session key is symmetric, so from here on both sides use fast bulk encryption. Diffie-Hellman was only needed to birth that key.

Diffie-Hellman gives you a shared secret with whoever is on the other end — but it does not tell you who that is. On its own, a man-in-the-middle (MITM) attacker could sit between you and the server, doing a separate DH exchange with each side and relaying traffic while reading all of it. Something must prove the server is the real server. That something is the host key.

Every SSH server has a long-term host key pair. During the handshake the server signs the key-exchange results with its host private key and sends the matching public key. The client verifies the signature with that public key, proving the server holds the corresponding private key — i.e., it really is the server that public key belongs to.

But that only shifts the question: how does the client know this host key is the legitimate one and not the attacker's? The answer is known_hosts — a file on the client that records the host key it saw the first time it connected to each server. This is Trust On First Use (TOFU).

The behavior you've seen firsthand:

• First connection — the client has no record, so it shows you the server's key fingerprint and asks you to confirm. Saying yes writes the key to known_hosts.
• Later connections — the presented host key is compared to the saved one. Match → connect silently. This is what makes the second login quiet.
• Mismatch — the host key differs from what's stored. SSH refuses with a loud REMOTE HOST IDENTIFICATION HAS CHANGED warning, because this is exactly what a MITM attack looks like (it's also what a legitimate server rebuild looks like — hence the warning, not silent failure).

The fingerprint the client shows is just a short hash of the host public key, so a human can verify it out-of-band:

# See the fingerprint of a server's host key before trusting it
ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub
# 256 SHA256:abc123... no comment (ED25519)

The server has now proven itself and an encrypted channel exists. The roles reverse: the client must prove who it is. There are two common methods, and the difference in how they work is the whole security story.

Password authentication is the simple one: the client sends the password over the already-encrypted channel, and the server checks it against its records. It's safe from eavesdropping (the channel is encrypted) but the secret itself travels to the server, it's only as strong as the password, and it's vulnerable to brute-force guessing attempts against the server.

Public-key authentication is stronger and the professional default, and it never sends the secret at all. The user has a key pair: a private key kept on their laptop (ideally passphrase-protected) and a public key copied into the server's ~/.ssh/authorized_keys file. Authentication is a challenge-response proving the client holds the private key without revealing it:

Walking the exchange: the client offers its public key; the server checks that key is listed in authorized_keys; the server sends a random challenge; the client signs that challenge with its private key and returns only the signature; the server verifies the signature using the stored public key. A valid signature is only possible if the client holds the matching private key — so possession is proven while the private key stays on the laptop, never crossing the network. Set it up with:

# Generate a key pair, then install the public half on the server
ssh-keygen -t ed25519
ssh-copy-id user@example.com   # appends your public key to authorized_keys

Once authenticated, everything runs inside the symmetrically-encrypted connection. A key design feature: a single SSH connection is not one stream but a multiplexer carrying many independent channels at once — your interactive shell is one channel, a file transfer another, a forwarded port another, all interleaved over the one authenticated, encrypted pipe.

The features this channel model unlocks are the reason SSH is far more than a remote terminal:

• Port forwarding (tunneling). SSH can carry other protocols' traffic inside its encrypted channel. Local forwarding exposes a remote service on your local machine (reach a database bound to the server's localhost as if it were yours); remote forwarding does the reverse. The tunneled traffic inherits SSH's encryption:

# Reach the remote host's private Postgres (5432) via a local port
ssh -L 5432:localhost:5432 user@example.com
# now `psql -h localhost` on your laptop hits the remote DB through the tunnel

• The SSH agent holds your decrypted private keys in memory so you don't retype the passphrase for every connection. Agent forwarding lets a server you've logged into use your local agent to authenticate onward to a third server — so the private key never needs to live on the intermediate host.
• Connection multiplexing reuses one already-authenticated TCP connection for subsequent sessions to the same host, skipping the handshake entirely and making repeat logins near-instant.

Throughout all of this, integrity is enforced per message: each packet carries a MAC (Message Authentication Code — a keyed checksum) so any tampering in transit is detected and the connection dropped. Confidentiality (encryption) and integrity (the MAC) travel together on every packet.

• SSH gives confidentiality, integrity, and mutual authentication over an untrusted network, replacing plaintext Telnet.
• It deliberately uses two crypto families: slow asymmetric crypto to set up and authenticate, fast symmetric crypto to encrypt the actual session — the handshake exists to birth a shared symmetric key.
• Diffie-Hellman key exchange lets client and server derive an identical secret over an open channel that an eavesdropper cannot reconstruct, and fresh per-connection values give forward secrecy.
• The server proves its identity with a host key; known_hosts remembers it on first use (TOFU), and a later mismatch triggers a MITM warning rather than silent trust.
• Client auth is either password (the secret crosses the encrypted channel) or public key (a challenge-response that proves possession of the private key without ever sending it) — public key is the strong default.
• One SSH connection multiplexes many channels: shell, file transfer, port forwarding, and agent forwarding, all inside the encrypted, MAC-protected pipe.

• [ ] I can explain why SSH uses asymmetric crypto for setup and symmetric crypto for the session.
• [ ] I can describe how Diffie-Hellman produces a shared secret an eavesdropper can't derive.
• [ ] I understand what a host key proves and how known_hosts / TOFU prevents MITM.
• [ ] I can explain the host-key-mismatch warning and why it isn't a silent failure.
• [ ] I can contrast password auth with the public-key challenge-response and say what crosses the wire.
• [ ] I can set up key-based auth with ssh-keygen and ssh-copy-id.
• [ ] I know that one connection carries multiple channels, and can set up a port-forward tunnel.