← All Posts

The Handshake You Never See

TLS: Certificates, Key Exchange, and 50 Milliseconds of Cryptography

Reading time: ~17 minutes

You typed https://. Before a single byte of your request left your machine, your browser and the server performed a cryptographic negotiation involving prime numbers, certificate chains, and key exchange algorithms designed to defeat nation-state attackers. It took 50 milliseconds. You didn't notice.

That negotiation is TLS -- Transport Layer Security. It's the reason your bank password isn't floating across the internet in plain text. It's the reason the coffee shop WiFi can see that you're talking to reddit.com but can't read what you're posting. And it happens so fast, so silently, that most developers have never watched it happen.

The Problem: Talking Privately on a Public Network

TCP gives you a reliable stream of bytes between two machines (post 08). But TCP doesn't care what's in those bytes, and it doesn't care who's listening. Every router between you and the server sees your traffic. Every switch on the coffee shop network. Every government tap on the backbone fiber.

Before TLS, HTTP was plain text. Your password, your cookies, your session token -- every intermediary could read them. Worse, any intermediary could modify them. An attacker on your local network could inject JavaScript into an HTTP page. Your ISP could (and some did) insert ads into pages you loaded.

TLS solves three problems at once:

  1. Confidentiality. Nobody between you and the server can read the data.
  2. Integrity. Nobody can modify the data in transit without being detected.
  3. Authentication. You're actually talking to the server you think you're talking to, not an impostor.

That third one is the hardest. And it's where certificates come in.

The TLS 1.3 Handshake

TLS sits atop TCP. After the TCP three-way handshake completes, TLS adds its own handshake before any application data flows. In TLS 1.2, this was two round trips. TLS 1.3 cuts it to one.

Here's what actually flies across the wire:

TLS 1.3 handshake sequence diagram showing ClientHello, ServerHello, encrypted extensions, and the 1-RTT exchange

The client sends a ClientHello with its TLS version, supported cipher suites, a key share (ECDHE), the SNI hostname, and ALPN preferences. The server responds with a ServerHello picking the cipher suite and key share, then immediately sends its certificate, signature, and Finished message — all encrypted (hence 🔒). The client sends its own Finished, and application data flows. One round trip.

The critical difference from TLS 1.2: the client sends its key exchange parameters in the first message. It doesn't wait to be asked. The client guesses which key exchange algorithm the server will pick (usually ECDHE with X25519 or P-256) and sends a key share for that guess. If the guess is right -- and it almost always is -- the server can start encrypting immediately in its first response.

That's why everything after ServerHello is encrypted. The server's certificate, the signature proving it owns the private key, the Finished message -- all encrypted. In TLS 1.2, all of that was plaintext. Anyone watching could see which certificate the server presented.

One round trip. ClientHello, server responds with everything, client confirms. Done. Application data can flow.

TLS 1.3 also supports 0-RTT resumption -- if you've connected to this server before, you can send application data in your very first message. But 0-RTT data can be replayed by an attacker, so it's only safe for idempotent requests. GET, not POST. Most implementations disable it by default, and that's the right call.

What a Certificate Actually Contains

When the server sends its certificate, what's actually in that thing?

A TLS certificate is an X.509 document -- a data structure defined in ASN.1, encoded in DER, usually wrapped in PEM (base64 with -----BEGIN CERTIFICATE----- headers). If those acronyms feel like alphabet soup, don't worry. The format is a bureaucratic nightmare, but the content is straightforward.

A certificate contains:

The SANs field is what your browser actually checks. The old CN (Common Name) field is basically ignored now. If you've ever created a self-signed cert with CN=localhost and wondered why Chrome still rejected it -- that's why. Chrome wants a SAN.

The Chain of Trust

Your certificate isn't self-contained. It's the bottom link of a chain:

Root CA (pre-installed in your OS/browser) signed Intermediate CA signed Your certificate

The root CA is a certificate that your operating system ships with. macOS has about 150 of them. Windows has a similar number. Mozilla maintains its own list for Firefox. These are the certificate authorities that the world has collectively decided to trust -- companies like DigiCert, Sectigo, and the Internet Security Research Group (ISRG, which runs Let's Encrypt).

Root CAs almost never sign leaf certificates directly. They sign intermediate certificates, and those intermediates sign your cert. Why? Because if a root CA's private key were compromised, every certificate it ever signed would be suspect. Root keys live in hardware security modules in locked rooms. Intermediates are the ones doing the day-to-day signing, and if an intermediate is compromised, only that intermediate's certificates need to be revoked.

When your browser receives a certificate, it builds the chain: leaf -> intermediate -> root. If it can build a chain to a trusted root, and every signature checks out, and the certificate hasn't expired, and the hostname matches a SAN -- you get the padlock.

Think about what that means. Any of those ~150 root CAs can sign a certificate for any domain. DigiCert can sign a certificate for google.com. So can Sectigo. So can the government-operated CAs that some countries run. If a root CA goes rogue — or is quietly compelled by a government to issue fraudulent certificates — they can mint a valid cert for any site in the world. Your browser would accept it. The padlock would show. The MITM would be invisible.

This isn't hypothetical. In 2011, DigiNotar (a Dutch CA) was compromised and attackers issued fraudulent certificates for google.com, used to intercept Gmail traffic in Iran. DigiNotar was removed from all browser trust stores within weeks — the company went bankrupt. In 2015, CNNIC (China's government CA) delegated authority to an intermediate that issued unauthorized certificates for Google domains; Google and Mozilla distrusted new certificates from CNNIC's root. Symantec was forced to sell its entire CA business to DigiCert in 2017 after years of misissuing certificates — browsers fully distrusted Symantec-issued certs by late 2018. The entire TLS trust model rests on the assumption that 150 organizations across dozens of countries are all competent and honest, all the time. Certificate Transparency (below) exists because that assumption has been broken repeatedly.

Certificate chain of trust: Root CA to Intermediate CA to leaf certificate with verification path

Certificate Pinning

Some applications go further. Instead of trusting any certificate signed by any trusted CA, they pin specific certificates or public keys. If the server presents a certificate that doesn't match the pin, the connection is rejected -- even if the certificate is perfectly valid.

Mobile banking apps do this. It prevents an attacker who compromises a CA (or a government that coerces one) from minting a fake certificate for your bank. Google Chrome used to pin Google's own certificates -- that's how they caught TURKTRUST, a Turkish certificate authority, issuing unauthorized certificates in 2012.

Pinning is powerful but dangerous. If you pin a certificate and then lose the private key or forget to update the pin before rotating certs, your application is bricked. No connection possible. Google removed static pinning from Chrome in 2022 because the operational risk outweighed the security benefit for most sites. Certificate Transparency (more on this below) provides many of the same guarantees with less operational risk.

Key Exchange: The Paint-Mixing Problem

Here's the fundamental puzzle. You and the server need to agree on an encryption key. But you're communicating over a channel that anyone can observe. How do you agree on a secret when someone is watching everything you say?

This is the Diffie-Hellman problem, and the solution is one of the most elegant ideas in computer science.

The standard analogy uses paint. Imagine you're at a pub with a friend, and a stranger is watching your entire conversation. You and your friend want to agree on a secret color.

  1. You both agree on a starting color -- say, yellow. The stranger hears this. Fine.
  2. You pick a secret color (red) and mix it with yellow. You get orange. You slide the orange across the table. The stranger sees the orange.
  3. Your friend picks a secret color (blue) and mixes it with yellow. They get green. They slide the green across the table. The stranger sees the green.
  4. You take your friend's green and mix in your secret red. You get a brownish color.
  5. Your friend takes your orange and mixes in their secret blue. They get the same brownish color.

The stranger saw yellow, orange, and green. But mixing paint is a one-way function -- you can't un-mix paint to extract the secret color. The stranger can't recover red or blue from the mixed results.

In real TLS, the "paint" is mathematical operations on elliptic curves or large prime numbers. The one-way function is modular exponentiation or elliptic curve point multiplication. The shared secret becomes the symmetric encryption key.

TLS 1.3 uses ECDHE -- Elliptic Curve Diffie-Hellman Ephemeral. The "ephemeral" part is critical. Every single connection generates a brand-new key pair. The key share in the ClientHello? That's a freshly generated public key. The server's key share in ServerHello? Also freshly generated. They exist for this one connection and are then discarded.

Why RSA Key Exchange Died

TLS 1.2 allowed a different approach: the client generates a random secret, encrypts it with the server's RSA public key (from the certificate), and sends it over. Only the server can decrypt it. Sounds fine.

The problem is forward secrecy. If someone records all your encrypted TLS traffic today and then steals (or subpoenas) the server's private RSA key next year, they can decrypt every single recorded session. The private key is static -- it doesn't change between connections. One key compromise, and years of traffic are exposed.

With ECDHE, stealing the server's long-term key buys you nothing. Each connection used a unique ephemeral key that was discarded after use. The server's long-term key is only used to sign the handshake (proving identity), not to encrypt the key exchange.

TLS 1.3 removed RSA key exchange entirely. Not deprecated. Removed. If you want TLS 1.3, you get forward secrecy. No exceptions.

This wasn't theoretical paranoia. The NSA's PRISM program reportedly collected encrypted traffic at scale, banking on future key compromises. Forward secrecy makes that strategy worthless. Don't get too cocky — the NSA probably just moved to building out their quantum computing platforms.

ALPN: Negotiating HTTP/2 During the Handshake

Buried in the ClientHello is a small extension called ALPN -- Application-Layer Protocol Negotiation. The client sends a list of application protocols it supports (like h2 for HTTP/2 and http/1.1), and the server picks one in its response.

This is how your browser upgrades to HTTP/2. There's no separate negotiation, no upgrade header, no extra round trip. It's decided during the TLS handshake itself. By the time the handshake completes, both sides already know they're speaking HTTP/2.

It's elegant, but it means HTTP/2 is effectively TLS-only in practice. The HTTP/2 spec allows cleartext HTTP/2, but no browser implements it. Chrome, Firefox, Safari -- they all require TLS for h2. ALPN is why. I think this is a great thing — encryption by default is the right default, and having the browsers enforce it for free is cleaner than any standards-body pronouncement could be.

Let's Encrypt and the ACME Protocol

Before 2015, getting a TLS certificate meant paying a certificate authority $50-$300 per year, manually generating a CSR, copying it into a web form, waiting for validation, downloading the cert, and installing it on your server. I'm not exaggerating. It was exactly that painful.

Let's Encrypt changed everything. Free certificates, automated issuance, 90-day validity (to force automation), and an open protocol called ACME (Automated Certificate Management Environment, RFC 8555, I genuinely thought this was something to do with Wile E. Coyote at first) for the whole thing.

The most common challenge type is HTTP-01. It works like this:

  1. You ask Let's Encrypt for a certificate for example.com.
  2. Let's Encrypt says: "Put this random token at http://example.com/.well-known/acme-challenge/<token>."
  3. You put it there.
  4. Let's Encrypt fetches that URL from multiple network vantage points.
  5. If the token is there, you've proven you control the domain. Certificate issued.

Simple, but it has a limitation: it only works for specific hostnames. You can't get a wildcard certificate (*.example.com) with HTTP-01 because there's no single URL that proves you control all subdomains.

For wildcards, you need DNS-01. Instead of an HTTP token, you create a TXT record at _acme-challenge.example.com with a specific value. This proves you control the domain's DNS, which implies control over all subdomains. DNS-01 requires API access to your DNS provider -- which is why tools like certbot have plugins for Cloudflare, Route 53, Google Cloud DNS, and others.

At work I don't touch any of this directly. cert-manager in Kubernetes handles issuance and renewal, and external-dns manages the DNS-01 challenge records automatically. A certificate is a YAML manifest. For my personal domains I use Cloudflare Pages, which handles TLS entirely — I've never manually provisioned a cert for nazquadri.dev. Either way, I haven't thought about certificate renewal in over a year. That's the entire point.

Certificate Transparency Logs

Here's a scenario that kept certificate authorities up at night: what if a CA issues a certificate for google.com to someone who isn't Google? It happened. In 2011, a Dutch CA called DigiNotar was breached and the attacker minted certificates for Google, Yahoo, Mozilla, and others. Iranian users were targeted with man-in-the-middle attacks using those certificates.

Certificate Transparency (CT) is the solution. Every publicly trusted CA must submit every certificate it issues to multiple public, append-only logs before the certificate is considered valid. These logs are auditable by anyone. Google runs several. Cloudflare runs one. Sectigo runs one.

Your browser checks for a Signed Certificate Timestamp (SCT) proving the certificate was logged. Chrome requires it. If a CA issues a certificate for your domain, it shows up in the CT logs, and you can find it. Services like crt.sh let you search these logs by domain.

I check crt.sh for my domains every few months. Not because I expect to find rogue certificates -- but because I've found test certificates from staging environments that were accidentally submitted to production CAs. CT logs are a debugging tool as much as a security one.

Certificate Transparency flow: CA submits to CT Log, receives SCT, browser verifies, domain owner monitors

SNI: The Hostname Leak

One piece of the TLS handshake is sent in the clear: the server name.

Server Name Indication (SNI) is an extension in the ClientHello that tells the server which hostname the client wants. The server needs this to pick the right certificate -- a single IP address might host blog.example.com, api.example.com, and shop.example.com, each with its own certificate.

Before SNI existed (first specified in RFC 3546 in 2003, revised through RFC 4366, and now formalised in RFC 6066), each HTTPS site needed its own IP address. That was expensive when IPv4 addresses were already scarce. SNI fixed the problem, but created a privacy issue: anyone watching your traffic can see which hostname you're connecting to, even though the rest of the connection is encrypted.

Your ISP can see that you're visiting specific-subreddit.reddit.com. Your employer's firewall can see which hostname you're hitting even if the content is encrypted. In some countries, governments use SNI to censor specific websites.

Encrypted Client Hello (ECH, formerly called ESNI) is the fix. It encrypts the SNI field using a public key published in the server's DNS records (post 07 covers DNS and DoH, which ECH depends on). Cloudflare has been rolling out ECH support since 2023, and Firefox supports it. But it requires DNS-over-HTTPS to be effective -- otherwise, the DNS query itself leaks the hostname. The privacy chain is only as strong as its weakest link.

Common TLS Errors (And What They Actually Mean)

I bet like me you've stared at TLS error messages in bewilderment. Here's what's actually happening behind the four most common ones.

Certificate expired. The notAfter date has passed. Let's Encrypt certs last 90 days, which is the good case — if your renewal automation broke, at least you find out before a full year has quietly slipped by. Check certbot renew --dry-run or whatever your renewal tool is, and look at your monitoring — you should be paging someone at day 75, not getting the error when day 90 hits.

Hostname mismatch. The hostname you requested isn't in the certificate's SANs. You're hitting api.example.com but the cert only covers example.com and www.example.com. Check with openssl s_client.

Self-signed certificate. No chain to a trusted CA. Fine in dev, always a misconfiguration in prod.

Incomplete chain. This one is the reason I'm writing a whole paragraph about it. The server sent the leaf certificate but forgot the intermediate. Your browser can't build the chain to a root CA. The cruel twist is that some browsers cache intermediates they've seen from other sites, which means Chrome works and Firefox doesn't. Or Firefox works on your laptop and fails on your colleague's laptop. Or it works in the morning and fails after a browser restart. It's intermittent, it's browser-dependent, and it's the kind of bug where you start questioning your own sanity. I lost a weekend in 2019 to exactly this — the fix was one line in the nginx config to concatenate the intermediate into the cert bundle, but finding it took twenty hours because the error only showed up on some clients. If you're ever debugging a TLS issue that reproduces on one machine and not another, check this first.

Debugging with openssl s_client

When TLS goes wrong, openssl s_client is the X-ray machine. It shows you every step of the handshake:

# Connect and show the full handshake
openssl s_client -connect example.com:443 -servername example.com

# Check certificate expiry
echo | openssl s_client -connect example.com:443 -servername example.com 2>/dev/null \
  | openssl x509 -noout -dates

# Show the certificate chain
openssl s_client -connect example.com:443 -servername example.com -showcerts

# Test a specific TLS version
openssl s_client -connect example.com:443 -tls1_3

The output is verbose, but the critical bits are near the top:

depth=2 O = Internet Security Research Group, CN = ISRG Root X1
verify return:1
depth=1 C = US, O = Let's Encrypt, CN = R3
verify return:1
depth=0 CN = example.com
verify return:1
---
Certificate chain
 0 s:CN = example.com
   i:C = US, O = Let's Encrypt, CN = R3
 1 s:C = US, O = Let's Encrypt, CN = R3
   i:O = Internet Security Research Group, CN = ISRG Root X1
---
...
New, TLSv1.3, Cipher is TLS_AES_256_GCM_SHA384

That tells you: TLS 1.3, the cipher suite, the full certificate chain (leaf at depth 0, intermediate at depth 1, root at depth 2), and whether verification succeeded. If any of those verify return:1 lines say verify return:0, something is broken.

Annotated openssl s_client output showing certificate chain, protocol, verification status, and certificate details

Here's an alias worth having before a deploy fails and Slack is lighting up. The last thing you want is to be constructing openssl flags from memory at 3am:

# ~/.bashrc
alias certcheck='f(){ echo | openssl s_client -connect "$1":443 \
  -servername "$1" 2>/dev/null | openssl x509 -noout -dates -subject; }; f'

# Usage: certcheck example.com

The Whole Picture

Between you and the server, a remarkable sequence unfolds every time you type https://:

  1. DNS resolves the hostname to an IP (post 07)
  2. TCP opens a connection with a three-way handshake (post 08)
  3. TLS negotiates cipher suites, exchanges keys, verifies certificates, and establishes encrypted communication -- in a single round trip
  4. HTTP/2 is selected via ALPN during that same handshake
  5. Your encrypted request finally flows

All of it in under 100 milliseconds on a good connection. WiFi negotiated its own encryption to get your packets to the router (post 16 -- WPA3 uses a Diffie-Hellman variant called SAE for its own key exchange, a cousin of what TLS does). DNS might have used DoH or DoT for its own encryption. Layers of cryptography, each protecting a different hop.

The padlock icon in your browser is the tip of an iceberg that goes all the way down to elliptic curve mathematics. Most of the time, the iceberg does its job and you never think about it. But when it breaks -- when the chain is incomplete, when the cert expired at 3 AM, when the SNI doesn't match -- knowing the layers means you can diagnose instead of guess.

Every certificate error is a story about trust. Machines trusting other machines, all the way up to a root key in a vault that nobody touches. The whole system is fragile, improbable, and somehow it works billions of times a day.

Further Reading

I'm writing a book about what makes developers irreplaceable in the age of AI. Join the early access list →

Naz Quadri has mass-renewed certificates at 2 AM often enough to mass-distrust the concept of "set it and forget it." He blogs at nazquadri.dev. Rabbit holes all the way down 🐇🕳️.