A walk-through of an SSL handshake
In my last post,
I walked through a complete TCP handshake which initiates, among other things,
browser/webserver interaction. Although TCP, and the internet itself,
had been around quite a while before HTTP was created, it was HTTP and the world-wide-web that
made the internet a household concept in the mid 90's. As HTTP was originally specified, though,
the TCP handshake should be immediately followed by an HTTP request - something of the form
GET /index.html HTTP/1.0 or similar. This meant that in its first incarnation,
all of the HTTP data would be visible to each hop between the client and the server.
Netscape revolutionized the world in 1995 by
commercializing the web browser, and one of the many innovations associated with that
commercialization was the introduction of a new "layer" between TCP and HTTP which they called
SSL. Before a single byte of HTTP could be transmitted, a secure cryptographic
context would be negotiated between the browser and the server, and all HTTP transmission would
be performed under that context. In this post, I pick up where I left off last time, walking
through the beginning of an SSL handshake as it takes place after the TCP handshake but before
any HTTP protocol data can be sent in an HTTPS connection. Before continuing, you may want to
of the TLS handshake if you're not familiar with the theory behind the TLS handshake. I'll focus
in this post on the client and server hello messages that begin any TLS negotiation.
A note on terminology: SSL has undergone 5 (soon to be six) revisions and one name change to the awkward "TLS" name (which,
for all practical purposes, nobody actually uses) and has been greatly expanded beyond its original
purpose as a secure layer between HTTP and TCP, but the fundamentals remain the same: before any
sensitive application is transmitted, SSL is responsible for performing a set of secure key
exchanges and ensuring that data is transmitted both encrypted and signed. To be technically
precise, I'll use the term
TLS rather than the more common
SSL in the
description below, since what I'm describing is not actually SSL, but TLS —
specifically, TLS 1.2, the latest version.
In figure 1, below, I've captured below the initial handshake between my Chrome browser and Wikipedia, which is configured to use up-to-date TLS configuration settings. I'll assume you're familiar with the TCP handshake (if not, review my previous post), and I'll jump right into an examination of the TLS handshake. Remember, all of this takes place immediately after the TCP handshake completes, and before the HTTP request for the index page is transmitted.
The first packet exchanged in any version of any SSL/TLS handshake is the
packet which signifies the client's wish to establish a secure context. I've omitted the IP and
TCP prologue below and jumped right to the payload which is a TLS handshake message.
16:41:51.180853 IP 10.32.2.6.55956 > text-lb.ulsfo.wikimedia.org.https: Flags [P.], seq 1:206, ack 1, win 4117, options [nop,nop,TS val 570453704 ecr 420531432], length 205 ... 0x0040: 1603 0100 c801 0000 c403 03ec 12dd 0x0050: 1764 a439 fd7e 8c85 46b8 4d1e a06e b3d7 0x0060: a051 f03c b817 470d 4c54 c5df 7200 001c 0x0070: eaea c02b c02f c02c c030 cca9 cca8 c013 0x0080: c014 009c 009d 002f 0035 000a 0100 007f 0x0090: dada 0000 ff01 0001 0000 0000 1600 1400 0x00a0: 0011 7777 772e 7769 6b69 7065 6469 612e 0x00b0: 6f72 6700 1700 0000 2300 0000 0d00 1400 0x00c0: 1204 0308 0404 0105 0308 0505 0108 0606 0x00d0: 0102 0100 0500 0501 0000 0000 0012 0000 0x00e0: 0010 000e 000c 0268 3208 6874 7470 2f31 0x00f0: 2e31 7550 0000 000b 0002 0100 000a 000a 0x0100: 0008 1a1a 001d 0017 0018 1a1a 0001 00
The apparently meaningless blob of bytes in figure 1 contains a complete TLS client hello message, which indicates to the server that the client wants to begin a TLS handshake. If the server responds with anything other than a legitimately formed TLS server hello message, the connection will immediately be aborted (and, in most cases, the browser will display an inexplicable error message). In figure 2, I've reformatted the client hello in pieces which I'll examine in more detail below.
0x0042: 16 Handshake message type 0x0043: 0301 Message version 0x0045: 00c8 content length 0x0047: 01 client hello message 0x0048: 0000c4 client hello length 0x004b: 0303 client hello protocol version 0x004d: ec12dd1764a439fd7e8c8546b84d1ea06eb3d7a051f03cb817470d4c54c5df72 random value 0x006d: 00 session ID length (would be followed by session ID if non-zero) 0x006e: 001c Cipher suites length 0x0070: eaea GREASE compatibility check 0x0072: c02b ECDHE/ECDSA/AES 128/GCM/SHA2 (RFC 5289) 0x0074: c02f ECDHE/RSA/AES 128/GCM/SHA2 (RFC 5289) 0x0076: c02c ECDHE/ECDSA/AES 128/GCM/SHA3 (RFC 5289) 0x0078: c030 ECDHE/RSA/AES 256/GCM/SHA3 (RFC 5289) 0x007a: cca9 ECDHE/ECDSA/CHACHA 20/POLY 1305/SHA2 (RFC 7905) 0x007c: cca8 ECDHE/RSA/CHACHA 20/POLY 1305/SHA2 (RFC 7905) 0x007e: c013 ECDHE/RSA/AES 128/CBC/SHA (RFC 4492) 0x0080: c014 ECDHE/RSA/AES 256/CBC/SHA (RFC 4492) 0x0082: 009c RSA/RSA/AES 128/GCM/SHA2 (RFC 5288) 0x0084: 009d RSA/RSA/AES 256/GCM/SHA2 (RFC 5288) 0x0086: 002f RSA/RSA/AES 128/CBC/SHA (RFC 5246) 0x0088: 0035 RSA/RSA/AES 256/CBC/SHA (RFC 5246) 0x008a: 000a RSA/RSA/3DES/CBC/SHA (RFC 5246) 0x0090: 01 number of compression methods to follow 0x0091: 00 the compression method "no compression" 0x0092: 007f length of extensions 0x0090: dada 0000 GREASE compatibility check 0x0094: ff01 0001 00 renegotiation info 0x0099: 0000 0016 0014 0000117777772e77696b6970656469612e6f7267 SNI 0x00b3: 0017 0000 Extended master secret 0x00b7: 0023 0000 Session ticket TLS 0x00bb: 000d 0014 0012 040308040401050308050501080606010201 Signature algorithms 0x00d3: 0005 0005 0100000000 Status request OCSP 0x00dc: 0012 0000 signed certificate timestamp 0x00e0: 0010 000e 000c 02683208687474702f312e31 ALPN: HTTP/1.1 0x00f2: 7550 0000 channel ID 0x00f6: 000b 0002 0100 Elliptic curve point formats uncompressed 0x00fc: 000a 000a 0008 1a1a001d00170018 Elliptic curves 0x010a: 1a1a 0001 00 GREASE compatibility check
Below, I'll examine each line of the captured handshake message show in figure 1 and describe what each piece is used for.
0x0040: 16 0301 00c8 01 0000c4 0303 | | | | | TLS Handshake len length version version hello
TLS is, like both IP and TCP, a "wrapper" protocol that serves to encapsulate actual data that's
of interest to the client and the server. This makes sense — TLS is responsible for
encrypting actual protocol data. This is taken one step farther by the actual protocol; the
handshake itself is wrapped inside the TLS record protocol. The first five bytes above are the
TLS record protocol header:
16 0301 00c8. 0x16 is the marker identifying what
follows - in this case, a handshake message. The TLS record protocol wraps four types of "sub-
messages": handshake, alert, change cipher spec, and application data. The goal of any TLS
handshake is to get to where application data (sufficiently encrypted) can be exchanged. The
next two bytes
0301 represent the version of the TLS record layer protocol —
in this case, 3.1.
Version numbers in TLS are a bit of a mess. The first version of SSL was version 2.0, identified as 0200, naturally. This was followed by version 3.0: 0300. So far, so good. However, the next version of SSL wasn't called SSL at all - it was renamed by the IETF (for no particular reason) to TLS, and given a version number of 1.0. However, for backwards compatibility, TLS version 1.0 is identified as 0301: SSL v3.1. This means that TLS 1.1 was actually identified as 0302, and TLS 1.2 as 0303. Notice above that the version is 0301, suggesting that Chrome is actually advertising TLS 1.0. In fact, that's not quite right - it's advertising version 1.0 of the record layer protocol but as you'll see, the handshake itself is version 1.2. This is necessary for backwards compatibility with very old servers.
The final two bytes of the record protocol are 0x00c8, which is decimal 200: the length of the remainder of the TLS segment. So now the receiver (the server) knows that what follows is 200 bytes that should be interpreted as an TLS handshake message. If you count up the bytes in the packet above, you'll see that it starts at 0x0046 (66) and ends at 0x10e (270): the rest of the packet is the payload of the wrapped TLS handshake packet. The first byte of the subsequent handshake message, then, is which type of handshake message. That's right - there are four types of TLS packets, but at least 10 types of handshake messages! ("At least 10?" you may be asking: stay tuned). This is where the client has first identified that what follows is a client hello message 0x01. This is itself followed by three bytes of length: 0000c4. It may seem strange (and I suppose it is, a little) that the wrapper protocol declares two bytes worth of length whereas the wrapped protocol declared three — you might be inclined to think that the high- order byte in this case would always be zero. Although this is, for all practical purposes, the case, it doesn't necessarily have to be. If, in the future, a very long client hello message were needed, it could be split across multiple TLS record layer messages — after all, the actual application payload will always span multiple TLS record layer messages.
Unsurprisingly, though, the length indicated here is 0xc4: 196 bytes, four less than the 200 declared by the record layer protocol (accounting for the four bytes already used up to declare the handshake type and the length). This is followed by another version declaration 0303 — TLS version 1.2. Although the record layer protocol can remain at version 1.0, the handshake must be recognized as 1.2, since the handshake itself will take advantage of TLS 1.2-specific semantics (it's worth noting that this actual client hello doesn't contain anything TLS 1.2 specific - it's valid and would be parsed correctly, other than the version number, by a TLS 1.0- only server. However, subsequent parts of the handshake do require TLS 1.2 semantics, and this version identifies the entire handshake).
0x0040: ec 12dd | random 0x0050: 1764 a439 fd7e 8c85 46b8 4d1e a06e b3d7 0x0060: a051 f03c b817 470d 4c54 c5df 72 00 | session ID
After the type, length and version comes the actual client hello parameterization, starting with a 32-byte random value which will be used for the remainder of the handshake to establish entropy and guard against replay attacks. Note that this random value is not secure against eavesdroppers - the point of the TLS handshake itself is to establish a cryptographic context which, of course, hasn't been established yet, so this random value is sent "in the clear". The random value is followed by a session ID — if the client is trying to re-establish a previously negotiated TLS session, this will be populated. In this case, it isn't, so the session ID is declared as empty by providing the single byte '0': the length of the session ID which follows.
0x0060: 001c |len| 0x0070: eaea c02b c02f c02c c030 cca9 cca8 c013 | cipher suites 0x0080: c014 009c 009d 002f 0035 000a 0100 | | compression
Probably the most important piece of information that the client is responsible for transmitting
during a client hello message is the list of supported cipher suites: specifically, which key-negotiation,
encryption and MAC algorithms the client understands. The next two bytes are
001c, indicating that there are 28 bytes of cipher suites available for use. Each is two bytes long (so
there are 14 cipher suites available). You might expect that the cipher suites would encode
a key exchange, followed by an encryption algorithm, followed by a MAC algorithm but this isn't
the case: TLS assigns a unique code to each possible triple of key exchange/encryption/MAC that
can be supported. The definitions for these codes are actually scattered all over the place
— although RFC 5246 which specifies TLS 1.2 defines some of them, other are defined by
supplemental RFCs that were introduced after the adoption of TLS 1.2. RFC 5246 doesn't even
define any elliptic-curve cipher suites! Interestingly, the first value in the list,
eaea isn't a cipher suite at all, but a failsafe check that is specific to Chrome.
Chrome deliberately advertises an invalid cipher suite in the first place to verify that the
server ignores it correctly; if it doesn't, then Chrome has the ability to report that the
server is not performing according to the specification. This is called
Extensions and Sustain Extensibility or
GREASE as a sort of stretched-thin
The TLS specification requires that the client follow the list of supported cipher suites with the list of supported compression methods available. When SSL was first specified, the designers expected it to take responsibility both for encrypting as well as for compression. This capability has been virtually unused and unimplemented (in fact, given the recent CRIME and BREACH TLS vulnerabilities, has been shown to be dangerous), but must be declared for a client hello message to be correct. Therefore, the next two bytes are 01 and 00: 1 byte of compression mechanisms, and one compression mechanism: 0, meaning "no compression".
According to RFC 2246, this is the end of the client hello message, but the designers of TLS knew that no protocol is set in stone; the next two bytes indicate how many of the subsequent bytes of the message are "extensions". Extensions, in turn, are type/length/data triples (type and length being mandatory, data being optional) whose function is defined elsewhere and which may or may not be understood by the receiver. I'll walk through each of the extensions present in this example handshake below — for the most part, as you'll see, client hello extensions tend to consist of the browser advertising additional capabilities which the server may or may not take advantage of later on. This does not illustrate all available TLS client hello extensions; there are actually dozens defined in various RFCs.
0x0090: dada 0000 ff01 0001 00 grease |renegotiate |
The first extension is another GREASE failsafe check; this is followed by the TLS renegotiation extension defined in RFC 5746. This extension is effectively always present and tells the server that the client is able to defend against the renegotiation flaw in the original TLS specification which was identified by Marsh Ray in 2009. If this extension is not reflected by the server, then the client will not attempt to renegotiate a TLS connection, since it can't be done safely without this capability.
0x0090: 0000 0016 00 1400 | SNI 0x00a0: 0011 7777 772e 7769 6b69 7065 6469 612e 0x00b0: 6f72 67
The most interesting of the extensions is the third, extension 0000: server name. The need for this came up in the early days of the web. It's less common now (but not unheard of), but a lot of web servers hosted multiple web domains under a single IP address. This presented a challenge for TLS: which domain should it negotiate a connection for? The server-name extension provides a way for the client to indicate which actual server it's trying to connect to. If you look at the data included in the extension, you'll see that it includes the ASCII-encoded name of the website I'm connecting to: www.wikipedia.org.
0x00b0: 0017 0000 0023 0000 000d 0014 00
| extended|session | signature algorithms
0x00c0: 1204 0308 0404 0105 0308 0505 0108 0606
0x00d0: 0102 01
Similar to the renegotiation info extension, the fourth extension, "extended master secret" guards against the "triple handshake" man-in-the-middle vulnerability that was identified in 2014, and will always be presented. The session ticket algorithm advertises that this client is capable of handling a server-side session ticket which simplifies session resumption; as you'll see below, the server doesn't support it, so this extension is ignored. This extension is followed by a list of signature algorithms: in the first version of SSL, the signature algorithms were hardcoded into the algorithm.
0x00d0: 0005 0005 0100000000 0012 0000 | OCSP request | signed certificate timestamp
The next extension is slightly different — rather than advertising a capability of the client,
it's requesting (if possible) one from the server: namely, OSCP stapling. The server is in
almost all cases required to present a certificate which includes a public key that will be used
either to perform a key negotiation or at minimum to sign one. That certificate must in turn
be signed by another key contained in another certificate which is either already trusted by
the client or signed by yet another key contained in yet another certificate which is trusted
ad infinitum. This series of certificates that vouch for one another is called a
chain. In theory, the client is responsible for keeping track of whether or not any
of these certificates has been found to be compromised (
Online Certificate Status
Protocol), but this extension requests that the server take on this responsibility.
As you'll see below, the server was in this case willing to do so.
This is followed by the signed certificate timestamp extension which is actually an extension to the previous extension - it requests that the OCSP status request returned by the server include log data that can be used to verify that the certificate has been sufficiently audited.
0x00e0: 0010 000e 000c 0268 3208 6874 7470 2f31 | ALPN 0x00f0: 2e31
The next extension is
Application Layer Protocol Negotiation: in this case, the
client (my browser) is just saying that it will speak HTTP/2 if available, but that it will fall
back to "plain old" HTTP/1.1 connection otherwise.
0x00f0: 7550 0000 000b 0002 0100 000a 000a |channelid|EC points |elliptic curves 0x0100: 0008 1a1a 001d 0017 0018 1a1a 0001 00 | grease
The next extension advertises the client's ability to support the channel ID TLS extension which is designed to cryptographically bind an authentication token (such as a cookie) to a specific "channel" (such as a TLS connection). It's an interesting, and very useful, capability, but not widely deployed as of yet. There are also a couple of extensions toward the end that describe how the server should use elliptic curve cryptography, should it choose to do so (which, as we'll see in a minute, it did).
Now, the server, if it accepts the connection at all, is responsible for selecting a cipher suite and providing all of the data that it needs to provide in order for the client to complete a secure key exchange under the selected cipher suite. With the TCP and IP headers omitted as before, the server hello message is shown below.
16:41:51.224463 IP text-lb.ulsfo.wikimedia.org.https > 10.32.2.6.55956: Flags [.], seq 1:1449, ack 206, win 59, options [nop,nop,TS val 420531442 ecr 570453704], length 1448 0x0040: 16 03 03 00 6a 02 000066 0303 0863b8 | ver | len | | len |ver |(random starts) type=handshake server hello
This begins just as the client hello packet did: advertising a TLS handshake packet (0x16), but now agreeing to version 3.3, followed by 006a = 106 bytes of content length. The first of these content bytes is 0x02, indicating that this is a server hello, rather than a client hello. This is followed by a length 0x000066 (102) and version 3.3. As part of the handshake's defense against replay attacks, the server must select its own 32 bytes of random data.
0x0050: 87e9 d4f7 26dd d2ce 267b 06c2 e853 cfdf 0x0060: 22f9 ed5f 9f1f 223d 7793 94c4 d820 b07d random | session ID
This is followed by a session ID. Remember that the session ID was empty in the client hello: if the client hello includes a session ID, that indicates that the client is trying to re-establish a previously negotiated TLS session. However, if the client doesn't provide one, the server will create a new one and return its value in the hello message, as it does here.
0x0070: 4f80 6a20 4eab b616 d330 5fa2 6969 e5c7 0x0080: f97a 01a9 6ff8 d27d 320b 417a 1c71 cca9 | cipher suite
After the session ID is the selected cipher suite. Here, the server has selected cipher suite ECDHE/ECDSA/ChaCha20/Poly-1305/SHA2 (defined in RFC 7905). This means that ECDHE (elliptic-curve diffie-hellman) will be used to negotiate a symmetric encryption key for the ChaCha20 encryption algorithm and ECDSA (elliptic-curve digital signature algorithm) will be used to validate the handshake. The SHA2 algorithm will be used as the secure hashing mechanism for the HMAC that signs each exchanged packet. Finally, a compression method must be selected from among those presented by the client. Unsurprisingly, the server here selected the one method that the client provided: 0 (no compression).
0x0090: 00 00 1e ff01 0001 00 000b 0004 0300 0102 | ext |renegotiate | ec curves point | | len | | format | 0x00a0: 0005 0000 0017 0000 0010 0005 0003 0268 | status | extended| ALPN | master secret 0x00b0: 32
Like the client hello, the server hello allows for extensions, and practically speaking, all client hellos and server hellos do include extensions. One particularly important server hello extension included here are ec_points_format 000b: since elliptic-curve cryptography was selected by the server for both handshake and digital signature, both sides must agree on a format for the exchange of the elliptic-curve parameters. For the most part, the server is just echoing back the client extensions - in essence, letting the client know that the extensions were understood, allowing it to either take advantage of extended functionality (such as in the case of the status request extension), disabling certain functions (for instance, if the server didn't recognize the renegotiation info extension) or potentially aborting the connection altogether, which might be the case if the extended master secret extension weren't recognized. It's worth noting in this case that the server did not echo the channel ID extension.
This is the end of the handshake message... but the packet continues! Since TCP is a stream-oriented protocol, TLS takes advantage of this and concatenates multiple handshake messages back-to-back. The very next byte begins a new TLS handshake message of length 0bd3: 3027. Digging further into the message, you see that it's of type 0x0b: certificate. Strictly speaking, the server doesn't have to present a certificate in order to perform a Diffie-Hellman key exchange (either elliptic curve or discrete logarithm). However, in order to guard against man-in-the-middle attacks, that Diffie-Hellman key exchange must also be signed, and in order for that signature to be verified, a public key must be exchanged and verified. What follows, then, is 3020 bytes os ASN.1 DER-formatted X.509 certificate. In this case, the certificate is issued to *.wikipedia.org, signed by GlobalSign, and contains a 256-bit elliptic curve public key. The certificate itself is a tad long, even as certificates go, because it's a multi-purpose (VERY multi-purpose) certificate, identifying many alternative names. Also, since the certificate was signed by an intermediate certificate, the full certificate chain is included.
The certificate itself, which begins at 0x00c0, is an ASN.1-encoded DER representation of an X.509 certificate. I won't cover the details here since I talked about this extensively in a previous post.
0x00b0: 16 0303 0bd3 0b00 0bcf 000b cc00 0760 0x00c0: 3082 075c 3082 0644 a003 0201 0202 0c10 0x00d0: e6fc 62b7 418a d500 5e45 b630 0d06 092a 0x00e0: 8648 86f7 0d01 010b 0500 3066 310b 3009 .... 0x0110: 032c f316 375d 67f1 a439 7949 a3c0 5dcc 0x0120: 55f9 2180 0ffb cee2 296a 5850 e9a6 d7eb 0x0130: 1c32 36b5 62a7 c1fa e6
Recall that my browser did ask for an OCSP status request; this is the next message in the handshake, message type 22. Remember I said that TLS defines "at least" 10 handshake messages? TLS extensions can define new handshake types in addition to new capabilities, and this one does. What follows is ASN.1 DER-encoded (just like a certificate) and signed certificate status; fortunately for this TLS connection, this one indicates that the status is "good".
0x0130: 16 0303 064f 1600 0x0140: 064b 0100 0647 3082 0643 0a01 00a0 8206 0x0150: 3c30 8206 3806 092b 0601 0505 0730 0101 0x0160: 0482 0629 3082 0625 3081 bfa2 1604 149c 0x0170: 4d00 9900 0e8b b001 8175 a1ba f0d0 25d7 0x0180: a01c 4718 0f32 3031 3730 3531 3531 3234 0x0190: 3130 335a 306f 306d 3045 3009 0605 2b0e ... 0x0160: 39b0 60f1 1717 c9fe 91f9 37e6 8d8f 505c 0x0170: 49dd 8ca0 70a6 24d5 ebd7 a3e2 5ccc 78c8 0x0180: 0893 1339 5760 1e08 c6bf b713 3985 cc16 0x0190: c35c 6a12 1680 700c 0748 4481 fe63 271d 0x01a0: db64 2756 dab8 eaa7 b3ed 4597 c8a2 4a58 0x01b0: 4dcc bee7 f0a7 db0a bbf7 b7fd 4a5b 8770 0x01c0: ccbb 9ec9 7e6c 8f77 be70 610e 4b18 3868 0x01d0: 111c 64ee adcc b350 9c3f a446 f3c4 373f 0x01e0: bfc0 fa4f 3f
The next step is the actual key exchange — in this case, since the server selected an ECDHE ciphersuite, this means that both sides must exchange points on an elliptic curve, as well as agreeing on an actual elliptic curve to use. This final handshake is complex and interesting, which will be the topic of my next post which will complete my byte-level walk-through of a TLS handshake.