HTTP Server with POST and SSL support

Last time, I walked through the development of a simple HTTP server in Java. Two major missing features in that server were the lack of support for HTTP POST as well as support for HTTPS. I'll rectify both in this post.

Post Bodies

The earliest use case for HTTP was to retrieve static documents from remote servers, hence the prominent support for the GET verb. GET specifies the document the client is looking for, but the transfer is considered one-way: the server isn't expected to change its own internal state based on the GET request. Although web application implementers found ways around this using URL parameters and cookies, the more complete approach is the POST verb which differs from GET in that it includes arbitrary data after the HTTP headers, just like an HTTP response does. This means that the client has to indicate to the server how much data it's sending so that the server knows when to stop reading. HTTP supports two ways to implement this: first, by explicitly declaring up-front the number of bytes in the POST body via the Content-Length header and second by prepending each "chunk" of data with its length (which is referred to as Chunked transfer encoding). The easier of the two to support is Content-Length, which I'll implement first.

Content-Length header

Recall from the last post that the class Request was responsible for parsing the command line and the headers. This doesn't vary between GET and POST; the difference between the two here is that the input can be effectively considered exhausted once the last header has been received in a GET request. I'll leave it up to the request handler to figure out what to do with the body rather than try to handle it in the request parser and instead provide a getBody invocation for the handler to get the "rest" of the request when it's ready:

public class Request  {
  ...
  public InputStream getBody() throws IOException {
    return new HttpInputStream(in, headers);
  }

Listing 1: getBody function

getBody, in turn, refers to an instance of a new class HttpInputStream which wraps the logic of parsing the headers to find the content length and indicate to the caller when the data has been completely consumed.

class HttpInputStream extends InputStream  {
  private Reader source;
  private int bytesRemaining;

  public HttpInputStream(Reader source, Map<String, String> headers) throws IOException  {
    this.source = source;

    try  {
      bytesRemaining = Integer.parseInt(headers.get("Content-Length"));
    } catch (NumberFormatException e)  {
      throw new IOException("Malformed or missing Content-Length header");
    }
  }

  public int read() throws IOException  {
    if (bytesRemaining == 0)  {
      return -1;
    } else  {
      bytesRemaining -= 1;
      return source.read();
    }
  }
}

Listing 2: HttpInputStream

Class HttpInputStream looks for the Content-Length header, keeps track of how many bytes have been read, and returns -1 when all have been consumed (per the java.io.InputStream specification). Note that just returning source.read() would not work here; source refers to an instance of Socket InputStream, which will never return -1 unless the underlying socket itself is closed which isn't what we want to do here.

The earliest version of HTTP did work that way, indicating the end of a request by closing the input side of the socket. This was changed to support HTTP KeepAlive semantics, where a single socket could be used to service multiple requests.

And that's it; the server now supports POST, at least when the POST body is explicitly stated in a request header. An example of how you might use it is shown in listing 3:

server.addHandler("POST", "/login", new Handler() {
  public void handle(Request request, Response response) throws IOException {
     StringBuffer buf = new StringBuffer();
     InputStream in = request.getBody();
     int c;
     while ((c = in.read()) != -1) {
       buf.append((char) c);
     }
     String[] components = buf.toString().split("&");
     Map<String, String> urlParameters = new HashMap<String, String>();
     for (String component : components) {
       String[] pieces = component.split("=");
       urlParameters.put(pieces[0], pieces[1]);
     }
     String html = "<body>Welcome, " + urlParameters.get("username") + "</body>";

     response.setResponseCode(200, "OK");
     response.addHeader("Content-Type", "text/html");
     response.addBody(html);
  }
});

Listing 3: Mock login handler

You could test this out with a curl command like:

$ curl -d "username=jdavies&password=secret" http://localhost:8080/login

Chunked transfer encoding

That's the easy case, of course - HTTP supports a more complex case where the client is streaming data of unknown size in which case the sender is responsible for breaking the data into chunks of known size and prepending each chunk with its length. The size of the pending chunk is provided as CRLF-delimited ASCII-formatted hexadecimal. So, for example, if the next chunk is 16,372 bytes long (0x3ff4), the chunk will be prepended by the byte sequence:

\r\n3ff4\r\n

Each of these 8 bytes must be stored and then discarded by the HttpInputStream, and 16,372 bytes provided to the caller. The 16,373rd byte must be \r, starting another chunk. The sender indicates completion by marking a chunk size of 0 (\r\n0\r\n, specifically). One minor irritant is that the first chunk size isn't preceded by CRLF — or rather, it is, but that CRLF is the indicator that the headers list is complete. Since it was already consumed by the header parser, I have to do a little bit of special handling to treat the very first chunk length indicator differently than the remaining ones. Listing 4 shows the changes to HttpInputStream needed to support chunked behavior: notice that the changes are entirely local to this class, and are completely transparent to the caller.

class HttpInputStream extends InputStream  {
  private Reader source;
  private int bytesRemaining;
  private boolean chunked = false;

  public HttpInputStream(Reader source, Map<String, String> headers) throws IOException  {
    this.source = source;

    String declaredContentLength = headers.get("Content-Length");
    if (declaredContentLength != null)  {
      try  {
        bytesRemaining = Integer.parseInt(declaredContentLength);
      } catch (NumberFormatException e)  {
        throw new IOException("Malformed or missing Content-Length header");
      }
    }  else if ("chunked".equals(headers.get("Transfer-Encoding")))  {
      chunked = true;
      bytesRemaining = parseChunkSize();
    }
  }

  private int parseChunkSize() throws IOException {
    int b;
    int chunkSize = 0;

    while ((b = source.read()) != '\r') {
      chunkSize = (chunkSize << 4) |
        ((b > '9') ?
          (b > 'F') ?
            (b - 'a' + 10) :
            (b - 'A' + 10) :
          (b - '0'));
    }
    // Consume the trailing '\n'
    if (source.read() != '\n')  {
      throw new IOException("Malformed chunked encoding");
    }

    return chunkSize;
  }

  public int read() throws IOException  {
    if (bytesRemaining == 0)  {
      if (!chunked) {
        return -1;
      } else  {
        // Read next chunk size; return -1 if 0 indicating end of stream
        // Read and discard extraneous \r\n
        if (source.read() != '\r')  {
          throw new IOException("Malformed chunked encoding");
        }
        if (source.read() != '\n')  {
          throw new IOException("Malformed chunked encoding");
        }
        bytesRemaining = parseChunkSize();

        if (bytesRemaining == 0)  {
          return -1;
        }
      } 
    }

    bytesRemaining -= 1;
    return source.read();
  }
}

Listing 4: HttpInputStream with support for chunked transfer encoding

The only potentially confusing part here is how I parse the chunk sizes. Suppose I get chunk size 3FF4: I'll first read in the byte '3', ascii code 51. I'll subtract that from '0' (ascii code 48) to get the numeric value 3, and store that in the chunkSize variable. So, as of now, the chunk size is 3. The next byte is 'F', ascii code 70. I'll subtract that from ascii code 'A' (65) and then add back 10 to get the correct value 15 for that character code. I'll then shift the bytes remaining over four bits (i.e. multiply by 16) and insert the new value 15 into the new low-order nybble. Table 1 summarizes what happens here, byte-by-byte.

Note that x << m | n is exactly equivalent to x * 2^m + n, but this implementation is just a bit faster.

parseChunkSize is effectively equivalent to Integer.parseInt(s, 16), but there's no real benefit in using the Java library here because I still have to gather up the characters in a StringBuffer to use it.

Otherwise, I take a peek at the headers in the constructor to figure out what sort of encoding I'm dealing with and then handle it in the reader. Note that I include a nod to exception handling by ensuring that the bytes \r and \n are included when they're expected, but a broken or malicious client could easily crash or at least hang this implementation.

It does pain me a bit to convert an InputStream to a BufferedReader and then back into an InputStream again — but I can't really see any compelling reason to take the time to implement the Reader interface in this case so I'll leave it as is.

SSL Support

One last change I'll make in this post is support for SSL. SSL was originally created to support HTTP, so unlike most other secure versions of protocols that use the same port for secure and non-secure connections, HTTP "cheats" by listening on a different port for secure connections. If a client connects on the "main" port (80, by default), the server expects the next byte to be part of an HTTP message. If the client connects on the secure port (443, by default), the server expects an SSL connections negotiation to take place before the HTTP message starts — otherwise, the presence of SSL is transparent to the client and the server, as it was designed to be.

Java includes support for secure sockets by default through the javax.net.ssl.SSLServerSocket class. You can modify the start code in the HTTP server to look like Listing 5:

public void start() throws IOException	{
    SSLServerSocketFactory factory = (SSLServerSocketFactory) SSLServerSocketFactory.getDefault();
    ServerSocket socket = factory.createServerSocket(securePort);

    Socket client;
    while ((client = socket.accept()) != null)  {

Listing 5: HTTP server with (non-working) SSL server socket

And it will compile and run - however, if you try to connect to it, you'll get an error message:

$ curl -v https://localhost:8443/hello
* STATE: INIT => CONNECT handle 0x6000579e0; line 1404 (connection #-5000)
* Added connection 0. The cache now contains 1 members
* STATE: CONNECT => WAITRESOLVE handle 0x6000579e0; line 1440 (connection #0)
*   Trying ::1...
* TCP_NODELAY set
* STATE: WAITRESOLVE => WAITCONNECT handle 0x6000579e0; line 1521 (connection #0)
* Connected to localhost (::1) port 8443 (#0)
* STATE: WAITCONNECT => SENDPROTOCONNECT handle 0x6000579e0; line 1573 (connection #0)
* Marked for [keep alive]: HTTP default
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
  CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* STATE: SENDPROTOCONNECT => PROTOCONNECT handle 0x6000579e0; line 1587 (connection #0)
* TLSv1.2 (IN), TLS header, Unknown (21):
* TLSv1.2 (IN), TLS alert, Server hello (2):
* error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
* Marked for [closure]: Failed HTTPS connection
* multi_done
* stopped the pause stream!
* Closing connection 0
* The cache now contains 0 members
* Expire cleared
curl: (35) error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure

Example 1: Failing SSL connection

The Java library puts a lot of work into making secure socket support as transparent as possible, but the SSL handshake itself requires that a server certificate be presented to the client, so most of the work in adding SSL support to an HTTP server in Java is centered around managing this certificate.

Before you can include it in the server code, then, you have to have the certificate to begin with. This certificate has to be "signed" by a certificate authority trusted by the client — certificate authorities like Verisign and Thawte are used for this purpose in commercial web sites, and can charge at least a few hundred dollars in exchange for their seal of approval. For testing purposes, though, you can generate a "self-signed" certificate and import it into your client. Java's keytool utility makes it easy to create a self-signed certificate and use it to protect a server socket.

$ keytool -genkey -keyalg RSA -alias httpserver -keystore httpserver.jks -storepass password
What is your first and last name?
  [Unknown]:  localhost
What is the name of your organizational unit?
  [Unknown]:
What is the name of your organization?
  [Unknown]:
What is the name of your City or Locality?
  [Unknown]:
What is the name of your State or Province?
  [Unknown]:
What is the two-letter country code for this unit?
  [Unknown]:
Is CN=localhost, OU=Unknown, O=Unknown, L=Unknown, ST=Unknown, C=Unknown correct?
  [no]:  yes

Enter key password for 
        (RETURN if same as keystore password):

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which 
is an industry standard format using "keytool -importkeystore -srckeystore httpserver.jks 
-destkeystore httpserver.jks -deststoretype pkcs12".

Example 2: self-signed certificate generation

This creates a new file named httpserver.jks which contains an encrypted keypair (for the purposes of this sample, you can ignore the warning about the format). You can start up the server as shown in example 3 to recognize this new key store/self-signed certificate (the Java runtime is "smart" enough to recognize that there's a single certificate in here and use it):

java \
	-Djavax.net.ssl.keyStore=./httpserver.jks \
  -Djavax.net.ssl.keyStorePass=password \
  -classpath . \
  com.jdavies.http.HttpServer

Example 3: Start up the server with the keystore

But you're still not quite out of the woods yet if you want to use it:

$ curl https://localhost:8443/hello
curl: (60) SSL certificate problem: self signed certificate
More details here: https://curl.haxx.se/docs/sslcerts.html

curl failed to verify the legitimacy of the server and therefore could not
establish a secure connection to it. To learn more about this situation and
how to fix it, please visit the web page mentioned above.

Example 4: CURL failing with unknown certificate warning

If you try this in a browser, you'll get a security warning urging you not to trust this page, but you can click through it and see the page. However, since my main use case here is mocking out REST APIs for testing purposes, I have to be able to trust the certificate. You can export the self- signed certificate and then instruct curl to trust it as shown in example 5.

$ keytool -export -keystore ./httpserver.jks -alias httpserver -file httpserver.cer -rfc
Enter keystore password:  password
Certificate stored in file <httpserver.cer>
$ curl --cacert ./httpserver.cer https://localhost:8443/hello?name=josh
<body>It works, josh</body>

Example 5: export and trust a certificate

Take note of the -rfc parameter at the end of the export command - if you leave this off, you'll get a certificate in DER (binary) format, which curl doesn't appear to accept (even though the documentation says it should, with the --cert-type parameter).

Next Steps

At this point, this HTTP server is pretty useful for its original intended purpose, which is to mock out external dependencies for testing purposes. Still, there are a couple of remaining deficiencies that are worth addressing: support for cookies, and support for the HTTP keep alive extension. I'll address both in my next post.

Add a comment:

Completely off-topic or spam comments will be removed at the discretion of the moderator.

You may preserve formatting (e.g. a code sample) by indenting with four spaces preceding the formatted line(s)

    headers.put(headerLine.substring(0, separator), headerLine.substring(separator + 1));

    headers.put(headerLine.substring(0, separator), headerLine.substring(separator + 2));

    String initialLine = in.readLine();
    log(initialLine);
    StringTokenizer tok = new StringTokenizer(initialLine);
    String[] components = new String[3];
    for (int i = 0; i < components.length; i++)  {
      // TODO support HTTP/1.0?
      if (tok.hasMoreTokens())  {
        components[i] = tok.nextToken();
      } else  {
        return false;
      }
    }

    method = components[0];
    fullUrl = components[1];

    // Consume headers
    while (true)  {
      String headerLine = in.readLine();
      log(headerLine);
      if (headerLine.length() == 0)  {
        break;
      }

      int separator = headerLine.indexOf(":");
      if (separator == -1)  {
        return false;
      }
      headers.put(headerLine.substring(0, separator),
        headerLine.substring(separator + 1));
    }
    
    

    // TODO should look for host header, Connection: Keep-Alive header, 
    // Content-Transfer-Encoding: chunked

    
    if (components[1].indexOf("?") == -1)  {
      path = components[1];
    } else  {
      path = components[1].substring(0, components[1].indexOf("?"));
      parseQueryParameters(components[1].substring(
        components[1].indexOf("?") + 1));
    }

    if ("/".e

I'm the author of the book "Implementing SSL/TLS Using Cryptography and PKI". Like the title says, this is a from-the-ground-up examination of the SSL protocol that provides security, integrity and privacy to most application-level internet protocols, most notably HTTP. I include the source code to a complete working SSL implementation, including the most popular cryptographic algorithms (DES, 3DES, RC4, AES, RSA, DSA, Diffie-Hellman, HMAC, MD5, SHA-1, SHA-256, and ECC), and show how they all fit together to provide transport-layer security.

Joshua Davies

Past Posts

• April 30, 2021: A Date Picker Control in Vanilla Javascript
• March 31, 2021: A Web Admin Console for Redis, Part Three
• January 27, 2021: A Web Admin Console for Redis, Part Two
• December 21, 2020: A Web Admin Console for Redis, Part One
• November 30, 2020: What is Procmail and why is it using up all my memory?
• September 30, 2020: Minimal Drag and Drop Support in Javascript
• August 31, 2020: Covariance and Contravariance in Generic Types
• July 31, 2020: How Spread Out Are the Floating Point Numbers?
• June 25, 2020: ERD Diagramming Tool, Part Three
• April 30, 2020: ERD Diagramming Tool, Part Two
• March 31, 2020: ERD Diagramming Tool, Part One
• February 28, 2020: MathJax and "t.setAttribute is not a function"
• December 30, 2019: Solving Systems of Equations with Python
• October 30, 2019: Linear Regression with and without numpy
• September 30, 2019: Reading a Parquet file outside of Spark
• August 30, 2019: UML Diagrams with MetaUML
• July 30, 2019: Clustering in Python
• June 25, 2019: A Walkthrough of a TLS 1.3 Handhsake
• May 31, 2019: A DataType Printer in Java
• April 30, 2019: A Simple HTTP Server in Java, Part 3 - Cookies and Keep Alives
• March 28, 2019: A Simple HTTP Server in Java, Part 2 - POST and SSL
• February 28, 2019: A Simple HTTP Server in Java
• January 29, 2019: Angular CLI Behind the Scenes, Part Two
• September 30, 2018: Angular CLI Behind the Scenes, Part One
• August 31, 2018: Into the MMIX MOR Instruction
• July 24, 2018: Undoing Percentage Changes in your Head
• June 30, 2018: Generating Langford Pairs in Scala
• May 25, 2018: Reflections on Three Years of Reading Knuth
• April 30, 2018: java.lang.NoSuchMethodError: org.junit.vintage. engine.descriptor.RunnerTestDescriptor. getAllDescendants
• March 30, 2018: An Excel Spreadsheet for the Academy Awards
• February 28, 2018: Git for Subversion Users
• January 31, 2018: The Evolution of AngularJS
• December 31, 2017: Numerical Integration in Python
• October 31, 2017: Gradle for Java Developers
• September 29, 2017: Reflections on another year of reading Knuth
• August 30, 2017: SSL OCSP Exchange
• July 27, 2017: A walk-through of an SSL certificate exchange
• June 30, 2017: A walk-through of an SSL key exchange
• May 31, 2017: A walk-through of the SSL handshake
• March 31, 2017: A walk-through of the TCP handshake
• February 28, 2017: The TLS Handshake at a High Level
• January 31, 2017: A Walk-through of a JWT Verification
• August 31, 2016: Reflections on a year of reading Knuth
• July 29, 2016: Matching a private key to a public key
• June 30, 2016: A Completely Dissected GZIP File
• May 31, 2016: Automatic Guitar Tablature Generator, Part 2
• April 28, 2016: Automatic Guitar Tablature Generator, Part 1
• March 31, 2016: Import an encrypted private key into a Java Key Store
• February 26, 2016: Import a private key into a Java Key Store
• January 31, 2016: Debian Linux on MacBook Pro
• December 29, 2015: Is Computer Science necessary or useful for programmers?
• November 30, 2015: Client certificate authentication vs. password authentication
• October 28, 2015: A Utility for Viewing Java Keystore Contents
• September 29, 2015: Debugging jQuery with Chrome's Developer Tools
• August 26, 2015: Getting Perl, MySQL and Apache to all work together on Mac OS/X
• July 30, 2015: Extract certificates from Java Key Stores for use by CURL
• June 29, 2015: Using the Chrome web developer tools, Part 9: The Console Tab
• May 28, 2015: Using the Chrome web developer tools, Part 8: The Audits Tab
• April 30, 2015: Using the Chrome web developer tools, Part 7: The Resources Tab
• March 30, 2015: Using the Chrome web developer tools, Part 6: The Memory Profiler Tab
• February 27, 2015: Using the Chrome web developer tools, Part 5: The CPU Profiler Tab
• January 31, 2015: Using the Chrome web developer tools, Part 4: The Timeline Tab
• December 31, 2014: Using the Chrome web developer tools, Part 3: The Sources Tab
• October 31, 2014: Using the Chrome web developer tools, Part 2: The Network Tab
• September 30, 2014: Using the Chrome web developer tools, Part 1: The Elements Tab
• August 11, 2014: Unable to find valid certification path to requested target
• June 30, 2014: Sort by a Hierarchy
• May 29, 2014: OpenSSL Tips and Tricks
• April 25, 2014: Heartbleed: What the Heck Happened
• February 28, 2014: Replace Microsoft Money with a Spreadsheet
• January 29, 2014: An Illustrated Guide to the BEAST Attack
• December 21, 2013: Where does GCC look to find its header files?
• October 24, 2013: Planning a Subversion import
• August 28, 2013: Compile and test an iOS app from the command line
• July 31, 2013: The Hidden Costs of Software Reuse
• June 26, 2013: Beware of mvn war:inplace
• May 29, 2013: Block Font Design Using Javascript
• April 4, 2013: Parsing a POM file using only SED
• February 22, 2013: Inside the PDF File Format
• December 31, 2012:How and why rotation matrices work
• November 27, 2012:Date Management in Java
• October 21, 2012: Installing Debian Without a Network
• August 14, 2012: My Review of Matt Neuburg's "Programming iOS 5"
• July 16, 2012: An example OAuth 1.0 Handshake and mini-library
• May 23, 2012: A Javascript one-liner to display cookie values
• April 27, 2012: How SSL Certificates Use Digital Signatures
• March 29, 2012: A breakdown of a GIF decoder
• February 15, 2012: The design and implementation of LZW (the GIF compression algorithm)
• January 16, 2012: Calculate the day of week of any date... in your head
• December 4, 2011: Understanding CRC32
• October 29, 2011: Efficient Huffman Decoding
• October 4, 2011: Extract a private key from a Gnu Keyring file
• September 5, 2011: From Make to Ant to Maven
• July 18, 2011: A bottom-up look at the Apache configuration file
• July 6, 2011: Fun with the HTML 5 Canvas Tag
• Jun 16, 2011: Pain and disfiguration upon all comment spammers
• May 31, 2011: Use of RSSI and Time-of-Flight Wireless Signal Characteristics for Location Tracking
• May 7, 2011: Implementing SSL
• Apr 24, 2011: Dissecting the GZIP format