Extract certificates from Java Key Stores for use by CURL

I recently found myself working with a Tomcat-based web application that required its clients to present a certificate to authenticate themselves. Being Tomcat, the whole thing was put together using Java of course; if you wanted to make a call to the server, you had to include a reference to a Java Key Store (.jks) file. One of my coworkers made a good case for using CURL for automated testing — unfortunately, CURL doesn't understand the .jks format. Well, as it turns out, you can extract the key and certificate information from a Java Key Store for use by another client application, but the process is a bit involved, so I thought I'd document it here in case anybody else finds themselves in a similar situation.

Java Trust Stores and Key Stores

To motivate this example, I'll walk through the setup of a simple Java and JKS-based client and server that perform mutual SSL authentication — although I originally ran into this in the context of a Tomcat server, I think this is easier to see without all the extra Tomcat stuff. If you know what keystores are and already have something working, and you just want to see how to extract a certificate for use by CURL, feel free to skip down to the extraction part. Listing 1, below, is the simplest server application I can think of; it accepts a connection on port 1234 and echoes back every character it receives. It doesn't deal properly with multiple concurrent clients, or handle errors, or make itself configurable, or offer and security or authentication, but since the topic of this post is simply connectivity, it'll do well enough.

import java.net.Socket;
import java.net.ServerSocket;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.IOException;

public class Server {
  public static void main(String[] args) throws IOException {
    ServerSocket listen = new ServerSocket(1234);

    Socket connection = null;
    while ((connection = listen.accept()) != null)  {
      InputStream in = connection.getInputStream();
      OutputStream out = connection.getOutputStream();

      int c;
      while ((c = in.read()) != -1) {
        out.write(c);
      }
			out.close();
			in.close();
			connection.close();
    }
  }
}

Listing 1: Very simple Java "server"

If you compile and run this, it will sit and wait for connections on port 1234. You can test it without a special client by just using the telnet utility as shown in example 1:

$ telnet localhost 1234
Trying ::1...
Connected to localhost.
Escape character is '^]'.
abc
abc
def
def
^]
telnet> quit
Connection closed.

Example 1: test using telnet

Now, of course, this connection is plaintext. We want to encrypt it; that's what SSL is for. Java has had built-in support for SSL since JDK 1.3 — of course, being SSL, it requires a bit of setup. The code changes, shown in listing 2, are relatively straightforward, requiring only a change to the server socket constructor:

import java.net.Socket;
import java.net.ServerSocket;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.IOException;

import javax.net.ServerSocketFactory;
import javax.net.ssl.SSLServerSocketFactory;


public class SSLServer  {
  public static void main(String[] args) throws IOException {

    ServerSocketFactory fact = SSLServerSocketFactory.getDefault();
    ServerSocket listen = fact.createServerSocket(1234);


    Socket connection = null;
    while ((connection = listen.accept()) != null)  {
      InputStream in = connection.getInputStream();
      OutputStream out = connection.getOutputStream();

      int c;
      while ((c = in.read()) != -1) {
        out.write(c);
      }
			out.close();
			in.close();
			connection.close();
    }
  }
}

Listing 2: SSLServerSocket

Now, if you try to connect and test using telnet, you'll be kicked out as soon as you try to type a character, and the server will respond with an error message as shown in example 2:

$ java -classpath . SSLServer
Exception in thread "main" javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
	at sun.security.ssl.InputRecord.handleUnknownRecord(InputRecord.java:671)
	at sun.security.ssl.InputRecord.read(InputRecord.java:504)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:927)
	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
	at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:882)
	at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
	at sun.security.ssl.AppInputStream.read(AppInputStream.java:69)
	at SSLServer.main(SSLServer.java:20)

Example 2: unrecognized SSL message

The server was expecting an SSL handshake, but didn't get one; it aborts the connection immediately. Since it throws an uncaught exception in this case, the whole server application terminates (I told you it didn't handle errors well). If you had openssl installed, you could use the s_client subprogram to try to connect: openssl s_client -connect localhost:1234. However, since I'll need it later, I'll show you how to develop a Java equivalent; the SSL client in listing 3.

import java.net.Socket;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import javax.net.ssl.SSLSocketFactory;
import javax.net.SocketFactory;

public class SSLClient  {
  public static void main(String[] args) throws IOException {
    SocketFactory fact = SSLSocketFactory.getDefault();
    Socket conn = fact.createSocket("localhost", 1234);
    OutputStream out = conn.getOutputStream();
    InputStream in = conn.getInputStream();
    out.write("abc".getBytes());
    int c;
    while ((c = in.read()) != -1) {
      System.out.print((char) c);
    }
    in.close();
    out.close();
    conn.close();
  }
}

Listing 3: SSL Server Client

However, whether you use openssl s_client or the Java SSL client of listing 3, you'll still get a failure on the server when you try to connect to it, as shown in example 3:

$ java -classpath . SSLServer
Exception in thread "main" javax.net.ssl.SSLHandshakeException: no cipher suites in common
	at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
	at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1884)
	at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:276)
	at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:266)
	at sun.security.ssl.ServerHandshaker.chooseCipherSuite(ServerHandshaker.java:894)
	at sun.security.ssl.ServerHandshaker.clientHello(ServerHandshaker.java:622)
	at sun.security.ssl.ServerHandshaker.processMessage(ServerHandshaker.java:167)
	at sun.security.ssl.Handshaker.processLoop(Handshaker.java:878)
	at sun.security.ssl.Handshaker.process_record(Handshaker.java:814)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1016)
	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
	at sun.security.ssl.SSLSocketImpl.readDataRecord(SSLSocketImpl.java:882)
	at sun.security.ssl.AppInputStream.read(AppInputStream.java:102)
	at sun.security.ssl.AppInputStream.read(AppInputStream.java:69)
	at SSLServer.main(SSLServer.java:20)

Example 3: No cipher suites in common

What the Java runtime is trying to tell you, in a very oblique way, is that the server is misconfigured; in particular, it's not presenting a certificate to the client. The server has to identify itself — this is how SSL prevents man in the middle attacks. You can set up a test server by create a self-signed certificate using Java's keytool as shown in example 4:

$ keytool -genkeypair -keystore server.jks -storepass password -alias server -keypass password
What is your first and last name?
[Unknown]:  localhost
What is the name of your organizational unit?
[Unknown]:  Blog
What is the name of your organization?
[Unknown]:  commandlinefanatic
What is the name of your City or Locality?
[Unknown]:  Dallas
What is the name of your State or Province?
[Unknown]:  TX
What is the two-letter country code for this unit?
[Unknown]:  US
Is CN=Joshua Davies, OU=Blog, O=commandlinefanatic, L=Dallas, ST=TX, C=US correct?
[no]:  yes

Example 4: creating a new, self-signed certificate

This creates a new file named "server.jks" that contains one self-signed certificate entry. You can verify this from the command line as shown in example 5.

$ keytool -list -keystore server.jks
Enter keystore password:  

Keystore type: JKS
Keystore provider: SUN

Your keystore contains 1 entry

server, Jul 28, 2015, PrivateKeyEntry, 
Certificate fingerprint (SHA1): 1A:E9:6D:DA:A0:21:AD:A3:0A:27:A1:02:F0:7B:30:4E:ED:EB:29:51

Example 5: list contents of keystore

If you're following along, the answers to the prompts don't matter, except for the answer to the first question, "what is your first and last name?" Since this will be used as a server certificate, you must answer with "localhost", or the DNS name of the server that this will be accessed through. Although the SSLClient won't care, this will be important later, when I start using CURL to access this server.

Now, to instruct the server to present this certificate on connection, you provide the path to the keystore and the password on the command line:

java -Djavax.net.ssl.keyStore=server.jks -Djavax.net.ssl.keyStorePassword=password SSLServer

This goes a little better when you run the client — at least, you'll get a different error message as shown in example 6:

$ java -classpath . SSLClient
Exception in thread "main" javax.net.ssl.SSLHandshakeException: 
sun.security.validator.ValidatorException: PKIX path building failed: 
sun.security.provider.certpath.SunCertPathBuilderException: 
unable to find valid certification path to requested target
	at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
	at sun.security.ssl.SSLSocketImpl.fatal(SSLSocketImpl.java:1884)
	at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:276)
	at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:270)
	at sun.security.ssl.ClientHandshaker.serverCertificate(ClientHandshaker.java:1439)
	at sun.security.ssl.ClientHandshaker.processMessage(ClientHandshaker.java:209)
	at sun.security.ssl.Handshaker.processLoop(Handshaker.java:878)
	at sun.security.ssl.Handshaker.process_record(Handshaker.java:814)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1016)
	at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1312)
	at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:702)
	at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:122)
	at java.io.OutputStream.write(OutputStream.java:75)
	at SSLClient.main(SSLClient.java:14)

Example 6: Untrusted server certiticate

What the client is saying here is that it found the certificate, but it didn't trust it, since it wasn't signed by a trusted certificate authority. That shouldn't come as a surprise, since the certificate was self-signed — clearly not a certificate authority that the client could have trusted. You can force it to trust this self-signed certificate by overriding its "trust store" when you run the client:

$ java -Djavax.net.ssl.trustStore=server.jks -Djavax.net.ssl.trustStorePassword=password SSLClient

And now, the client and the server will connect with each other.

Note that this is still "wrong", although it does work. Giving the client access to the server's full key store defeats the purpose of creating the keystore in the first place. Normally, you'd export just the certificate and distribute it. I'll show you how to do that later.

At this point, the client trusts the server because it recognizes the signer of the server's certificate. The server trusts the client because, by default, SSL servers trust any client that tries to connect to it. You can change this behavior by requiring the client to present a certificate as well:

ServerSocket listen = fact.createServerSocket(1234);
((SSLServerSocket)listen).setNeedClientAuth(true);

Socket connection = null;

Now, if you try to run the client against the server again, you'll get a failure as you'd expect:

$ java -Djavax.net.ssl.trustStore=server.jks -Djavax.net.ssl.trustStorePasswd=password SSLClient
Exception in thread "main" javax.net.ssl.SSLHandshakeException: Received fatal alert: bad_certificate
	at sun.security.ssl.Alerts.getSSLException(Alerts.java:192)
	at sun.security.ssl.Alerts.getSSLException(Alerts.java:154)
	at sun.security.ssl.SSLSocketImpl.recvAlert(SSLSocketImpl.java:1959)
	at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:1077)
	at sun.security.ssl.SSLSocketImpl.waitForClose(SSLSocketImpl.java:1705)

Now, in order to get this to run, you'll need to generate another certificate and configure the client to present it and the server to trust it.

Technically, this would work if the client and the server both trusted and presented the same certificate (e.g. server.jks), but that makes it hard to see exactly what's going on — and definitely not something that would make sense in any sort of real-world scenario.

$ keytool -genkeypair -keystore client.jks -storepass password -alias client -keypass password
What is your first and last name?
  [Unknown]:  Joshua Davies
What is the name of your organizational unit?
  [Unknown]:  Blog
What is the name of your organization?
  [Unknown]:  commandlinefanatic
What is the name of your City or Locality?
  [Unknown]:  Dallas
What is the name of your State or Province?
  [Unknown]:  TX
What is the two-letter country code for this unit?
  [Unknown]:  US
Is CN=Joshua Davies, OU=Blog, O=commandlinefanatic, L=Dallas, ST=TX, C=US correct?
  [no]:  yes

$ java -Djavax.net.ssl.trustStore=client.jks -Djavax.net.ssl.trustStorePassword=password \
-Djavax.net.ssl.keyStore=server.jks -Djavax.net.ssl.keyStorePassword=password SSLServer

$ java -Djavax.net.ssl.keyStore=client.jks -Djavax.net.ssl.keyStorePassword=password \
-Djavax.net.ssl.trustStore=server.jks -Djavax.net.ssl.trustStorePassword=password SSLClient
abc

Notice how I've inverted the trust store and key store between the server and the client in this case.

Before I move on, I'll make one last minor change to the SSLServer in listing 4 — rather than having it echo back whatever it was presented, I'll have it act as a semi-valid HTTP server by responding to every request with a canned HTML page:

OutputStream out = connection.getOutputStream();


int c[] = new int[4];

while ((c[3] = in.read()) != -1)  {
  if (Arrays.equals(c, new int[] {'\r', '\n', '\r', '\n'})) {
    break;
  }
  for (int i = 0; i < 3; i++) {
    c[i] = c[i + 1];
  }
}

System.out.println("Sending response now");

out.write("HTTP/1.1 200 OK\r\n".getBytes());
out.write("Connection: Close\r\n".getBytes());
out.write("Content-Type: text/html\r\n".getBytes());
out.write("Content-Length: 47\r\n\r\n".getBytes());
out.write("<html><body><h1>Nothing here</h1></body></html>".getBytes());
out.close();

connection.close();

Listing 4: Return a valid HTTP response

$ curl -XGET https://localhost:1234/index.html
curl: (60) SSL certificate problem: self signed certificate
More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). If the default
 bundle file isn't adequate, you can specify an alternate file
 using the --cacert option.

$ keytool -exportcert -rfc -keystore server.jks -storepass password -alias server > server.pem

$ curl --cacert server.pem https://localhost:1234/index.html
curl: (35) error:14094412:SSL routines:ssl3_read_bytes:sslv3 alert bad certificate

$ curl --cacert server.pem --cert client.pem https://localhost:1234/index.html
curl: (58) unable to set private key file: 'client.pem' type PEM

$ keytool -importkeystore -srckeystore client.jks -destkeystore client.pfx -deststoretype PKCS12 
-srcalias client -deststorepass password -destkeypass password
Enter source keystore password:

$ openssl pkcs12 -in client.pfx -out client.p12 -nodes
Enter Import Password:
MAC verified OK

$ curl --cacert server.pem --cert client.p12 https://localhost:1234/index.html
<html><body><h1>Nothing here</h1></body></html>

    $ keytool -list -keystore <path/to/keystore>

    curl --cacert server.pem --cert client.p123 <url to publish>

I'm the author of the book "Implementing SSL/TLS Using Cryptography and PKI". Like the title says, this is a from-the-ground-up examination of the SSL protocol that provides security, integrity and privacy to most application-level internet protocols, most notably HTTP. I include the source code to a complete working SSL implementation, including the most popular cryptographic algorithms (DES, 3DES, RC4, AES, RSA, DSA, Diffie-Hellman, HMAC, MD5, SHA-1, SHA-256, and ECC), and show how they all fit together to provide transport-layer security.

Joshua Davies

Past Posts

• April 30, 2021: A Date Picker Control in Vanilla Javascript
• March 31, 2021: A Web Admin Console for Redis, Part Three
• January 27, 2021: A Web Admin Console for Redis, Part Two
• December 21, 2020: A Web Admin Console for Redis, Part One
• November 30, 2020: What is Procmail and why is it using up all my memory?
• September 30, 2020: Minimal Drag and Drop Support in Javascript
• August 31, 2020: Covariance and Contravariance in Generic Types
• July 31, 2020: How Spread Out Are the Floating Point Numbers?
• June 25, 2020: ERD Diagramming Tool, Part Three
• April 30, 2020: ERD Diagramming Tool, Part Two
• March 31, 2020: ERD Diagramming Tool, Part One
• February 28, 2020: MathJax and "t.setAttribute is not a function"
• December 30, 2019: Solving Systems of Equations with Python
• October 30, 2019: Linear Regression with and without numpy
• September 30, 2019: Reading a Parquet file outside of Spark
• August 30, 2019: UML Diagrams with MetaUML
• July 30, 2019: Clustering in Python
• June 25, 2019: A Walkthrough of a TLS 1.3 Handhsake
• May 31, 2019: A DataType Printer in Java
• April 30, 2019: A Simple HTTP Server in Java, Part 3 - Cookies and Keep Alives
• March 28, 2019: A Simple HTTP Server in Java, Part 2 - POST and SSL
• February 28, 2019: A Simple HTTP Server in Java
• January 29, 2019: Angular CLI Behind the Scenes, Part Two
• September 30, 2018: Angular CLI Behind the Scenes, Part One
• August 31, 2018: Into the MMIX MOR Instruction
• July 24, 2018: Undoing Percentage Changes in your Head
• June 30, 2018: Generating Langford Pairs in Scala
• May 25, 2018: Reflections on Three Years of Reading Knuth
• April 30, 2018: java.lang.NoSuchMethodError: org.junit.vintage. engine.descriptor.RunnerTestDescriptor. getAllDescendants
• March 30, 2018: An Excel Spreadsheet for the Academy Awards
• February 28, 2018: Git for Subversion Users
• January 31, 2018: The Evolution of AngularJS
• December 31, 2017: Numerical Integration in Python
• October 31, 2017: Gradle for Java Developers
• September 29, 2017: Reflections on another year of reading Knuth
• August 30, 2017: SSL OCSP Exchange
• July 27, 2017: A walk-through of an SSL certificate exchange
• June 30, 2017: A walk-through of an SSL key exchange
• May 31, 2017: A walk-through of the SSL handshake
• March 31, 2017: A walk-through of the TCP handshake
• February 28, 2017: The TLS Handshake at a High Level
• January 31, 2017: A Walk-through of a JWT Verification
• August 31, 2016: Reflections on a year of reading Knuth
• July 29, 2016: Matching a private key to a public key
• June 30, 2016: A Completely Dissected GZIP File
• May 31, 2016: Automatic Guitar Tablature Generator, Part 2
• April 28, 2016: Automatic Guitar Tablature Generator, Part 1
• March 31, 2016: Import an encrypted private key into a Java Key Store
• February 26, 2016: Import a private key into a Java Key Store
• January 31, 2016: Debian Linux on MacBook Pro
• December 29, 2015: Is Computer Science necessary or useful for programmers?
• November 30, 2015: Client certificate authentication vs. password authentication
• October 28, 2015: A Utility for Viewing Java Keystore Contents
• September 29, 2015: Debugging jQuery with Chrome's Developer Tools
• August 26, 2015: Getting Perl, MySQL and Apache to all work together on Mac OS/X
• July 30, 2015: Extract certificates from Java Key Stores for use by CURL
• June 29, 2015: Using the Chrome web developer tools, Part 9: The Console Tab
• May 28, 2015: Using the Chrome web developer tools, Part 8: The Audits Tab
• April 30, 2015: Using the Chrome web developer tools, Part 7: The Resources Tab
• March 30, 2015: Using the Chrome web developer tools, Part 6: The Memory Profiler Tab
• February 27, 2015: Using the Chrome web developer tools, Part 5: The CPU Profiler Tab
• January 31, 2015: Using the Chrome web developer tools, Part 4: The Timeline Tab
• December 31, 2014: Using the Chrome web developer tools, Part 3: The Sources Tab
• October 31, 2014: Using the Chrome web developer tools, Part 2: The Network Tab
• September 30, 2014: Using the Chrome web developer tools, Part 1: The Elements Tab
• August 11, 2014: Unable to find valid certification path to requested target
• June 30, 2014: Sort by a Hierarchy
• May 29, 2014: OpenSSL Tips and Tricks
• April 25, 2014: Heartbleed: What the Heck Happened
• February 28, 2014: Replace Microsoft Money with a Spreadsheet
• January 29, 2014: An Illustrated Guide to the BEAST Attack
• December 21, 2013: Where does GCC look to find its header files?
• October 24, 2013: Planning a Subversion import
• August 28, 2013: Compile and test an iOS app from the command line
• July 31, 2013: The Hidden Costs of Software Reuse
• June 26, 2013: Beware of mvn war:inplace
• May 29, 2013: Block Font Design Using Javascript
• April 4, 2013: Parsing a POM file using only SED
• February 22, 2013: Inside the PDF File Format
• December 31, 2012:How and why rotation matrices work
• November 27, 2012:Date Management in Java
• October 21, 2012: Installing Debian Without a Network
• August 14, 2012: My Review of Matt Neuburg's "Programming iOS 5"
• July 16, 2012: An example OAuth 1.0 Handshake and mini-library
• May 23, 2012: A Javascript one-liner to display cookie values
• April 27, 2012: How SSL Certificates Use Digital Signatures
• March 29, 2012: A breakdown of a GIF decoder
• February 15, 2012: The design and implementation of LZW (the GIF compression algorithm)
• January 16, 2012: Calculate the day of week of any date... in your head
• December 4, 2011: Understanding CRC32
• October 29, 2011: Efficient Huffman Decoding
• October 4, 2011: Extract a private key from a Gnu Keyring file
• September 5, 2011: From Make to Ant to Maven
• July 18, 2011: A bottom-up look at the Apache configuration file
• July 6, 2011: Fun with the HTML 5 Canvas Tag
• Jun 16, 2011: Pain and disfiguration upon all comment spammers
• May 31, 2011: Use of RSSI and Time-of-Flight Wireless Signal Characteristics for Location Tracking
• May 7, 2011: Implementing SSL
• Apr 24, 2011: Dissecting the GZIP format