Useful Links

ASN.1 Made Simple

HTTPS is based on PKI, which in turn is based on X.509 certificates. The standard encoding for these certificates is ASN.1; this tutorial walks through the concepts of ASN.1 and its encoding rules.

My writings, presentations, and publications.

  • March 31, 2017 A walkthrough of the TCP handshake

    In this post, I capture the tcpdump output of a TCP handshake and walk through each byte of it, what each means, and what it's for.

  • February 28, 2017 The TLS Handshake at a High Level

    I recently got a note from a reader of my book who asked if I wouldn't mind putting together a high-level description of the TLS handshake for a non-implementation audience.

  • January 31, 2017 A Walk-through of a JWT Verification

    A JWT-token is a Base64 encoded, digitally signed JSON structure. As it turns out, they're pretty easy to make sense of once you peel away the different parts. I'll do that in this post

  • August 31, 2016 Reflections on a year of reading Knuth

    For a long time, I've seen other programmers write in awed, hushed tones about the multi-volume series of books written by Donald Knuth titled "The Art of Computer Programming" (TAOCP). As somebody who genuinely enjoys reading computer books, I've been meaning to settle down and read this series for a long time. I finally finished volume 1 and I must say, although it took some effort to get through, it was well worth the time I spent on it.

  • July 29, 2016 Matching a private key to a public key

    If you do much work with SSL or SSH, you spend a lot of time wrangling certificates and public keys. As keys age and things get shuffled around, though, you may often find yourself (as do I) trying to figure out which private keys go with which public keys. I've put together a quick reference here for anybody (including myself) who's faced with the same problem.

  • June 30, 2016 A Completely Dissected GZIP File

    My very first post on this blog, 5 years ago, was a walk-through of the source code for a sample gunzip implementation. I've gotten quite a bit of feedback on it, mostly positive; it's still the most detailed post I've been able to put up here. Part of that write-up included bits and pieces of a gunzip session of an attached gzipped file, bit-for-bit. It occurred to me that a good companion piece would be a complete walk through of the gunzip process for the sample attachment.

  • May 31, 2016 Automatic Guitar Tablature Generator, Part 2

    Last month, I presented the code for a blank guitar tablature page generator. This becomes much more interesting if it can also render tablature based on user input. I won't present a full ASCII-art tablature parser here, but sort of a middle-of-the-road utility that takes as input triplets indicating the guitar string (E,A,D,G,B or e), the fret that that string should be played on and the duration of the note.

  • April 28, 2016 Automatic Guitar Tablature Generator, Part 1

    A project I've been kicking around for a while is working out how to translate ASCII-art tablature into proper tab, along with the corresponding music notes. As it turns out, it's mostly a math exercise, but sort of an interesting one.

  • March 31, 2016 Import an encrypted private key into a Java Key Store

    Last month, I talked about parsing a decrypted OpenSSL-formatted RSA key into a JKS-formatted Java Keystore. The utility that I presented in last month's post, though, has one slightly annoying limitation - you must manually decrypt the private key file before you can import it this way. As it turns out, the encryption that OpenSSL uses is well documented and well standardized, and it's possible to extend my KeyImport utility to decrypt the file in memory, granted (of course) that you have the decryption key handy.

  • February 26, 2016 Import a private key into a Java Key Store

    OpenSSL is the de-facto standard for SSL operations, including creation and management of assymetric encryption keys. However, Java has its own format. Most of the time, you can easily convert between the two, but I've run into problems related to private keys in the past — so much so that I finally broke down and wrote a little command-line utility to deal with the conversion.

  • January 31, 2016 Debian Linux on MacBook Pro

    I have an old (early 2011) MacBook Pro that's been getting slower and slower as OS/X keeps upgrading, so I finally decided, yet again, to repurpose the hardware as a Debian host.

  • December 29, 2015 Is Computer Science necessary or useful for programmers?

    UK Developer Mike Hadlow brought the wrath of the internet down upon him earlier this month when he suggested that not everybody can be taught to program. His post is an oblique swipe at the coding "bootcamps" that have been popping up in the last few years to address the problem of the programmer shortage that the Western world is purportedly facing. What about a college degree? Does a degree in computer science (or anything else) teach us anything useful about programming?

  • November 30, 2015 Client Certificate Authentication vs. Password Authentication

    Although password-based authentication is prevalent in most systems that require some sort of identity management, client certificates can be used to perform the same function with a different set of security tradeoffs.

  • October 28, 2015 A Utility for Viewing Java Keystore Contents

    I end up dealing with a lot of certificates and private keys, and since I still work primarily in Java, I necessarily end up dealing with quite a few Java Key Store files. You need them to get Tomcat up and running, you need them to do mutual SSL authentication, you need them to sign jar files... yet I always find the keytool that comes standard with Java a bit lacking. I finally got tired of repeating myself and broke down and wrote a utility to output exactly what I wanted.

  • September 29, 2015 Debugging jQuery with Chrome's Developer Tools

    The jQuery library has done a lot to improve the state of the art in Javascript development, but the way it works to separate code from presentation can make using Chrome's developer tools difficult to use when troubleshooting. There are ways to force it to work, but they vary from one jQuery version to the next.

  • August 26, 2015 Getting Perl, MySQL and Apache to all work together on Mac OS/X

    So, I've been running this blog for a few years now; over time I've been adding bits and pieces of dynamic functionality - not the least of which is a comment section. A little while ago, a reader pointed out that the comment section doesn't allow for formatted source code, which is something I wanted to fix. Before I did so, though, I wanted to set up a local test bed on my computer that I could verify my fixes against. The comments section (and other dynamic functionality) of my miniature content-management system is built on Perl and MySql, so to create a local instance, all I ought to need is Apache, Perl & MySql. As it turns out, Mac OS/X comes with Apache, MySql, and Perl all pre-installed... so it should be a simple matter to create a "dev" environment copy of my blog and get them all working together, right? Well, as it turned out, not quite.

  • July 30, 2015 Extract certificates from Java Key Stores for use by CURL

    I recently found myself working with a Tomcat-based web application that required its clients to present a certificate to authenticate themselves. Being Tomcat, the whole thing was put together using Java of course; if you wanted to make a call to the server, you had to include a reference to a Java Key Store (.jks) file. One of my coworkers made a good case for using CURL for automated testing — unfortunately, CURL doesn't understand the .jks format. Well, as it turns out, you can extract the key and certificate information from a Java Key Store for use by another client application, but the process is a bit involved, so I thought I'd document it here in case anybody else finds themselves in a similar situation.

  • June 29, 2015 Using the Chrome web developer tools, Part 9: The Console Tab

    In part 9 of my examination of the Chrome debugger tools, I'll conclude by talking about the Console tab that allows you to evaluate arbitrary Javascript expressions in the context of the page being viewed.

  • May 28, 2015 Using the Chrome web developer tools, Part 8: The Audits Tab

    In part 8 of my examination of the Chrome debugger tools, I'll talk about the Audits tab that can give you pointers on speeding up the client-side performance of your pages.

  • April 30, 2015 Using the Chrome web developer tools, Part 7: The Resources Tab

    In part 7 of my examination of the Chrome debugger tools, I'll talk about the resources tab that shows you how your web page or application makes use of client-side storage.

  • March 30, 2015 Using the Chrome web developer tools, Part 6: The CPU Profiler

    In this installment of the Chrome debugger tools examination, I'll cover the Heap profiler.

  • February 27, 2015 Using the Chrome web developer tools, Part 5: The CPU Profiler

    In this installment of the Chrome debugger tools examination, I'll cover the CPU profiler.

  • January 31, 2015: Using the Chrome web developer tools, Part 4: The Timeline Tab

    In part 4 of my multi-part series on the Chrome Debugger tools, I'll examine the timeline tab that helps performance profiling.

  • December 31, 2014: Using the Chrome web developer tools, Part 3: The Sources Tab

    In part 3 of my multi-part series on the Chrome Debugger tools, I'll examine the sources tab.

  • October 31, 2014: Using the Chrome web developer tools, Part 2: The Network Tab

    Welcome back to my multi-part series on the Chrome Debugger tools. Last time, I examined the first tab in the Chrome debugger tools, the Elements tab. Working left-to-right, the next tab is the Network tab, which I'll explore here. Whereas the Elements tab is useful for debugging and troubleshooting code that's not rendering properly once it has already been downloaded, the Network tab focuses on how the data that is rendered gets loaded into the browser in the first place.

  • September 30, 2014: Using the Chrome web developer tools, Part 1: The Elements Tab

    You kids and your fancy debuggers, these days. Back in my day, we had to debug Javascript in IE 4 using nothing but alert() popups. If we got stuck in a loop, IE would idiotically keep alert()-ing us until we killed the browser manually (which, incidentally, wasn't always that easy to do). The state of the art in Javascript/client-side debugging has come a long way since those days; I hardly ever have to code alert()s any more. These days, my weapon of choice is Chrome; it comes with a built-in debugging extension that, for the most part, can do everything I need it to do.

  • August 11, 2014: Unable to find valid certification path to requested target

    A few weeks ago, I upgraded my laptop. Due to a bug in the latest OS/X, I wasn't able to transfer all of my files from my old computer to the new one, but since everything I do is in Subversion anyway, I didn't anticipate a major issue just reinstalling everything I needed. When it came time to install Java, I installed the latest JDK (1.8). Thinking little of it, I went back to my normal work, ran Maven, and immediately got the following error: unable to find valid certification path to requested target

  • June 30, 2014: Sort by a Hierarchy

    I came across this problem a few weeks ago and when I searched for a solution, I couldn't find one. As it turns out, it's a pretty straightforward algorithms application once you see how to do it; I thought it was interesting enough to share.

  • May 29, 2014: OpenSSL Tips and Tricks

    You're probably at least peripherally familiar with OpenSSL as a library that provides SSL capability to internet servers and clients. OpenSSL, however, in addition to providing a library for integration, includes a useful command line tool that can be used for effectively every aspect of SSL/PKI administration. It's a bit under-documented though; this post doesn't aim to fully document it, but I've come across some fairly useful shortcuts that I thought I'd share with you, in "cookbook" style format.

  • April 25, 2014: Heartbleed: What the Heck Happened

    So, anybody who administers a web server — or anybody who uses a webserver administered by somebody — has by now heard of the catastrophic "11 on a scale of 1 to 10" HeartBleed vulnerability in OpenSSL. Other than being a very cool, very memorable name that spurs even non-technical people into action, what exactly is HeartBleed?

  • February 28, 2014: Replace Microsoft Money with a Spreadsheet

    I had been using Microsoft Money to track my personal finances for years, but after the version I had was just too old to run on my computer any more, I decided to sit down and figure out how to replace it with a simple spreadsheet.

  • January 29, 2014: An Illustrated Guide to the BEAST Attack

    The BEAST (Browser Exploit Against SSL/TLS) attack was published two years ago. Although it made quite a splash when it was first announced, it's hard to find technical details about how it works — in particular, to determine just how worried you should be. This post describes exactly how it works and how to determine how vulnerable you are to it.

  • December 21, 2013: Where does GCC look to find its header files?

    Partly by convention and partly by design, C programs are split into source files that describe the functionality of the program itself and header files that describe how to invoke that functionality from other source files. So, where does the compiler look to find these files? I was bitten by an edge case last week — it turns out there's more complexity surrounding header file searches than you may expect.

  • October 24, 2013: Planning a Subversion Import

    svn import is a bit of a landmine. It's not obvious from the command format, nor from its (almost nonexistent) documentation exactly what shape the imported repository will actually take once it's imported — and it's difficult to impossible to rectify a mistake once you've made one, since Subversion also doesn't offer a way to preview an import or delete an Subversion import is a bit of a landmine. I've gone through and documented a couple of things that can go wrong and how to recover from them as well as how to get the right import format the first time.

  • August 28, 2013: Compile and test an iOS app from the command line

    If you, like me, are an old-time Unix command-line fanatic now doing iOS development, you've probably wondered if you can build an iPhone app from scratch, entirely outside of XCode. After all, Mac OS/X is a *nix, and all the familiar tools — Make, cc, ld — are all there. So, can you build and compile completely outside of XCode? As it turns out, yes, you can, and there are actually some speed advantages to doing so.

  • July 31, 2013: The Hidden Costs of Software Reuse

    I have a confession to make. I don't automatically reuse code. I've been met with incredulous stares when, for example, a co-worker finds out I wrote my own drag-and-drop handling for my web page rather than just drop jquery in there and be done with it. But it seems to me that those same co-workers suffer from selective memory with regards to the hidden costs of software reuse.

  • June 26, 2013: Beware of mvn war:inplace

    So hopefully I can save somebody else four hours of frustrating, hair-tearing debugging... a few days ago I had to make a minor change to one of the webapps I maintain and redeploy it. I hadn't touched it in a while, but since I'm of course using Maven for build automation, I made the change, ran mvn clean install and pushed the new .war out to my Tomcat cluster. I went to verify the change and - nothing.

  • May 29, 2013: Block Font Design in Javascript

    One of my pet projects for a while has been a "font designer" in Javascript. I was working on a presentation that needed to show a countdown that could be rotated, sized and colored arbitrarily. Although there are probably better tools for the job, it's the sort of thing that interests me, so I've been poking at it for a while.

  • April 4, 2013: Parsing a Maven POM file using only SED

    My entry in this year's "stupid SED tricks" is a command-line run tool for Maven-based Java projects. Maven's a great choice for managing complex dependencies, especially because of the central repository that allows you to just declare, for example, that your project uses Apache Commons HTTPClient and let Maven resolve and download it for you. But you need its help to run the finished product. Can you do it entirely using command-line tools?

  • February 22, 2013: Inside the PDF File Format

    I'm sure you've come across a PDF file or two in your web browsing days. Google even gives you a PDF warning to let you know that you're about to download one so that you don't potentially end up wasting your time. But did you know that PDF is actually a textual format? Did you know that you can write a valid, well-formed PDF file using nothing but a text editor? This month, I'll show you how.

  • December 31, 2012: How and why rotation matrices work

    If you've done any sort of work with graphics, you're no doubt familiar with matrix operations — scaling, translating and rotating. In the matrix model of graphics operations, each operation is parameterized by a specific matrix, and the individual coordinates of the shapes to be manipulated are represented as column vectors. Scaling (resizing) and translating (moving) are easy enough to understand, but I've never seen the rotation matrix adequately explained. Most books about linear algebra try to frame rotation matrices around the context of changes of basis of orthogonal vectors (say that five times fast!) — I'm sure that's a useful explanation to somebody, somewhere, but I prefer a geometrical explanation.

  • November 27, 2012: Date Management in Java

    Considering how crucial date representation and computation is to so many of the sorts of commercial transactions that computers manage, you'd expect computers to be better at handling them. Contrary to reasonable expectation, though, date representation, parsing, and storage have been a thorn in the side of programmers since the dawn of computers. The standard Java JDK has quite a few routines that aim to simplify date management, but there are performance considerations and limitations to consider when using them.

  • October 21, 2012: Installing Debian Without a Network

    I've probably done a few dozen Debian installs in my life, and each one is unique. This time I had to do it with no network support at all — I'm used to not having wireless without a bit of struggle, but this time I didn't even have wired network.

  • August 14, 2012: My Review of Matt Neuburg's "Programming iOS 5"

    I review the book "Programming iOS 5" by Matt Neuburg.

  • July 16, 2012: An example OAuth 1.0 Handshake and mini-library

    I walk through an OAuth 1.0 handshake byte-by-byte, explaining the purpose of each element and developing a simple example code library along the way.

  • May 23, 2012: A Javascript one-liner to display cookie values

    A single line javascript function that will display the value of any cookie set in the current domain.

  • April 27, 2012: How SSL Certificates Use Digital Signatures

    Digital certificates are the underpinnings of the trust model that the modern internet relies on, but a lot of web users — even experienced ones — are still confused about how they work and what they defend against. This article walks through the signature verification process of a real SSL certificate.

  • March 29, 2012: A breakdown of a GIF decoder

    Last month, I detailed how the LZW (Lempel-Ziv Welch) compression algorithm works, along with a code sample. I hinted that LZW is the compression method used by the GIF image format, but I didn't go into specifics about exactly how GIF employs it. This month, I'll modify that code slightly and incorporate it into a working GIF decoder.

  • February 15, 2012: The design and implementation of LZW (the GIF compression algorithm)

    You probably know that image files on the internet are compressed before being transmitted - there are a few competing standards such as JPG, PNG and GIF. Although the graphics formats themselves — how the individual pixels are represented — are important, the details of the algorithms behind their compression are equally important. This article examines in detail the LZW algorithm that GIF employs, along with code samples.

  • January 16, 2012: Calculate the day of week of any date... in your head

    If you want to really impress people at cocktail parties — and by impress I mean freak them out with your uncanny, weird, rainman-like borderline Asperger's-syndrome abilities — compute the day of week of any date in your head. With a little bit of memorization and some basic computational arithmetic skill, you can figure out that, for example, Jul. 4 1776 fell on a Thursday, or that your friend's birthday will fall on a Sunday three years from now.

  • December 4, 2011: Understanding CRC32

    Finally finishing out my series on the GZIP algorithm I started back in April, this posting details the workings of the CRC-32 that provides integrity to gzipped archives. Starting from the concepts of polynomial division, it works up to a standards-compliant CRC32 implementation.

  • October 29, 2011: Efficient Huffman Decoding

    A while back, I posted an article examining the details of the GZIP compression algorithm. GZIP depends, among other things, on Huffman code compression. Although the easiest to understand (IMHO) implementation of Huffman codes and Huffman code inflation is tree-based, as detailed in my previous article, it's not very fast, nor is it very memory efficient. A better implementation is the table-based one that the open-souce gzip application uses.

  • October 4, 2011: Extract a private key from a Gnu Keyring file

    I recently found myself in the position of having to use an SSL certificate generated using the GNU Java keytool in an Apache server. Although there are a handful of documents out there that detail how to convert a certificate generated using Sun's keytool into an Apache-friendly format, I couldn't find anything about converting a GKR-formatted certificate. After much trial and error, I managed to figure it out; I've documented it here in case anybody else runs across the same problem.

  • September 5, 2011: From Make to Ant to Maven

    I've been using Maven now for quite awhile, having migrated off of Ant in favor of it for its superior dependency management. However, although I've managed to get the hang of it now, I initially found it pretty frustrating - Maven defines a lot of default behavior implicitly, and if you don't know what's going on under the hood, Maven has a nasty tendency to surprise you.

  • July 18, 2011: A bottom-up look at the Apache configuration file

    The Apache server is something that I've always just taken for granted. It's always there, it always works, and I never had to worry about it. To get a real sense of how the whole thing worked, I decided to strip the configuration file down to the smallest one that would possibly work, and add features incrementally to see what each one did.

  • July 6, 2011: Fun with the HTML 5 Canvas Tag

    I've been playing around with some of the new HTML 5 features lately; the one I'm most excited about is the HTML 5 "canvas" tag that lets you draw arbitrary shapes into a web page.

  • Jun 16, 2011: Pain and disfiguration upon all comment spammers

    Well, it's been two weeks since I've updated this blog, but not for lack of trying. I've been fighting comment spammers. It remains to be seen who's won this round.

  • May 31, 2011: Use of RSSI and Time-of-Flight Wireless Signal Characteristics for Location Tracking
  • A write-up of my Master's thesis was accepted to the 4th international conference on pervasive technologies related to assistive environments - I'll share more when Dr. Athitsos gets back, since he's the one who presented it.

  • May 7, 2011: Implementing SSL
  • A presentation I recently gave at the University of Texas at Arlington on SSL/TLS and real-world cryptography in action. This covers, at a very high level, the same material as my book.

  • Apr 24, 2011: Dissecting the GZIP format
  • An article I wrote about the GZIP format and the DEFLATE algorithm it relies on.

My Book

I'm the author of the book "Implementing SSL/TLS Using Cryptography and PKI". Like the title says, this is a from-the-ground-up examination of the SSL protocol that provides security, integrity and privacy to most application-level internet protocols, most notably HTTP. I include the source code to a complete working SSL implementation, including the most popular cryptographic algorithms (DES, 3DES, RC4, AES, RSA, DSA, Diffie-Hellman, HMAC, MD5, SHA-1, SHA-256, and ECC), and show how they all fit together to provide transport-layer security.

My Picture

Joshua Davies

Past Posts