X.509 Encrypted/Authenticated Sockets From a Ruby Client

On a system I'm working with, our Rails application makes a socket
connection to a Java application. The end-client has a requirement, rightly or wrongly, that
the socket is encrypted and authenticated using X.509 (the Java
application is running on a non-publicly accessible machine behind multiple firewalls and is the least-effective place for an attacker to hit, there are much better targets).

This article isn't to discuss the merits of the
situation. However, how to do an X.509 authenticated and encrypted
socket (from the client side of the equation) in Ruby isn't documented
that widely, so I thought I'd post the solution to try and help some
future poor sole who has to do it.

Assumptions

I'll assume that you have Ruby and OpenSSL installed. I'll also assume that you have Apache/OpenSSL installed and are fully capable of running Unix commands and administering Apache.

So, even though in my situation I'm going to be connecting to a Java application that side of it is out of my control (the Java side of the business is developing that), I am just going to set up Apache to use SSL and only accept connections from an X.509 authenticated client with a given set of identification details. This will mimic the Java-side of the equation adequately for the socket work.

OpenSSL Configuration

OpenSSL is particularly picky about having a correct configuration file. At the "signing your webserver certificate" step it should have enough information on the command line to cope with it (or you'd reasonably expect it to ask you for information as it does at every other step), but Noooo... So, the first thing we have to do is configure your openssl.conf file (on Ubuntu this is /etc/ssl/openssl.cnf, on Mac OS X it's /System/Library/OpenSSL/openssl.cnf).

The important items you need to change (at least from Leopard's defaults) are:

dir		    = /etc/ssl

certificate	= $dir/CA/MyCompany.crt

private_key	= $dir/CA/MyCompanyCA.key

You can also choose to change the defaults in the "req_distinguished_name" section so the defaults for certificates are created without you having to type them every time.

OK, let's create the necessary certificates and get Apache set up correctly.

Root CA Certificate

We're going to be using self-signed certificates (no need to an expensive certifying authority in this situation - it's effectively an inner-circle trust kind of arrangement), so we need to create a new Certifying Authority (CA) Root certificate. The first step, we need to make the folders to store our certificates/keys/etc:

# cd /etc

# mkdir -p /etc/ssl/CA /etc/ssl/certs /etc/ssl/csrs /etc/ssl/keys

In the real scenario the CA certificate will need to be install on both ends, but for the purposes of development we'll just put them all on the same machine.

OK, so let's generate the CA key:

# cd /etc/ssl/CA

# openssl genrsa -out MyCompanyCA.key 2048

Generating RSA private key, 2048 bit long modulus

Now we need to create a certificate request from this key:

# openssl req -new -key MyCompanyCA.key -out MyCompanyCA.csr

[keep pressing enter, no need to type anything in any field]

And finally we self-sign the certificate request to make our final CA certificate:

# openssl x509 -req -days 365 -in MyCompanyCA.csr -out MyCompanyCA.crt -signkey MyCompanyCA.key 

Signature ok

subject=/C=GB/ST=Herts/O=NextGen Development Ltd

Getting Private key

As you can see it's created the key using the default values in the OpenSSL configuration file. If you don't change them (which you can see I have) it wouldn't be great in production, but as it's just for our uses to play with X.509 it'll do fine.

Web Server Certificate

In the last step we created a certificate that lets us act as a free version of Verisign (although it won't work commercially because no browsers have our MyCompany CA certificate installed). Now we need to create a certificate for our server process (in this worked example Apache, in my client's final version the Java application) and sign it using our CA.

# cd /etc/ssl/keys

# openssl genrsa -out MyCompanyServer.key 2048

Generating RSA private key, 2048 bit long modulus

# cd ../csrs

# openssl req -new -key ../keys/MyCompanyServer.key -out MyCompanyServer.csr

[You can generally just keep pressing enter, but make sure you enter the hostname you'll use in Apache's ServerName directive for Common Name/FQDN - I used 'my.secure.service']

# cd ../certs

# openssl ca -in ../csrs/MyCompanyServer.csr -cert ../CA/MyCompanyCA.crt -keyfile ../CA/MyCompanyCA.key -out MyCompanyServer.crt

You'll need to answer 'y' twice (once to confirm the certificate and once to commit it to the database). I also had errors on the first time I tried this due to missing index.txt and serial files. If you get them, I did this to fix them:

# touch /etc/ssl/index.txt

# echo "01" > /etc/ssl/serial

At this point you have a signed "MyCompanyServer.crt" in your /etc/ssl/certs/ folder and you're ready to configure Apache. As I said, I'm doing this all locally, but if your server is remote you'll need to copy over the files /etc/ssl/CA/MyCompanyCA.crt, /etc/ssl/certs/MyCompanyServer.crt and /etc/ssl/keys/MyCompanyServer.key to the server.

Apache Configuration

Assuming you know how to create a VirtualHost in Apache, all you have to do is add the configuration options:

Listen 443

<virtualhost *:443>

  ServerName my.secure.service

  DocumentRoot /Users/andy/Sites/my.secure.service/

  SSLEngine on

  SSLCertificateFile /etc/ssl/certs/MyCompanyServer.crt

  SSLCertificateKeyFile /etc/ssl/keys/MyCompanyServer.key

  SSLCACertificateFile /etc/ssl/CA/MyCompanyCA.crt

</virtualhost>

You'll also need to add my.secure.service to your DNS or /etc/hosts file and then restart Apache. At this point I can go to https://my.secure.service/ using Firefox and see the index page I placed in /Users/andy/Sites/my.secure.service. I do get a security exception (as Firefox doesn't recognise My Company as a certifying authority - damn them!) which I have to approve, but then it works.

Now it's time to break it again... We're going to require that the client connects with a client certificate with certain details in it.

Add the following lines to the Apache VirtualHost:

SSLVerifyClient require

SSLVerifyDepth 2

<directory /Users/andy/Sites/my.secure.service>

  SSLRequireSSL

  SSLRequire %{SSL_CLIENT_S_DN_O} eq "NextGen Development Ltd" and \

	%{SSL_CLIENT_S_DN_OU} eq "Rails Development"

</directory>

The SSL_CLIENT_S_DN_O variable refers to the Organisation field you complete when creating the certificate (this time we will type one) and the SSL_CLIENT_S_DN_OU is the Organisational Unit field you'll complete.

Note: It's VERY important that you put the SSLRequireSSL and SSLRequire options within a Directory block. Otherwise an apachectl configtest will tell you "SSLRequireSSL not allowed here" and Apache will fail to start.

At this point it should all break and you won't be able to connect to the server any more. My Firefox gives me a message "Though the site seems valid, the browser was unable to establish a connection.". Bingo!

Client Certificate Creation

Although I'm doing this locally (and partially as an experiment) I'm going to generate the client certificate details in a folder from my home folder rather than in /etc/ssl/ to simulate that it's done on a different machine.

$ mkdir ~/RubyX509

$ cd ~/RubyX509

$ mkdir certs csrs keys

We'll now follow pretty much the same steps as above: Create the key, create the certificate signing request, sign the CSR and create the key. I won't go through it step by step again, I'll just paste the commands I used (I've snipped where it's asked questions I just pressed enter to and left in any where I actually had to type something):

$ cd ~/RubyX509

$ openssl genrsa -out keys/MyCompanyClient.key 2048

Generating RSA private key, 2048 bit long modulus

$ openssl req -new -key keys/MyCompanyClient.key -out csrs/MyCompanyClient.csr

Organisation Name []:NextGen Development Ltd

Organisational Unit Name []:Rails Development

Common Name (FQDN) []:Andy Jeffries

$ sudo openssl ca -in csrs/MyCompanyClient.csr -cert /etc/ssl/CA/MyCompanyCA.crt -keyfile /etc/ssl/CA/MyCompanyCA.key -out certs/MyCompanyClient.crt

If you like you can check this is all OK using:

$ openssl x509 -in certs/MyCompanyClient.crt -text

We now have the certificate/key we require for using Ruby. However, to be sure everything is working, we're going to install it in Firefox first to make sure the communication works before we get in to Ruby code. Firefox needs a certificate in a different format (such as PKCS#12) so we need to convert it before we can import it.

$ openssl pkcs12 -export -clcerts -in certs/MyCompanyClient.crt -inkey keys/MyCompanyClient.key -out certs/MyCompanyClient.p12

[Just press enter for the password prompts]

To import this in Firefox (for Mac OS X Leopard / Firefox 3, other O/S may vary) do the following: Firefox > Preferences > Advanced > Encryption > View Certificates > Your Certificates > Import > Choose File.

You'd think you'd be done now, but there is one last step. Firefox needs to know about the Root CA Certificate. To get this to work you need to copy /etc/ssl/CA/MyCompanyCA.crt to an accessible webserver (I assume you have something running) and try to open it using Firefox (it will then ask you whether you want to import/trust it - tick all the boxes and click OK).

So now you should be able to connect to my.secure.server using your client certificate and all works. You can verify that it's checking your O/OU by change the require line in your Virtual Host by one character and it should then give you a 403 Permission Denied error.

How To Do It From Ruby

Let's start off with some code that just simply connects to a normal HTTP socket, requests the root path and prints the response (before we start getting in to X.509 authenticated sockets, we need to make sure our socket-fu (not a real Gem) is all working fine).

#!/usr/bin/ruby



require 'socket'



socket = TCPSocket.new('my.normal.server', 80)



socket.puts("GET / HTTP/1.0")

socket.puts("")

while line = socket.gets

  p line

end

I'm sure this is all clear (this isn't an article on socket programming in Ruby), but just to make sure: we include the socket library, create a new TCPSocket pointing to the host 'my.normal.server' on port 80, send a standard HTTP GET request for the homepage (followed by a blank line so the server knows the header has ended) and then print each line returned. If I try this, replacing my.normal.server with www.googlecom, I get the source of Google's homepage. OK, time to make it SSL/X.509 authenticated.

The first step is to include the OpenSSL library, so change the require line to have both socket and openssl:

#!/usr/bin/ruby



require 'socket'

require 'openssl'



socket = TCPSocket.new('my.normal.server', 80)



socket.puts("GET / HTTP/1.0")

socket.puts("")

while line = socket.gets

  p line

end

Then we'll need to change the server name and port to point to our my.secure.server and port 443 (my client's Java application will run on a different port, but that's not relevant here):

#!/usr/bin/ruby



require 'socket'

require 'openssl'



socket = TCPSocket.new('my.secure.service', 443)



socket.puts("GET / HTTP/1.0")

socket.puts("")

while line = socket.gets

  p line

end

This is where things get tricky. We need to create an SSL Context and wrap it with our socket into an encrypted socket and use that instead of the original socket (making sure we call sync_close so both the original socket and the encrypted layer are closed together):

#!/usr/bin/ruby



require 'socket'

require 'openssl'



socket = TCPSocket.new('my.secure.service', 443)



ssl_context = OpenSSL::SSL::SSLContext.new()

ssl_socket = OpenSSL::SSL::SSLSocket.new(socket, ssl_context)

ssl_socket.sync_close = true

ssl_socket.connect



ssl_socket.puts("GET / HTTP/1.0")

ssl_socket.puts("")

while line = ssl_socket.gets

  p line

end

At this point we should have an encrypted connection, but if you run the script you'll get an error stating "handshake failure". Of course, we should expect this as we know that the server will only respond correctly if it recognises the client certificate. So, we now need to add our certificate and key file to the Ruby script (note, I've created this ruby script in my ~/RubyX509 folder so the certificate and key are in subfolders of the current location - this would need a slight refactor in a production environment so the keys are in a known/absolute place):

#!/usr/bin/ruby



require 'socket'

require 'openssl'



socket = TCPSocket.new('my.secure.service', 443)



ssl_context = OpenSSL::SSL::SSLContext.new()

ssl_context.cert = OpenSSL::X509::Certificate.new(File.open("certs/MyCompanyClient.crt"))

ssl_context.key = OpenSSL::PKey::RSA.new(File.open("keys/MyCompanyClient.key"))



ssl_socket = OpenSSL::SSL::SSLSocket.new(socket, ssl_context)

ssl_socket.sync_close = true

ssl_socket.connect



ssl_socket.puts("GET / HTTP/1.0")

ssl_socket.puts("")

while line = ssl_socket.gets

  p line

end

And bingo!!!

On my machine I happily get all the headers and my index file's source printed to the command line. We now have a working SSL encrypted socket to our server and uses X.509 authentication.

Other articles you may like:

• 25 Tips for Intermediate Git Users
• Debug-level Logging for a Single Rails Production Request
• DTrace and Ruby on Rails with Leopard/Snow Leopard
• Real World Uses of Rails Metal and Middleware
• Review of the jQuery UI 1.6 Book

The content and design of this site is ©copyright and all rights reserved Andy Jeffries (2001-2010), unless specified.
This site is written in Ruby on Rails, hosted on an Apache/Passenger server on Bytemark hardware.
Please give an attribution if you reference/use the content on your site.