Publishing Octopress over FTPES

2014-02-21

My webhost does not support ssh, the only secure transfer method that they provide is FTPES. I therefore had to put together my own Octopress deployment method. Since rake generate also copies unchanged files to public/ (and thus modifies their mtime stamps), I combine the following three steps to only upload changed files and delete stale files on the server:

Use checksum-based rsync locally to identify changed files
Use lftp to synchronize the content with the web server
Use openssl for secure transfer

None of this is difficult. But hopefully, reading this post saves you some time if you have to solve the same problem.

Rsync

rsync provides fast incremental file transfer. It can reliably detect file changes using checksums, so it is not necessary to rely on file size changes (as mentioned above, mtime stamps are useless).

rsync is used locally to synchronize public/ with a newly created directory called publish/

rsync --delete -crhv public/ publish/

This indirection over an additional directory assures that time stamps in publish/ are accurate, and can be used by lftp to identify changed files.

LFTP

LFTP is a powerful command line utility for file transfer. The Homebrew version comes with OpenSSL support activated by default.

The following command file mirror.lftp always activates SSL, opens a connection to the FTPES server and mirrors publish/ with the httpdocs/ directory on the server

set ftp:ssl-force on
set ftp:ssl-protect-data on
set ftp:use-feat off
set ssl:check-hostname on
set ssl:verify-certificate on

debug 1
open -u username ftp://server.com
mirror -R -e -vvv /path/to/publish httpdocs
ls httpdocs

Running lftp -f mirror.lftp executes the command file and reverse mirrors the directory.

OpenSSL

DISCLAIMER: I don’t know enough about OpenSSL, don’t rely on what follows for your security.

Since certificate verification is turned on in mirror.lftp, it is necessary to install the certificates to establish the chain of trust for the FTPES server. Never having done it before, I found this article useful.

It is possible to specify a certificate in lftp using ssl:ca-file, but I could not get this to work. I installed the certificates in the standard SSL base directory instead, which is /System/Library/OpenSSL on OS X.

Obtaining the Certificates

A neat way to get the necessary certificates (already in PEM format) is to use the built-in SSL client

openssl s_client -showcerts -starttls ftp -connect server.com:21

In my case, this command returns the certificates for the hosting company and the intermediary CA. The first certificate is copied to a textfile called server.pem (used in the last step for verification), the second is saved as rapidssl_ca.pem (RapidSSL is the intermediary CA in this example). Not included was the root certificate. The following two lines of the server reply show that the root is “GeoTrust Global CA” in my case.

...
  npTfWVW4fzU/rqm9Y6c6zrPluJhWYo5yd1zpugOfNssdrzFmTg==
-----END CERTIFICATE-----
 1 s:/C=US/O=GeoTrust, Inc./CN=RapidSSL CA
   i:/C=US/O=GeoTrust Inc./CN=GeoTrust Global CA
-----BEGIN CERTIFICATE-----
MIID1TCCAr2gAwIBAgIDAjbRMA0GCSqGSIb3DQEBBQUAMEIxCzAJBgNVBAYTAlVT
...

GeoTrust provides this certificate in PEM format, which I saved in a textfile called geotrust_global_ca.pem.

Certificate Verification

The next step is fingerprint verification

openssl x509 -noout -fingerprint -in geotrust_global_ca.pem

I compared the fingerprint of the certificate with the fingerprint given on GeoTrust’s webpage (over a different internet connection). I wonder if the company support would be willing to read the fingerprint over the phone…

Certificate Installation

As the root user, copy both CA certificates to the SSL base directory and make sure that they are world-readable. Because OpenSSL looks for the certificates using their hash, also create the respective symbolic links

ln -s rapidssl_ca.pem `openssl x509 -hash -noout -in rapidssl_ca.pem`.0 	
  ln -s geotrust_global_ca.pem `openssl x509 -hash -noout -in geotrust_global_ca.pem`.0

Finally, verify the hosting company certificate (as a regular user)

openssl verify -CApath /System/Library/OpenSSL/certs server.pem

to ensure that the certificate installation is successful.

Rakefile

Everything is tied together by adding this task to the Octopress Rakefile:

desc "Publish to server via FTPES"
task :publish do
  system "rsync --delete -crhv public/ publish/"
  system "lftp -f mirror.lftp"
end

Enjoy efficient publishing of your Octopress blog using rake generate && rake publish.

LearningBits