No Hassle Blog Automation

Forget servers and OS Updates.

Managing a blog can be a hassle. Operating system updates, blog software updates, and server security take up tons of time. Don’t forget about scaling your blog if you get popular. Inspired by the Rackspace DevOps post on their new blog format, I’ve setup my own version using Pelican instead of Octopress.

Resources

This tutorial will assume you have two systems to manage your blog.

  • Local Workstation
  • Remote Server

The local workstation will be used to manage your blog posts, as well as uploading the content. This system will need git and Pelican installed.

The remote server will be used to generate and upload the HTML for your blog. This system will need git, Pelican, Jenkins, and pyrax installed.

The Tools

Let’s take a look at the tools we will need to automate our blog:

Now I’m going to detail out the setup instructions for each tool.

git

We are going to use git to provide a method of version control for our blog posts. In addition, we will be pushing our code to GitHub.com. A full git tutorial is outside the scope of this post. If you need a crash course in git, checkout this awesome tutorial.

Installing git should be fairly simple. Instructions for various operating systems can be found here. Install this on your local workstation and remote server.

Pelican

Installing Pelican is simple assuming you have pip available on your system. I’ve found the following method best for installing both pip and pelican. If you don’t have access to the easy_install command, try finding the python-setuptools package in your repositories.

Do this for your local workstation, as well as your remote server.

easy_install pip
pip install pelican Markdown

Once you have this installed on your local workstation, you need to generate the necessary files using pelican-quickstart. You will need to run this in the git repository we clone from GitHub. I highly recommend the Pelican Quick Start Documentation to help you with the details of configuring Pelican.

Note: Do not generate your html on your local workstation. We are going to configure Jenkins to do this for us automatically on the remote server. No need to clutter our git repository with this output.

Jenkins

Installation

Jenkins will need to be installed on the remote server only. Installation instructions for Jenkins can be found here. Check the right side of the page for your operating system. I chose CentOS 6.3 for my Jenkins server.

After installation, be sure your service starts on boot and your firewall configuration has port 8080 open.

Security

I highly recommend enabling Security for your Jenkins instance. There is an existing page on this topic. I made use of the Unix user/group database option, as well as allowing Logged-in users can do anything.

Install Plugins

Go ahead and update the existing plugins on your system. In addition, install these additional plugins:

  • Jenkins GIT plugin
  • GitHub API Plugin
  • GitHub plugin
  • GitHub Authentication plugin

Restart Jenkins after these install successfully.

Configuring a New Job

Configure a new job as a free-style software project. Let’s update the following options on the configuration page:

  • Discard Old Builds - Choose your retention history.
  • GitHub project - Full link to the GitHub repository we create later.
  • Source Code Management - Git.
  • Repository URL - Specify the URL of this remote repository. This uses the same syntax as your git clone command.
  • Branches to build - I specified master in my configuration.
  • Build Triggers - Build when a change is pushed to GitHub.
  • Build - Add an Execute shell build step for the following commands:
/usr/bin/make html
/usr/bin/python ${WORKSPACE}/cf_pyrax.py container_name ${WORKSPACE}/output REGION path_to_creds

Put these in separate build steps. Be sure to save your settings.

Locate your hook URL

Login to Jenkins, and browse to Manage Jenkins > Configure System. Look for GitHub Web Hook towards the bottom of the page, and expand the help option on the right. This should provide you with a hook URL we’ll use later. It should look similar to the following:

http://servername:8080/github-webhook/

This should be all we need to do with Jenkins for now. Let’s move on to GitHub.

GitHub

GitHub will be the central repository for our blog’s code. It will also be the launch point for the rest of our automation.

  1. Create a public repository for your code. Use whatever name you want. I suggest you allow GitHub to create the README.md file for you automatically.
  2. Clone this repository to your local machine.
  3. As discussed in the Pelican section, use pelican-quickstart to setup the necessary files on your local workstation. Feel free to push your changes to your repository. Caution: Make sure nothing in your code contains sensitive information. Once added to a git repository, it is available forever, even if you delete the local copy.
  4. Edit the settings for your repository, and choose Service Hooks. Locate Jenkins GitHub Plugin in the list, and enter your Jenkins Hook URL. Don’t forget to activate the hook and Update settings.

pyrax

Installation instructions for the pyrax modules can be found here. We are relying on pip again for this installation. This should be installed on the remote server.

I’ve written a script to automatically empty a Cloud Files container and upload the static content Pelican generates with the make html command. The source for this script can be found on my GitHub.

This script takes a series of arguments via argparse. Here is the usage of this tool explained. Feel free to reference the Build step we added to Jenkins earlier for an example.

usage: cf_pyrax.py [-h] container folder region credentials

positional arguments:
  container
  folder
  region
  credentials

optional arguments:
  -h, --help   show this help message and exit

Rackspace Cloud Files

If you don’t have a Rackspace Cloud Account, you will need to sign up.

Once you have the account, make note of your username and apikey. Once you have access to your control panel, create a Cloud Files container, and publish it to the CDN. You will need to set a relatively low TTL for your container after publishing it to the CDN. Make sure you have the following information to add to your pyrax script:

  • Username
  • Apikey
  • Region
  • Container Name
  • Folder on your Jenkins server where Pelican outputs your static files.

Caution: Do not add this to your git repository with this sensitive information in the script. Your Apikey should be protected at all costs.

Putting it all Together

Final Steps

Now that all the pieces are in place, let’s make the necessary updates to our DNS settings so our blog is live. The following records will need to be created:

Note: If your DNS provider doesn’t support URL Redirects, you should point to a web server with the necessary redirect in place.

Daily Work Flow

Now, let’s define the work flow for updating your blog:

  1. Create your content in the necessary format and location in your git repository.
  2. Commit this new content or changes to your repository.
  3. Push these changes to GitHub.
  4. GitHub will notify Jenkins that changes have been made.
  5. Jenkins will pull the latest changes from GitHub.
  6. Jenkins will run the necessary Pelican command to generate the static content.
  7. Jenkins will run the cf_pyrax.py script to empty your Cloud Files container, and then upload the latest static content.
  8. Your updates are now live!

Final Notes

Good work! If everything went well, you now have a hassle free, automated blog. I recommend you read through the Pelican documentation, and play with themes for your blog. Feel free to reference my settings and file structure on my GitHub.