Incite Code

Blog of a Rails Developer

Migrated to Octopress

I spent some time earlier this Holiday Season migrating this blog to octopress. It’s something that I’d considered doing a while back, but hadn’t had the time to do. The motivating factor? I didn’t want to pay to renew a worpress.com annual subscription, when I haven’t blogged in years – not when I can host free using Octopress and heroku anyway. Ironically of course, migrating to a new platform means I might write a few more posts now that this blog is at the forefront of my mind.

It’s long overdue, and it’s probably about time I wrote something technical about Chef since I’ve spent the last couple of years doing spending more time doing system administration and automation with Chef, than I have doing Ruby on Rails work.

Stay tuned for more posts on chef or Octopress customisation. For now: Happy Holidays all.

Writing Your Own Ruby Gems

I’ve recently converted several ruby on rails plugins I wrote from the old rails plugin style to be packaged as rubygems (gvis and custom change messages), which is a much preferred format these days, as tools such as bundler make package dependency a breeze. Here I’ll cover some of the best practices that I’ve come across, a few personal preferences for managing things, and how to setup a template for making writing gems the way you like dead simple.

While starting this process I tried out several gem management solutions, starting with jeweller, which I found to have far too much extra code polluting rake tasks, and tried to manage things for me a little more than I’d like. I did find it’s default readme to be perfect though for managing an open source gem on github with sections on how to contribute via forking. Bundler also provides a mechanism for creating a skeleton gem via

 bundle gem <gemname>

I quite liked the default gem layout bundler provided, along with the nice options for switching frameworks such as testing. Since bundler is a lot newer, it’s format was much more up to date with current trends in the way gems are written too. I ended up making small tweaks to the skeleton generated here, like using a readme based on jeweller’s though largely sticking to the bundler format opting for minitest, and rcov. After figuring out a nice setup, some of the best practices, and ways of managing things I can recommend are:

Keep your ruby source in the lib directory. The default require for a gem is lib/my_gem.rb (with a gem named my_gem). If your code base is big enough to require namespacing, keep a folder for each namespace, and require the given files from lib/my_gem.rb That way nothing quirky needs to be done to require your gem, and you can still organise your source code in a nice fashion.

Bundler for gem dependencies. The new kid on the block, bundler is a fantastic way to manage gem dependencies. I won’t cover how to use bundler here, though rather than keep your list of dependencies in both the gemspec and the Gemfile, instead you can simply get the Gemfile to refer to the gemspec file with one simple line of “gemspec” and bundler will handle setting up the default group for runtime dependencies, and the development group for development dependencies. I’ve opted to ignore the Gemfile.lock for gem development (though you should definitely check it in for rails application development) because you’re not testing code for a known production environment here, so you don’t need to lock your gem versions to what you might have deployed. With gem development it’s much more important to frequently update the gems you depend on, and ensure your code is compatible (by running tests), as other developers installing your gem will likely have the latest version of your dependencies (especially if they didn’t have them previously installed).

Define a constant for the gem version. I’m a fan of defining a constant MyGem::VERSION = “x.x.x” and referring to that constant in the gemspec. I find this much nicer, as it makes it much easier for other developers using your gem to find out the version they’re using, and introspect this at runtime if they’re integrating with it. As a nice bonus, whenever you bump the gem version, only lib/my_gem/version.rb need be changed, rather than change the gemspec file.

I was also keen to try out the new minitest framework, so I used that as a default development dependency and setup a rake task for running the tests (and rcov to keep tabs on my test coverage). I found minitest quite nice to work with (and it replaces testunit in ruby 1.9), though test framework is one of the places where more so than anywhere else, it’s each to their own preferences. The most important thing is that you’re writing tests.

That also brings me to a couple of important points for making a sucessful gem that others developers will want to use: write tests and documentation. Even if you don’t write perfect yard formatted inline documentation throughout the whole of your source code (though let’s be honest, writing inline comments throughout most of it, is really helping you more than anyone else) a decent readme file is incredibly useful to anyone encountering your project for the first time. Including a summary of what problem you’re trying to solve with the gem, how to install it (a must if it’s non standard), and some basic usage instructions makes all the difference between someone guessing what the heck’s your gem is even for based on the name of it (which isn’t very helpful whether you picked a sensible name, or a trendy vowel scarce one). Writing thorough tests is also quite important because it can make all the difference between your gem being instable and no more useful than some academic code that explores an interesting domain, and actually having some robust code suitable for production usage.

Now how to wrap all that best practice into a re-usable template? well hoe provides a nice format for doing that. I’d pretty much settled on a preferred setup by this point, so I didn’t pay much attention to the default setup that hoe provides, setting up one or more templates with hoe is super easy, placing your templates in ~/.hoe_template (place your default configuration into a directory called “default” in there. You can have dynamic files by simply appending .erb and putting erb code inside any of the files here. Creating a gem with this format is then as simple as

sow <gemname>

For those interested, here’s the default structure and gem template that I use.

Finding Bugs With Git Bisect

If you’ve ever had a bug that got introduced to an application you’re working on at some point, but haven’t been able to track down what caused it, then git has a really useful feature that you can use to track it down called bisect.

This happened to Jordan Carter and I recently. We had a bug to do with the javascript interface and some values not being inserted into a form correctly. These kind of bugs can sometimes be hard to track down, because they can be affected by either changes to the javascript, or the html is manipulates. We came across the bisect function in a google search, and decided to give it a go, the process is pretty simple

# First create a local branch to do the tracking in
git checkout -b find_that_bug
# Next, start bisecting!
git bisect start

After that, you need to mark some commits or references that you know either contain or don’t contain the bug. In our case we know that the bug currently exists in master, so we github commits to find one old enough that we were sure wouldn’t have the bug.

# the master branch is broken
git bisect bad master
# but this commit is good
git bisect good 0f54590e

This is where it starts to get fun. We were initially confused because we expected the next part to take a linear approach, but it wasn’t. Git tells you how many commits are between the two,  that’s the number of commits left to check, (if they were the ones to introduce the bug). Then git will load up a commit somewhere in between and you mark it as either good or bad, depending if it’s got the bug checking either the unit tests, or in our case actually using the web interface. The smart thing is that it zips back and forwards between the good and bad end, zeroing in on the commit that introduced the bug, so you may only end up checking around 6 commits out of 40.

Once the “original bad commit” is found, git places you at the commit just before that, and you can look at the diff for what was changed in the bad commit, and try applying some of those changes if necessary to see exactly what it was that broke your code. In our case the offending change was something as innocuous as adding an extra closing div tag to a partial that was misleadingly enough not self contained, and the div wasn’t supposed to be closed.

All in all, we were pretty happy to have found the bug so quickly, and found git bisect incredibly useful. Seeing how useful it was made highlighted for me the necessity of self contained commits (e.g. when refactoring, commit the entire refactor. Don’t break something, commit, then fix it in a later commit if you can help it) because this will prevent you being able to find bugs for that feature with git bisect.

Utilizing the Google Visualization API With Ruby on Rails

If you haven’t seen it yet, Google have recently released a public visualization API for generating many different types of graphs including Geo Map, and Motion Charts. The graphs are all in the nice clean style that you see in Google Analytics. The end result of the graphs is really nice, and is produced with nothing more than javascript, however anyone utilizing a framework for dynamic Web applications will likely want a more developer friendly way to embed the data.

For those of you using Ruby on Rails, I’ve written a plugin gvis that makes the syntax for embedding data a lot cleaner, and most of all, accepts ruby data arrays, handling the library loading and javascript data formatting for you. As an example to see how easy this is, lets go through setting up a new rails application and drawing a graph with this plugin.

rails graphapp
cd graphapp
script/plugin install git://github.com/jeremyolliver/gvis.git
# note you'll need git installed for the above line to work.

Now lets add a single controller and add routes for it:

script/generate controller home
config/routes.rb
1
2
3
#...
map.resources :home
#...
app/helpers/application_helper.rb
1
2
3
4
5
module ApplicationHelper
  # include the plugin module in application helper
  # so that we can add a graph anywhere in our app
  include GoogleVisualization
end

Lets add a simple layout, it need only include google’s javascript source for the visualization api, and then call render_visualizations

app/views/layouts/application.html.erb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
<html>
  <head>
    <title>Test app</title>
    <%= include_visualization_api %>
    <%= render_visualizations %>
    <%= yield :head %>
  </head>
  <body>
    <%= yield %>
  </body>
</html></pre>
Now lets render a nice fancy <a title="MotionChart - Google Visualization API" href="http://code.google.com/apis/visualization/documentation/gallery/motionchart.html">MotionChart</a> on our home page.
<pre># app/views/home/index.html.erb
<h1>Rendering a cool MotionChart</h1>

<%# This data would typically be pulled from the database,
    but lets just put some sample data into an array %>
<% chart_data = [
   ["Apples", Date.new(1998,1,1), 1000,300,'East'],
   ["Oranges", Date.new(1998,1,1), 950,200,'West'],
   ["Bananas", Date.new(1998,1,1), 300,250,'West'],
   ["Apples", Date.new(1998,2,1), 1200,400,'East'],
   ["Oranges", Date.new(1998,2,1), 950,150,'West'],
   ["Bananas", Date.new(1998,2,1), 788,617,'West']
 ] %>

<% visualization "chart_id", "MotionChart", :width => 600, :height => 400,
                             :html => {:class => "graph_class"} do |chart| %>
 <% chart.string "Fruit" %>
 <% chart.date "Date" %>
 <% chart.number "Sales" %>
 <% chart.number "Expenses" %>
 <% chart.string "Location" %>

 <% chart.add_rows(chart_data) %>
<% end %>

And Vóila! we now have a beautiful Motion Chart displayed in our rails app in almost no time. Let me know if you find this useful, or you have any thoughts on how this could be improved.

Using MacRuby to Write a .qif File Converter

I heard about the MacRuby project not too long ago and was quite excited about it for several reasons, the first is  that although it’s early days yet and version 0.4 doesn’t cover the full ruby spec, it’s showing signs of being a well performing ruby implementation. Others have already gone into significant detail about the differences, but to summarise: The original ruby implementation isn’t the best at memory management for long running processes, and this is the most recent of several other implementations that provide better performance. The other reason to get excited about this is that because MacRuby is essentially the ruby language written in Objective-C (not just a bridge), it means you can utilise both ruby’s syntactical sugar and the Cocoa library to write native Mac OS X applications using ruby code. This is pretty cool to me, because Objective-C has a pretty verbose syntax, manual memory allocation, and is generally not quite as pretty to read and write as ruby code.

Of course, nothing illustrates this quite as well as seeing how short a sample (trivial) GUI program is when written with MacRuby and HotCocoa:

example.rb
1
2
3
4
5
6
7
8
9
require 'hotcocoa'
include HotCocoa
application do |app|
  window :size => [100,50] do |win|
    b = button :title => 'Hello'
    b.on_action { puts 'World!' }
    win << b
  end
end

The equivalent code in Objective-C is about 3 times as long, less readable and much more confusing. That’s what I love about well written ruby code: It should be clear what’s happening, and straight to the point.

I couldn’t wait to try out writing a desktop app with MacRuby, so when a colleague needed some financial data converted from .qif format to .csv I had a look for a free tool to do this on Mac OS X, but couldn’t find one that fit the bill. So after utilising a bit of regex magic to convert the files, I decided to test out just how easy it was to write a desktop app in MacRuby, by writing an app with a GUI to do the file conversion process.

After trying out using XCode to create a new MacRuby project without much luck, I found that the HotCocoa library that comes with MacRuby (0.3+) can generate a blank app for you. Once MacRuby is installed (I’m using version 0.4) you can simply type

 hotcocoa myapp

from the terminal to generate a new application, with some laid out code, just waiting for you to add the visual elements and logic you need to your app. This sets up some rake tasks for compiling the application too using macrake (MacRuby’s binaries are just like you’re used to but prefixed with mac, macruby, macrake, macgem etc) including for when your application is finished, you can package up the MacRuby framework inside the final application for distribution, so that others can run your .app file without having to have MacRuby installed on their computer.

cd myapp
macrake        # This builds and launches the .app
macrake deploy # This builds a fully packaged version ready to ship

Overall, the process was pretty simple, I added some horizontal and vertical layouts to configure the window looked, and after only a few days of reading the documentation and coding in my spare time, I came up with Qif2csv. It’s a bit rough, and I still didn’t figure out how to make a check box with the ruby syntax, though feel free to download it, and browse the source. It was fun to try out something a bit different, and it’s always good to keep building new skills. Now of course, we can all anticipate the release of MacRuby 0.5 which is a largely performance focused release, and I believe will also support native threading.

Git Branch Shown in Command Prompt

I’ve been using git for source control for a while now, and I’m really enjoying using a distributed system over the conventional server-client system like svn. A distributed versioning system is somewhat conceptually different in the way that is works, and while it’s worth taking the time to understand the differences, one of the immediate differences is in the way branches are stored in the file-system. With a traditional system, checking out a branch of your code involves creating a whole new folder for your local checkout, duplicating a whole lot unnecessarily, though it is pretty obvious what’s going on.

In contrast when you are using git, your local computer has it’s own entire repository (hence distributed), this is stored in a hidden folder inside the project root (myproject/.git/). When you checkout a new branch, the files in the project directory are updated in place with the files from the branch. This means that multiple branches all reside within the same folder on your computer, and only one can be checked out at a time, it can get somewhat confusing at times to remeber which branch is currently checked out, and it’s annoying having to type “git branch” every time you want to find out. Here’s a bash script that displays which git branch is currently checked out whenever your terminal is residing in a git repository.

~/projects/myproject(master) $

Simply add the following code to ~/.bash_profile (create it if it doesn’t exist), and you’ll always know which git branch you currently have checked out.

~/.bash_profile
1
2
3
4
parse_git_branch() {
  git branch 2> /dev/null | sed -e '/^[^*]/d' -e 's/* (.*)/(1)/'
}
PS1="w$(parse_git_branch) $ "

Custom Change Messages - Rails Plugin

While working on some code maintenance I noticed a prime opportunity to refactor some code in the spirit of keeping the code DRY. There was code in both the controller action and in mailer methods that created a short paragraph of text describing what fields had been edited on several models for the purposes of both logging this for users to reference later, and to send email notifications.

My solution for this problem was to create a new plugin (custom_change_messages) (EDIT: now converted to a ruby gem) which extends ActiveRecord::Base to provide a few simple methods on instances of your models to generate this text summary of the changes, before the model is saved back to the database (taking advantage of the code in the ActiveRecord::Dirty module). The plugin provides an easy way to customise the messages that are displayed and their format, and handles belongs_to associations nicely as well: Given a Task model that belongs_to a User model, re-assigning a task would produce the message “user has been changed from ‘Jeremy’ to ‘Jordan’” rather than the less informative “user_id has changed from 1 to 2”. See the README for more details on usage.

item.rb
1
2
3
4
5
6
7
8
9
10
11
12
class Item < ActiveRecord::Base
  belongs_to :person

  custom_message_for :person, :display => :username
  custom_message_for :due_on, :as => "Due Date",
  :message => "has been rescheduled", :format => :pretty_print_date

  # this method is used for formatting the due_on field when it changes
  def pretty_print_date(value = self.due_on)
    value.strftime("%d/%m/%Y")
  end
end
items_controller.rb
1
2
3
4
5
6
7
8
9
class ItemsController < ApplicationController

  def update
    @item.attributes = params[:item]
    Mailer.deliver_item_update(@item, @item.change_messages.to_sentence)
    @item.save!
  end

end