Tuesday, 27 May 2008


Ok, so I'm bored. After Ben's suggestion for improving POST redirections I decided to implement a "controllerize" inflector ;)

Not sure yet how to get this into String (so I can do: "user".controllerize). I tried going via class String, module Inflections, and via module CoreExt with no joy... so for now you still have to invoke it via Inflector. I welcome suggestions, but I really should stop distracting myself with nifty metaprogramming. ;)

Anyway, this is the code - stuff it into somewhere like environment.rb:

module Inflector
  # turns a given class name into the properly capitlkaised controller name.
  # This can be fed to "constantize" to get the actual controller by this
  # name
  def controllerize(word)
    # make sure it's the raw class name, camelize then add controller 

Examples of use follow:

>> Inflector.controllerize("user")
=> "UsersController"
>> Inflector.controllerize("my_widget")
=> "MyWidgetsController"
>> Inflector.controllerize("String::Thingy")
=> "ThingiesController"
>> ctrl = Inflector.controllerize("user").constantize
=> UsersController

Link: 21 Ruby Tricks

Not an article, just a link to a good one. Peter Cooper has written a nice article named 21 Ruby Tricks You Should Be Using In Your Own Code. They're pretty neat! Most of them had me nodding along, but there were several that made my eyes widen in appreciation.

Wednesday, 21 May 2008

Rails gotchas: more TzTime

While I'm neck-deep in timezone land, I'll also address one little problem wih the tztime plugin. While running the functional tests I began getting errors like the following:

NoMethodError: undefined method `tzinfo' for #<TZInfo::LinkedTimezone: UTC>

I could fix it by forcing a timezone into the TzInfo object at the beginning of each failing test - but that seemed a bit of a hack. Looks like the problem has been submitted as a bug in rails ticket 11150. The suggested codefix seems to work just fine, but requires more hacking on your vendor/plugins/tztime directory. Still, it's a one-line fix, so not much to do.

The failing tests were replaced with a new error:

<Tue Apr 01 00:00:00 +0000 2008> expected but was
<2008-04-01 00:00:00 UTC>.

The failing line is in the functional test thus:

assert_equal Time.now.at_beginning_of_month, assigns(:month)

This is an instance of a problem I've seen before... ie that you don't seem to be able to compare Times with DateTimes, Dates or (now) TzTimes. The quick solution is to forcibly cast all your time-like objects into a common time-like object before testing for equality. My suggestion is to_datetime, which works for all the instances I've needed, thus the previous test case would become:

assert_equal Time.now.at_beginning_of_month.to_datetime, assigns(:month).to_datetime

Converting a TimeZone to a TzinfoTimezone

The TzInfo gem and plugins: tzinfo_timezone and tztime provide a bunch of timezone-related nifty things that intend to make your life easier by providing extra helper stuff on top of the standard Rails TimeZone class.

Personal preferences

However, I personally hate the dodgy-looking "Country/City" style of timezone list that the plugin provides. I much prefer "GMT+10:00 Sydney" provided with Ruby's TimeZone code. Especially when dealing with clients that can be expected to understand what a timezone is.

I figured I could take davantage of the niceties of TzTime/TzInfo for converting, displaying and updating activerecord etc... but use the standard Rails time_zone_select helpers on the user-preference form, and just convert the users chosen TimeZone string into a TzinfoTimezone before applying it.

But when I tried it, I suddenly started getting something along the lines of:

TZInfo::InvalidTimezoneIdentifier: no such file to load -- tzinfo/definitions/Tijuana

One big problem

Delving into the code, it seems that neither the TzInfo::Timezone class nor the tzinfo_timezone plugin's TzinfoTimezone class actually provide a method to convert from the standard Ruby TimeZone string and back... even though the plugin seems to do this internally (to generate the new list of timezones for the drop-down list).

This seems a bit of an oversight as the plugin already contains a mapping (conveniently named MAPPING) between the two.

The monkeypatch

The required code change is pretty small if done inside of tzinfo_timezone. The TzinfoTimezone class' new method tries to fetch out the appropriate timezone object, returning nil if not found. A one-line change fixes it so that after trying with the given timezone, it checks if it's in the given mapping before trying again thus:

  # replace:
  def new(name)

  # with:
  def new(name)
    self[name] || self[MAPPING[name]]

The form drop-down

Adding the field to a user is fairly simple - assuming the string column is named "time_zone" - I use the pretty standard:

<%= time_zone_select 'user', 'time_zone', TimeZone.all, {:include_blank => true} -%>

The Controller changes:

According to the tutorial on using TzTime, we put an around_filter in the controller to prep TzInfo with the logged-in user's chosen timezone (or a default if they have none, or there's nobody logged in). Becuase the plguin doesn't directly update the TzInfo::Timezone class, we have to alter that to directly call our updated method.

  # old way to set up default timezone
  around_filter :set_timezone
  def set_timezone
    # pull pref from the user - if they've supplied it
    if logged_in? && !current_user.time_zone.nil?
      TzTime.zone = TZInfo::Timezone.new(current_user.time_zone)
      # otherwise use the environment's default = UTC
      TzTime.zone = TZInfo::Timezone.new(ENV['TZ'])

  # new way to set up default timezone
  around_filter :set_timezone
  def set_timezone
    # pull pref from the user - if they've supplied it
    if logged_in? && !current_user.time_zone.nil?
      TzTime.zone = TzinfoTimezone.new(current_user.time_zone)
      # otherwise use the environment's default = UTC
      TzTime.zone = TZInfo::Timezone.new(ENV['TZ'])

A validation for good measure

just to add to the pile, a validation to make sure the user's chosen timezone actually exists...:

  validates_each :time_zone do |model, attr, val|
       model.errors.add(attr, "is not a valid timezone") unless val.blank? || TimeZone.all.any? {|tz| tz.name == val }

Tuesday, 20 May 2008

Selling yourself tomorrow to buy today

Wow - I just read a really thought-provoking piece on The Register titled: Privacy? Forget it. Sell your brain and desires to the highest bidder

It discusses the future intentions of companies seeking to buy your purchasing habits; and provides insight into the reasons behind Phorm, Facebook's Beacon, and many of the cutting-edge market-data gathering machines out there.

Nothing new you say? So thought I until I got through it all - it's a fresh and entertaining angle on an argument that we may have heard a lot about - but which still has room for improvement. Well worth a read.

Here's a snippet to get you started:

All around us the toasters are getting smarter. Sadly, we don't seem to be keeping up with the program. We remain poor schlubs.
In the good old days, a toaster was just a toaster. It warmed bread and issued the odd electrocution. Tomorrow's toaster, however, brings with it a new set of functions that travel well beyond bread.
As we hear it, the toaster of the future will contain more silicon and communications systems. It will have sensors that can detect an upcoming failure, and alert you, via e-mail perhaps, to order a new toaster. Or maybe the toaster will order a new version of itself from the internet if you've enabled the self-upgrade feature. That future toaster may also talk to other gadgets in the home. Perhaps they'll agree on when they should shut down to save energy or to avoid a fire. Maybe they'll agree on the next brand for their upgrade cycle and make sure you get matching stainless steel appliances throughout the home or that your current penchant for pastel kit is obeyed...

Read on...

Friday, 9 May 2008

Lay a Wizard over your controllers

Adding a wizard interface to a pre-existing system

Our system requires a user to enter in a large data set and several preferences to complete their order. It all works ok, but new users can get confused when presented with a busy and convoluted screen full of scaffolds and forms - which the initial (advanced) interface currently looks like. So we decided to add a step-by-step "create your order" wizard to help new users get an idea of what they need to do in order to create, edit and submit their orders.

As it happens - the ordering (and number) of steps in the middle don't matter too much. It's all just entering random bits of information. As long as the user creates the order at the beginning and submits it at the end - it doesn't matter if they add their widgets first, or set up their preferences first. In fact, given we don't care about the specifics of the steps it makes some sense to make sure this wizard is highly flexible.

Lucky for us - Ruby gives us that luxury.

Now, we already know the system does all of the existing parts of the process - it's currently happily creating/updating and submitting orders as we speak. So "all we need" is a new overlay of views and a new path through the controllers.

But how to do that in a nicely independant way?

First - make thy controller

We'll need one of those, and we'll need an action for each step in the process - plus an index action thrown in for good measure. So lets start with:

script/generate  controller WizardController create_order add_widgets specify_preferences review_and_submit index

Adding an action per step lets us make nice, named routes that look good in the URL-bar for our user. They also make routing easy. We only need to add the route:

map.wizard '/orders/:id/wizard/:action', :controller => 'wizard', :id => nil

Note that this is great for all actions on an existing order thats saved in the db and has its own id. However, a new order won't have an id yet, so we'll need a route to cover that too:

map.new_wizard '/orders/new/wizard/', :controller => 'wizard', :action => 'index'

Here's the basic wizard controller. Note that we try to fetch out the order before each step with a common before_filter. The "index" action will dump the user into the second wizard-step if they specify an existing order id. This is so they can come back to the interface from, say, a list of orders and not get dropped (confusingly) on the "new order" page.

# controls the "new order wizard" functionality
class WizardController < ApplicationController
  # this list holds the current ordering of the wizard steps. It's used by the 
  # workflow-display code and anything that needs to know what the "next step" is
  WIZARD_STEPS = [:create_order, :add_widgets, :specify_preferences, :review_and_submit]

  before_filter :login_required
  before_filter :prep_for_step

  # the various steps in the process of the wizard
  # Note - they are named, rather than numbered so that they can be
  # adjusted/renamed/removed without worrying about their ordering.
  # There's no code as most of them all operate simply off the order 
  # (which is fetched out for every step) and have a same-name template
  def create_order
  def add_widgets
  def specify_preferences
  def review_and_submit
  def index
    # if we've found an existing order, go straight to the first data-entering step
    return render(:action => WIZARD_STEPS[1]) if @order
    render :action => WIZARD_STEPS.first # otherwise begin at the beginning

  private ############################################################
  # preload the order, if an id has been passed  in - or create a temporary one if not.
  def prep_for_step
    return false unless logged_in? # to be sure, to be sure
    @order = current_user.orders.find(params[:id]) unless params[:id].blank?
    @order ||= Order.new(:user => current_user)
    @wizard_step = params[:wizard_step] # save current step to pass into templates

Make a template for each step

Each action will need an associated template showing the options available to a user for that step of the process. This template will probably import partials from your pre-existing views. After all, you're laying this interface over an existing one - so template re-use is a Good Thing. The view re-use gives similar operations a common appearance across both the wizard and advanced interfaces. This helps a user make the step from the simple to the complex interface, due to a pre-existing familiarity with the appearance.

So why not just use the same views as your existing system? There are a few benefits to having separate wizard templates.

First is to provide a unifying, common "wizard interface". Eg a visualisation showing where the user is in the process workflow. Possibly ven a subtle change in the page style to make sure a user knows when they are "in the wizard" as opposed to the rest of the site. Clues as to orientation are surprisingly helpful, especially to new users who are unfamilliar with the layout and content of your site. Every bit helps.

Second - this interface should contain extra instructions on the requirements of the current step. You know for a fact that the wizard will be used by the novice users of your system - so make sure you support their needs by providing extra help. The purpose of this wizard is to hand-hold your newbies through a difficult and complex procedure. Now that you've cut it up into bite-sized pieces, make sure you don't let them down by failing to explain each step.

Finally, the wizard interface should be much simpler than the normal view templates. As aleady stated, these users will be brand-new to your site. They don't want the advanced power-user features just yet - they just want to know how to get started. Any highly complex extras should be left out. You can keep the advanced uber-functions for when you user is confident enough to move beyond the wizard to your standard (now advanced) interface. Just keep the bare minimum that a user will need for this wizard to be functional, without crippling it so badly as to be useless. ;)

Displaying the workflow

Your user will need to know: a) what steps are in the wizard b) what step they are currently on. This is basically just a fancy tab-navigation structure, with the current one highlighted. I've covered this sort of thing in my tabbed navigation articles, so I won't repeat the details here.

You will need to store the set of steps somewhere accessible. In this case, we figured it made most sense to store the list as a constant in the WizardController itself. I also recommend using little images of right-facing arrows in between each "tab" to give the perception of flow.

I generally add a step-specific overview (eg "In this step, add widgets to your order") to get the user oriented. For us this goes directly underneath this tab-list so there's an obvious visual connection between the overview and the current-step. Don't make it long or it won't get read. Brevity gets the point across better!

Interfacing with the existing controllers

So, we have nifty templates that display what a user should be updating next - eg a list of possible widgets for the user to add to their order. So when they click on the "add widget" button - what happens next?

We already have an "add widget" action in our WidgetsController and we want to re-use that because duplicating controller functionality is just a nasty maintenance nightmare waiting to happen. So, the forms that are displayed on the templates need to point at the usual controller actions just like your original views. The problem is that when the WidgetController's "create" action is done, it's likely to send the user on to "order_widgets_path" (just like normal) - when we really want it to come back to the current step in the wizard.

So - how does the WidgetController know we want the wizard instead of the usual control path? and if so - how do we know what step of the wizard we need to pass on to? and finally, can we do all this without becoming terribly hard-coded and brittle?

Firstly, we can tell the other controller that we are coming from the wizard by passing the current wizard step in a variable in the required forms/links thus:

  <% button_opts = {:order_id => @order.id, :escape => false, 
                    'wizard_step' => 'add_widgets' } -%>
  <%= button_to('Add to order', new_order_widget_path(button_opts.merge(:id => widget.id))) -%>

This will probably require some hacking on any existing partials that have embedded forms to add a field-repeater eg:

  <%= hidden_field_tag('wizard_step', @wizard_step) if @wizard_step -%>

Now you can add a check in the relevant controller actions that tests if this step is present. If so, we need to go back to the given wizard step... but checking specifically for param[:wizard_step] is a bit nasty. What happens if we have to change how we specify that we're currently executing a wizard-step? So pull it into a common method that does a "test and redirect" thus:

In application.rb:

  # redirects the user on to the given step in the wizard process
  def redirect_to_wizard_step(the_id, step)
    return false unless current_wizard_step # allow controller to continue on as normal
    redirect_to wizard_path(:id => the_id, :action => current_wizard_step)
    return true

  # Redisplays the current step in the wizard process.
  # Note: only use this if the current action was *not* successful.
  def redisplay_current_wizard_step(the_id)
    return false unless current_wizard_step # allow controller to continue on as normal
    # render template is necessary as we are likely not coming via the
    # wizard controller
    render :template => "wizard/#{current_wizard_step}"
    return true

  def current_wizard_step

In WidgetController:

  def create
    @widget = Widget.new(params[:widget])
    @widget.order_id = params[:order_id]
    # to re-render wizard-step in view - if present
    @wizard_step = current_wizard_step 
    respond_to do |format|
      if @widget.save
        format.html { 
         # if we came here through the wizard - move on via the wizard process
          return if redirect_to_current_wizard_step(@widget.order.id)
          redirect_to order_path(@widget.order) }
        format.xml  { head :created, :location => order_widget_path(@widget.order, @widget) }
        format.html { 
         return if redisplay_current_wizard_step(@widget.order.id)
          render :action => "new" }
        format.xml  { render :xml => @widget.errors.to_xml }

That's about it - happy wizarding.

Tuesday, 6 May 2008

Rails gotchas: Data Migrations conflict with validations

So the problem is that we have some basic data that we like to preload into the database - eg an initial admin user. We created the data migration to do this and it ran fine, validated correctly and was loaded into the database for us.

Over time we added new migrations and new validations and at every step the db and code were in-synch.

Now we need to set up a new testing server. We ran the mmigrations from VERSION=0 and suddenly we get an error on the data migration along the lines of:

rake aborted!
undefined local variable or method `group_id' for #<User:0x3367bc8>

Obviously this column wasn't needed when we wrote that migration, and by now we already have the admin user in our system, so the migration hasn't been run on our latest code... until now.

The problem is that the code *now* requires a group_id for the user to be valid; but at this point we haven't reached the migration that adds the groups table to the database, along with the corresponding group_id column on the users table.

However, all the latest code is in our model, including the validation that requires valid user to have a group_id. *BUT* the group_id method simply doesn't exist yet, because ActiveRecord will only find that method by reflection once the table has the appropriate column.

We have a conundrum. We don't want to remove that validation, but we also need that data to be in the database.

So what can we do?

There are several nasty options we considered, such as adding a rescue-block around the validations and catching "just that error". Unfortunately this has the problem of hiding any future errors that we may need to fix. Allowing errors to get artificially caught, like this, can lead to them ending up in production code without us having noticed their existance. :P

We could also go back and delete the data-migration from the original file and generate a new migration for it. The new migration will be called after all the current ones and so will match match the current code. This just delays the inevitable, though. We will cimplu come up against the same problem at some future time when new validations are brought in. There needs to be some solution that will fix this for all time.

There were two reasonable solutions that we considered.

The first is to put the required data into a fixture. The fixture can be kept up-to-date with the code and therefore will always pass the validations of the present-set of code... it could then be loaded on-demand (via the console), or we could alter the original data migration to load the fixture instead of creating an ActiveRecord object. The latter has the same problems as the original issue - it will try to load in things like the group_id column at a point in the migrations that the column doesn't yet exist. The former would work - but we'd need to remember to do it every time - which means it's prone to user error.

The alternative involves a simple one-word fix... which is why it won out.

The Validations module overrides the standard ActiveRecord::save function with the save_with_validations function. This latter function takes a single, optional parameter that can tell ActiveRecord to ignore validations. ie you call save(false) and it will save the record without checking that the data is valid.

If we can assume that our own data in the data migration is valid for our purposes, then we can use this to solve the problem, as the validations will simply be ignored.

Thursday, 1 May 2008

Up or Out - the attitude for contractors

I've just read the article Up or Out on The Daily WTF. This is related to/based on Bruce Webster's article Wetware crisis: Dead Sea effect. They provide a fascinating look into the culture of the skilled IT workforce as it pertains to over-zealous attempts at retaining skilled workers.

So I got to thinking about how that applied to myself, and my past experience in IT. I've worked both as an employee and, recently, as a contractor - and I've certainly noticed the difference in attitude between the two.

I believe that contractors automatically have this "up or out" mentality. They know that their required skills are wide and varied, and that to keep ahead of the game, they need to keep their skill-level up. Staying in one job for a long term leads (eventually) to skill-stagnation. Not because the company no longer has anything to teach a contractor, but because of a case of diminishing returns.

As the article states, any skilled worker can learn a lot more by exposing themselves to new opportunities, than by sticking with a single firm. However, contractors seem to be far more aware of this fact than long-term employees (even the skilled ones). Contractors seem to have learned the lesson that current and diverse skills are marketable, and that in-house knowledge of a specific system, while beneficial to keeping a specific job, doesn't lead to expansion of a skill-set. This leads contractors to constantly push themselves to try new and different things to stay ahead of the game - to keep themselves a marketable commodity.

Employers don't seem to have kept up with this understanding. The majority of employers look poorly on hiring contractors (except for specific skills lacking in-house), compared with potential long-term employees. Contractors fall foul of the attitude mentioned in the article - the feeling that contractors are just "dating around" and aren't serious about their committment. This despite the fact that employers are no longer loyal towards employees (a job is no longer a career-for-life).

So are contracters flighty or are they simply being honest? We know that our skills improve by varied experience. Stagnating in a single shop isn't good for us *or* for the company - yet we constantly get the look-down-the-nose treatment as though we are far more unreliable than their indentured employees.

Workers and companies both benefit from fresh-blood. There is definitely a balance involved here - continual turnover can make it impossible to keep hold of institutional knowledge (which is why documenting procedures is so important!), but a company without fresh-blood will stagnate and lose market-share due to a lack of new and innovative insights coming into the company. Any firm can benefit from a good mix of the two. The fact that contractors have accepted and embraced this fact should be a sign of maturity, rather than flightiness.

I especially dislike that skeptical tone that comes with with the phrase "oh, so you're *not* interested in long-term employement". I feel I'm simply being realistic. But employers (and recruitment agencies) that employ the tone seem to imply I'm being disloyal... or committment-phobic. Neither of which is the case.

A few rare cases even seem to imply that I'm simply being greedy - shopping around for a better cash-deal, which is far from the case. While I'll take a higher salary over a lower one, I'm far more motivated by an interesting project and an opportunity to learn. That's why I'm in Rails rather than many other technologies that pay well. I would never take a higher salary just to do something that was mind-numbingly boring and, in my mind, dead-end. I'm just stoked that I get to work in a field that is cutting edge *and* well paid at the same time ;)

I have no problem with sticking around with a group that values me and that contributes to my own experience. My current contract (with SIRCA) has lasted more than a year. I fully believe that I have added value to this project. I also have learned a great deal. I have also had smaller contracts where I have provided some chunk of functionality or code-review - each of which added visible benefit to the site I worked upon, and also provided an opportunity for me to learn some new aspect of the technology I work with. This is an equal-footing relationship - where both parties receive value.

My previous employment has included roles in which I felt my input was not valued, and in which I had no mentor from which to learn. Effectively I was gaining no value from them, and they were not gaining full value from me either. I stuck around for a long time trying to help out with the project - and eventually left due to burn-out. In hindsight I should have left far earlier and moved on somewhere that I had an opportunity to grow, and that valued the insights I was able to bring. It would have been a better opportunity for me - and the company I was with could have employed somebody that they would have felt comfortable with - increasing their return-on-investment as well.

In a free market - there's no point in sticking with a relationship that is not valuable, or which provides value only to one party or the other. I won't leave for reasons of greed or lack of committment - I'll only leave if you and I are no longer gaining enough value for the professional relationship to be worthwhile.

If an employer doesn't feel that this is reasonable - then we need a good talk on the concept of "fairness".

Healthy economic relationships come about when both parties benefit. Anything else doesn't make economic sense. When the benefit to one or the other fades over time, then it should be understood that this will lead to the eventual ending of the relationship. It's simply a matter of good business practice - and shouldn't be looked on as "lack of committment" any more than moving to a cheaper/more efficient supplier.

</whinge> :)

Update: Bruce Webster has written a follow-on article: some thoughts on up or out which gives a few ideas on how to avoid developer-churn or the Dead-sea effect (and even counteract the thermocline of truth) by providing a non-managerial track for techies to follow. I completely agree. If there was a way "up" that didn't involve "out" I'd go for it!