Wednesday, 29 July 2009

CSV views with FasterCSV and csv_builder

We were asked to add a "Download as CSV" link to the top of each of our reporting actions. We already had a "revenue_report" controller-action with it's own html view template... and just wanted to add something similar in CSV.

FasterCSV seemed to provide some great CSV-generation functionality... but it builds the csv and spits it out right there in the controller.

uuuuugly!

We shouldn't put view-generation code into the controllers - that splits up our nicely-decoupled MVC stack!

I've also seen some ideas about putting a to_csv method into an individual model. Now, if I'm leery about putting view-generation in the controller... I'm even less impressed by putting it into the model! But I can see the benefits of simplicity.

However - I'm still not sure it fits with our requirements. A @widget.to_csv call works just fine for the WidgetsController#show action... and probably even the WidgetsController#index action (where you can run @widgets.map &:to_csv or similar)... but most of our reports span multiple associations and the data is grouped, filtered, and summarised. Each of these sets needs headings explaining what it is and laying it out nicely in a way that makes sense to the user reading the CSV file. Putting that kind of visual-only functionality in your model is where it really starts to get ugly again. I am left with one big thought:

Why can't I just put my CSV-template into the views directory?

enter: csv_builder plugin

csv_builder comes with extremely minimal documentation... but it's not that hard to use. Here's a step-by-step with examples to follow:

  1. install fastercsv and csv_builder
  2. add a .csv option to the respond_to part of your action
  3. add your <actionname>.csv.csvbuilder template into your views directory
  4. Add a "Download as CSV" link for your user

Step 1: Install FasterCSV & csv_builder

sudo gem install fastercsv
./script/plugin install git://github.com/econsultancy/csv_builder.git

Don't forget to restart your server!

csv_builder has already set up a format-extension handler for csv so you don't need to add it to the mime-types yourself, just start using it in your actions.

Step 2: In your controller action:

csv_builder will work out of the box even if you just add the simplest possible respond_to statement:

  respond_to |format| do
    format.html # revenue_report.html.erb
    format.xml  { render :xml => @payments }
    format.csv  # revenue_report.csv.csvbuilder
  end

With nothing more than the above, csv_builder will grab out your template and render it in csv-format to the user.

You can tweak the FasterCSV options by setting some @-params in your action. The csv_builder gem README explains how to use those, but the most useful would be to set @filename to give your users a meaningful filename for their csv-file. You can also specify things like the file-encoding, but go look at the gem README to find out the details

Here's an example of a full controller action:

# GET /sites/1/revenue_report
# GET /sites/1/revenue_report.xml
# GET /sites/1/revenue_report.csv
def revenue_report
  @site = Site.find(params[:id])
  @payments = @site.revenue_report_payments
  respond_to do |format|
    format.html # revenue_report.html.erb
    format.xml  { render :xml => @payments }
    format.csv do
      # timestamping your files is a nice idea if the user runs this action more than once...
      timestamp = Time.now.strftime('%Y-%m-%d_%H:%M:%S')
      # passing a meaningful filename is a nice touch
      @filename = "revenue_report_for_site_#{@site.to_param}_#{timestamp}.csv"
    end # revenue_report.csv.csvbuilder
  end
end

Step 3: add a template into your views directory

The final step is to add a .csv.csvbuilder template to your views directory. This is just like adding an html.erb template for your action, except that the file will have the extension .csv.csvbuilder. AFAIK csv_builder can only support template that have the same name as your action, so in my example the template would be called revenue_report.csv.csvbuilder

Your view template is where you can stash away all the FasterCSV guff that will build your csv file. This works the same way as any other FasterCSV-generated content - but just make sure you generate it into a magic array that is named 'csv'. Here's an example:

  # header row
  csv << ["Revenue report for site: #{@site.name}"]
  csv << [] # gap between header and data for visibility

  # header-row for the data
  csv << ["Date", "", "Amt (ex tax)", "Amt (inc tax)"]

  row_data = [] # for scoping
  @payments.each do |payment|
    row_data = [payment.created_at.to_s(:short)]     
    row_data << "" # column gap for visibility
    # note you can use the usual view-helpers
    row_data << number_to_currency(payment.amount_ex_tax)
    row_data << number_to_currency(payment.amount_inc_tax)

    # and don't forget to add the row to the csv
    csv << row_data.dup # it breaks if you don't dup
  end # each date in date-range
  csv << [] # gap for visbility

  # and the totals-row at the very bottom
  totals_data = ["Totals", ""] # don't forget the column-gap
  totals_data << @payments.sum &:amount_ex_tax
  totals_data << @payments.sum &:amount_inc_tax
  csv << totals_data

Step 4: add a CSV link

Something like the above code will generate a nice plain csv file and spit it out at the user in the correct encoding... but now we need something to actually let the user know that they can do this.

This is pretty simple - it just requires adding the format to your usual path-links. eg:

<%= link_to 'Download Report as CSV', site_revenue_report_path(:id => @site.to_param, :format => :csv) -%>

Testing

Testing for non-html formats still seems pretty crude. You have to manually insert the "accepts" header in the request before firing off the actual request - then manually check the response content_type. It'd be nice if we could just add :format => :csv to the request... anyway, here's a sample shoulda-style test case:

  context "CSV revenue_report" do
    setup do
      @request.accept = 'text/csv'
      get :revenue_report, :id => @site.to_param
    end

    # variables common to all formats of revenue-report
    should_assign_to :site
    should_assign_to :payments

    should_respond_with :success
    should "respond with csv" do
      assert_equal 'text/csv', @response.content_type
    end
  end # context - CSV for revenue report

Gotchas:

Don't forget to add config.gem!"

I fond I needed config.gem 'fastercsv' and a require 'csv_builder' in my environment.rb. You may need a gem-listing for both (esp if you're using bundler)

Rails 3 compatitbility?

The original csv_builder plugin is not rails 3 compatible and is in fact no longer being supported. But it has officially changed hands, to the newer csv_builder gem. This is Rails-3 compatible and not backwards compatible - though he maintains a 2.3.X branch for people to submit bugs.

csvbuilder or csv_builder

While the plugin is named csv_builder, be careful to name your files with the extension: csv.csvbuilder or you'll spend hours pulling your hair out about a Missing template error while it fruitlessly searches for your csv.csvbuilder file!

All your report data must be generated in the controller!

This makes you more MVC-compliant anyway, but if you had any collections being fetched or formatted in the view... now's the time to move them into the controller action as your CSV-view will need them too.

duplicate inserted arrays

You'll notice that the example template has a set of data being added to the csv array... you'll also notice that each row is generated on the fly - then I save a *duplicated* version into the csv array. If you don't duplicate it... you may do funky things with overwriting the row each time. In my case I also needed to declare the temporary array outside the loop in order to preserve scope. YMMV - I'd appreciate any Ruby-guru answer as to why this doesn't work without that.

don't reset csv

Don't do this: csv = [] it breaks everything!

Rendering as html

I've noticed there's a pathalogical condition where everything is going well - and it's even getting o the template... but it's *still8 rendering html.

I eventually figured out that it's actually rendering the layout around the csv... and it occurs when you've explicitly stated a layout (eg layout 'admin') somewhere at the top of your controller. To fix it, you just need an empty render without layout false eg

  respond_to do |format|
    format.csv do
      timestamp = Time.now.strftime('%Y-%m-%d_%H:%M:%S')
      @filename = "revenue_report_for_site_#{@site.to_param}_#{timestamp}.csv"
      render :layout => false  # add this to override a specifically-stated layout
    end # revenue_report.csv.csvbuilder
  end

Tuesday, 28 July 2009

rails gotchas: undefined method `expects'

If you've just installed rails edge and run the tests, they may fail with: NoMethodError: undefined method `expects' for ... etc etc over and over again (along with a few other errors).

Install the mocha gem via rubygems to get rid of a lot of these errors.

It was the line NoMethodError: undefined method `stub' for ... that clued me in.

It seems that rails requires a short list of gems/packages (beyond rails itself) simply to run its own tests... yet there is no rake gem:install task available to help you figure out which ones... and they aren't in the "contributing to rails" guide. I'll be submitting a patch shortly...

Following on from this I got:

MissingSourceFile: no such file to load -- openssl

and farther down the list:

LoadError: no such file to load -- net/https

Unfortunately, these errors occur when Ruby hasn't been installed with openssl-support. If you're running ubuntu, you can just run apt-get install libopenssl-ruby.

Monday, 27 July 2009

40% speedup using Nokogiri!

Cut to the chase...

To cut your XML-processing time dramatically, sudo gem install nokogiri then add the following to config/environment.rb inside the Rails initializer.

config.gem "nokogiri"
ActiveSupport::XmlMini.backend='Nokogiri'

Back-story and pretty pics

The problem

So, our site makes heavy use of ActiveResource [1], meaning that most of our data is located remotely.

Not surprisingly, some of our pages are *really* slow, so I landed the task of speeding them up. Apart from page-caching (not possible), fragment caching (only helps on the *second* hit), or some complicated messy idea of data-caching locally (tedious and likely to be evil); my first thought was to reduce the number of network hits. Clearly that's a high pain point, especially on our heavy pages that have many resource fetches.

Before I dove into performance hacks and updating the business logic into twisty little data reuse-patterns for network-hit reduction... I decided to actually try profiling.

I've been setting up a ruby-prof and kcachegrind recently[2]... and figured I should at least give that a look-at to see if my assumptions are correct.

I'm really glad I did, because when I ran it over our heaviest action, I saw that all the highest-weight method-calls led back to some form of ReXML parsing.

Searching on the ReXML components showed that the heaviest ReXML method took up a whopping 1 million process cycles. When our total process-cycles came to 5.8 million - that's a significant chunk of time spent in that one library.

As I mentioned - our site makes heavy use of ActiveResource, and one *big* problem with ActiveResource is that all your objects are parsed and re-parsed as xml for every fetch of data... so, in hindsight, it's fairly obvious that our site would spend a *lot* of time in the XML-parsing library. Any speedup in that department would help us immensely.

The solution?

We've recently been to Rails underground, and one of the lectures[3] had a slide comparing the speed of ReXML to several other ruby XML-parsing libraries[4]. Nokogiri came out as a clear winner in the speed department. The loser was equally clear... that being the Rails-default: ReXML

So, switching out the library would be an obvious speed win.

As it turns out - it's really easy to do this. Just install the gem, and require it in your Rails initializer using the instructions at the top of this post

But did it really help?

It seemed faster... but can we prove it?

From ReXML to Nokogiri - 40% speedup

Yup.


Notes

[1] through the HyperactiveResource plugin
[2]I'll be giving a talk at LRUG on 12/08/2009 on how to use and interpret ruby-prof and kcachegrind
[3]During the talk by Maik Schmidt on Sneaking Ruby & Rails Into Big Companies
[4] I'm not sure, but it's possibly the one from this page comparing Ruby-XML performance benchmarks

Sunday, 26 July 2009

Rails underground

Just a quick post to say I attended the Rails underground conference and it rocked!

There were some fantastic talks by a wide range of speakers. My favourite being Jim Weirich's talk on the search for the Grand Unified Theory of Software Development. But I also enjoyed Yehuda's keynote talk on the future of Rails 3, Obie Fernandez's talk on setting up a rails business, and the panel discussion with all of the above, and a video-feed from DHH.

I also met some great people, including Desi McAdam, who introduced me to dev-chix, Geoffrey Grosenbach who chatted to me about my plans for my HyRes plugin and Yehuda Katz who I also chatted with about my plugin - and how to go about bringing it into Rails 3.

Friday, 17 July 2009

Adding prompt to select_tag

Rails's select method is pretty advanced. You can pass in all sorts of funky options to format your selector nicely, including :include_blank => true and the related :prompt => 'Pick one or else!'

select_tag, however, seems to have fallen behind in the usefulness-stakes and implements neither of the above... even though it does an extremely similar thing.

So here's a quick-hack function you can drop into ApplicationHelper to implement that functionality for you.

  # override select_tag to allow the ":include_blank => true" and ":prompt => 'whatever'" options
  include ActionView::Helpers::FormTagHelper
  alias_method :orig_select_tag, :select_tag
  def select_tag(name, select_options, options = {}, html_options = {})
    # remove the options that select_tag doesn't currently recognise
    include_blank = options.has_key?(:include_blank) && options.delete(:include_blank)
    prompt = options.has_key?(:prompt) && options.delete(:prompt)
    # if we didn't pass either - continue on as before
    return orig_select_tag(name, select_options, options.merge(html_options)) unless include_blank || prompt

    # otherwise, add them in ourselves
    prompt_option  = "<option value=\"\">" # to make sure it shows up as nil
    prompt_option += (prompt ? prompt.to_s : "") + "</option>"
    new_select_options = prompt_option + select_options
    orig_select_tag(name, new_select_options, options.merge(html_options))
  end

Wednesday, 15 July 2009

Gotcha: UTC vs Local TimeZones with ActiveResource

So... your database is filled with datetime data and it's all configured to localtime, not UTC... We also have this you-beat nifty ability to set all our datetime-handling functionality to a given timezone by setting, say: config.time_zone = 'London' in config/environments.rb... or do we?

If you also use ActiveResource (or the new, actually-working HyperactiveResource), you'll find that suddenly you're getting a UTC-local timezone issue once more.

The problem is that the xml that comes back from a remote API is converted into a Date/DateTime using the core-extension to the Hash.from_xml method... which has the following LOC:

"datetime"     => Proc.new  { |time|    ::Time.parse(time).utc rescue ::DateTime.parse(time).utc }

The fix

You need to do two things. Firstly. Hack that line[1] and replace it with:

"datetime"     => Proc.new  { |time|    ::Time.zone.parse(time) rescue ::DateTime.zone.parse(time) },

Secondly... somehow it doesn't pick up the timezone even though it's been helpfully added in via the config... so you need to open up config/environments.rb (or create a rails initializer) and put:
Time.zone = 'London'[2]
in there (outside the rails initialization block).

Notes:
[1]To hack rails, you can either
a) hack on your own rails gem = risky... will be overwritten the next time you sudo gem update or
b) rake rails:freeze:edge - which means you have your rails in your own vendor/rails directory... but means you have to rake rails:update manually... up to you which you hate more.

[2]Obviously substituting your own timeZone as appropriate here. See the TimeZone doc for what you can pass in.

Thursday, 11 June 2009

On the premature death of Array.to_a

hmm... after finally discovering that Array.to_a doesn't double-array now I'm getting warnings that Array.to_a is being obsoleted (with the unhelpful message: warning: default `to_a' will be obsolete). Aunty google tells me I should now use Array(thing)!

Can I just say (from a purely aesthetic POV):
yuk!

When you use to_a you can just read your code left-to-right eg:

Thing.do_stuff(args).to_a.do_something_else

You get a clear idea of what order your messages are passed in.
but having to use a stonking great Array() around one section means you have to jump back to the left again!

Array(Thing.do_stuff(args)).do_something_else

You have to read left-right-left-right :P
It's not as pretty.

So why exactly are they obsoleting Array.to_a ???

Tuesday, 9 June 2009

acts_as_good_style

Originally published in 2007, this article is updated on a semi-regular basis and republished when I add something new...

Rails Idiom

There are a number of small pieces of rails idiom that contribute to better rails code. I've been collecting them at work as they come to me (often by reading other people's code). I know that a lot of this can be gleaned simply by reading the "Agile rails" book... but apparently this would actually take effort, and thus doesn't occur as frequently as I'd like. A lot more is just basic good common sense... but we all know how common that is. In any case, I'll probably put this up on my website soon and keep adding to it.

Note - I don't always have a justification for these things... as with everything - it's up to you to pick out the bits that work for you.

Basic good sense

Clarify your code

This is something that I learned from Joel Spolsky from his article on making wrong code look wrong. He suggests that code smells should stand out a mile off. Doing so will make it much less likely that we'll miss one and leave a hole for a bug to form in.

He further suggests that we should change the way we code to systematically remove all habits (even benign ones) that muddy the waters. If we make it really obvious what is right and what is potentially wrong - then it'll be easier for us to discriminate between the right and wrong bits of the code. This follows from basic psychology. By analogy: it's easier to notice black when all you can see is black or white - but adding many shades of grey makes it harder to detect. If we make a habit of writing only white (or pale-grey)... it's harder for us to miss a black spot.

Joel goes on to discuss a technique for using Hungarian Notation (in the way it was orignally intended) to increase visbility of safe/unsafe strings - which is meant to be just an example of one technique. (So don't take it as the main essence of his message).

Many of the things I bring up in this Idiom-Guide are techniques that draw from this philosophy.

Don't comment-out dead code

Don't comment out code that's no longer needed - just delete it. It'll still be there in the SVN repository if we ever need it again.
Dead code just clutters up the project and slowly makes it less readable/maintainable. :(

Avoid Magic global variables where at all possible

The "don't use magic globals" argument has been around for a while, but here's the rails take on it.

The Rails equivalent of a magic global variable is the "@variable" (though I've also seen some nasty uses of the params hash). I know that instance variables are essential for passing data from the controller to the view. But they should not be relied upon for anything else! eg do not set them and assume they exist when calling methods on a model - or in a library.

From a given piece of code that uses said variable, you cannot tell where the variable originates. If it were simply passed as a parameter to the method, there would be an obvious line of command. Instead, you have to trawl through all the earlier functionality to figure it out or resort to a recursive grep on the rails directory.

This becomes a problem if you have to maintain the code. Given a magic global variable: Can I delete the value? Can I change it without affecting later code (that relies magically on its value)? If I run across a bug whereby it comes in with a value I didn't expect - how do I find which method set the value incorrectly?

On balance - it's much better to simply pass an explicit parameter through the usual method-call interface, so at least you have a chance of tracing the line of command in future.

Don't add ticket/bug comments into the code

I don't wanna have to wade through hideous piles of comments referencing some obscure (fixed) ticket/bug such as:

# DRN: ticket ARC256 -- remove currency symbol
# end ticket ARC256

It wastes time and eyeball-space. This stuff can conveniently go into the SVN log message - where it'll be picked up by the appropriate ticket in the "subversion comments" section. This way it'll only get read by people who actually care about the ticket/bug in question.

Alternatively - what we get is a buildup of comments referencing random tickets that are probably no longer relevant. Especially because the ticket numbers mean nothing when taken outside the context of our ticketing system.
Think about it this way: how long would we keep that comment in the code? when does it get removed? whose job is it to remove the ticket comment or do they all just build up until we have more ticket-comments than code? How unmaintainable!

Please just don't do it.


Ruby idiom

Don't use .nil? unless you need to

There are very few times when you are specifically checking for actual nil results. Mostly you are just checking to see if an object has been populated, in which case use: if foo don't bother with if !foo.nil?
This also brings me to:

Use unless where appropriate

Don't check if !foo instead use: unless foo

This is just the ruby-ish way to do things, and it reads better in many circumstances... given that code is meant to be read by humans, this is a Good Thing.

Do not use and/or - instead use &&/||

The words "and" and "or" mean something quite different to the symbols: &&/||
Unless you understand *exactly* what you are doing - you should use the latter (&& / ||) instead.

Any expression that contains and/or is almost certainly inelegent and should probably be rewritten.

do-end vs {} blocks

{} should be used for single-line blocks and do-end for multi-line blocks... this is a "taste" thing - but it seems to be fairly common practice now.

Use single-line conditionals where possible

if you're only doing one thing based on a conditional, use: foo_something if x
rather than:

if x
   foo_something
end

Obviously this doesn't work if you're doing more than one thing or when you have an else-clause, but it's useful for reducing space eg:
return render(:action => :edit) unless @successful
which also brings me to:

Nested function calls need parentheses

In a nested function call eg :
return render :action => :edit
Rails can make a good guess at whether :action => :edit is the second parameter of the "return" function, or the first parameter of the "render" function... but it's not wise to rely on this and it will generate warnings. It's much better to parenthesise parameters to any nested function eg:
return render(:action => :edit)

However:

Don't parenthesise where unnecessary

When parentheses are not required for the sake of clarity, leave them out.
do_something foo, :bar => :baz


Basic Rails security

h( x-site-scripting )

In your view templates, surround all user-input data with the "h" function. This html-encodes anything within it - which will prevent cross-site scripting attack.

Do this even if the source is trusted!

If you get into the habit of making it part of the <%=h opening, it will just become natural. An example:

  <p> Name: <%=h @object.name -%>
  <br />Description: <%=h @object.descr -%>
  <br /><%= image_tag image_url_for(h @object.image.file_name) -%> 
  <br /><%= link_to "Edit object", edit_object_path(@object) -%>
  </p>

:conditions => ['protect_from = ?', sql_injection]

Most SQL queries are built up by Rails automatically. But adding conditions to your finds or your paginators brings in a slight, but real possibility of introducing sql-injection attacks.
To combat this, always pass everything in as variables. eg:

:conditions => ['name like ? AND manager_id = ? OR is_admin = TRUE', 
@user.name, @user.id]

Generators, Plugins and Gems... oh my!

Generators

use scaffold_resource

When you create a new model, generate a scaffold_resource, even if you aren't going to use the view pages.
It generates wonderful tests as well as your model and migration for free.

It's worth it!

It's also worth it to go through the hassle of entering all of the attributes on the command-line. Again because it generates tests, fixtures, migrations, views etc for you.
For usage: script/generate scaffold_resource --help

-x and -c

If you pass the "-c" flag to a generator (script/generate whatever), it will automatically svn add the generated files. This isn't necessary, but it's a really useful thing so you don't have to hand-add every file individually.

There is a similar flag (-x) for when you install plugins. This will automatically add the plugin to the project via svn:externals.

Plugins

Don't use a plugin unless it does exactly what you want

It's so easy to write functionality in Rails that there's no point in using somebody else's plugin unless it does *exactly* what you want it to do.
Especially don't bother using a plugin if you have to hack it to pieces and pull out half the useful functionality. You are just creating a maintenance nightmare in your future.

That being said, if a plugin has half the functionality you need... it's a great place to learn the bits you need to use.


DB and migrations

Use migrations

But you weren't going to hand-hack the db anyway were you?
If you don't use migrations, then capistrano can't deploy the changes automatically. That only means more work for you - and means we can't auto roll-back if something horrible goes wrong.

Don't go back and edit old migrations!

I now have a policy that states "migrations only go forwards". This comes from several bad experiences where a developer (sometimes me) has gone back to edit an old migration and totally locked up the system for everybody else.

You may know that a deployment requires you to "migrate down then back up again". But sometimes that either gets forgotten, or is actually harder to do in practice than in theory eg if you're using capistrano to auto-deploy your site, it's harder to get it to do twisty migrations than just to "move forwards one".

Besides, which, it's a hassle for you to tell everyone on the dev team "before you do an svn up, migrate down to VERSION X, then svn up, then migrate back up again"... what if somebody doesn't want to lose their existing data? what if they're an svn-junkie and have already done an svn up before you get to them? what if somebody is not in today and you have to catch them before they come in tomorrow?

It's way too easy to get your bodgied-up back-hacked migrations into a wedged state where a new person cannot back up or go forward, and often cannot even start from scratch (eg from a fresh copy of a database). This becomes a serious problem for you if you ever have to change servers (or set up a new testing server or whatever) and have to run all the migrations from 0 to get the db ready.

Even if it all runs as you expect, it's three extra steps to remember than just "rake db:migrate", which raises the spectre of Human Error.

It's annoying to make it work, prone to error and bloody awful to fix when it breaks. All in all much easier just to write a new migration to change the existing stuff.

Extra migrations don't actually hurt you - they really don't take up that much more space in your SVN repository - they're worth having them around for the knowledge that anyone, anywhere will have a completely up-to-date system even if they start from complete scratch.

Add messages to data migrations

All of the basic ActiveRecord migrations tend to spit out their own status messages. Data migrations don't, and it's often useful to know what your migration is doing - especially if you've got a long data migration that would otherwise sit there in silence, looking like it's hung your machine.
So drop in a few status messages eg say "migrated #{orders.size} orders, moved widgets over to new wodget style"

Don't add excessive status messages to migrations

Obviously the above messaging can go too far. Don't write out a line for every piece of data you migrate or you'll get heaps of scrolling text rather than an idea of which migration we're up to - that useful data can get lost in the noise. Just a status message showing how much has been done or what stage we're up to in the migration will do.

Use "say" in migrations

Instead of using "p" use "say" - it gets formatted nicely.


MVC-specific idiom

If you are unsure about what MVC is all about - I suggest you do a bit of basic research on the subject. It's really important to keeping the code clean in a Rails site. Rails is completely founded onthe principle of keeping the MVC layers separate - if you begin tying them together, your run the risk of causing the code to become brittle - losing the flexibility that is the reason why you're using Rails in the first place.

Basic MVC principles

Keep view code out of the model

Your model should be independant of your views. A widget doesn't need to know whether it's being displayed as html, XML or via a screen-reader. It is an independantly-addressable object that contains "data and some knowledge of how to maniupulate that data".

Despite this I consistently see things like methods named "show_edit_link" which apparently tell the html whether this model's edit link should be visible on the html list page. That sort of logic should be in the view only. The model can have methods that, say, determine if they should be "editable" by the current user... and the code that determines if the edit link is displayed could check that method, but should remain inside the view.

If there is common code in your views - then it can be pulled into the helper file for that model. If it's used across multiple models, then it should go in application_helper.rb. It should never appear in the model.

The other common booboo I see are "display" functions that include html (eg "format field X to display nicely"). This is also very wrong! It means that, later, when you want to change html styling across the entire site you have to dig around inside every file in the site to change it... rather than simply changing the views - which is How It Should Be. From having to do just such a nightmareish site-upgrade, I can tell you that the fewer files you need to change - the easier your life will be in the long run.

All view code should remain in the views - with commonality pulled out into helpers.

Data manipulation code should go in the models - not views or controllers

Ok, so you have a report that you want to display that pulls data out of your widget... and your report view pulls it out bit by bit and does some calculations on it - then displays it.

So what happens when your client asks you to now do that report in PDF format... or CSV as well?

If all your data-manipulation methods are on the model itself, then you only need to pass the model object into the view and the view can then pull the data out in the same way for each view style . The alternative is to have a messy set of views/controllers where the same code is duplicated, violating the DRY principle and adding to your future maintenance headaches.

All data-manipulation code should remain on the model that owns the data.

Model idiom

Group relationship commands at top of model

One of the most important and useful things to know about a model is what it is related to - eg the "belongs_to"/"has_many" (etc) relationships.
If these calls get scattered throughout the model object, they will get lost amongst the noise and it will be hard to tell whather a given relationship has been declared...
Thus it's Good Practice to group them all together - and to put them right at the "top" of the class. eg:

class Widget < ActiveRecord::Base
  belongs_to :widget_class
  has_many :widget_parts
  has_one :pricing_plan

  #... rest of the model goes here
end

IMO this goes also for composition objects, "acts_as_<whatever>" declarations and auto_scope declarations - all of which declare the "shape" of the model and its relationships with other objects.

Group validations

The second-most important group of information about a model is what consists of valid data for the model. This will determine whether or not the data is accepted or rejected by ActiveRecord - so it's useful to know where to find it. So again, do not scatter it all about the model - put it near the top... preferably just below the "relationship" data. EG:

class Widget < ActiveRecord::Base
  belongs_to :widget_class
  has_many :widget_parts
  has_one :pricing_plan

  validates_presence_of :widget_class, :pricing_plan
  validates_numericality_of :catalogue_number, :integer_only => true
  def validate
     errors.add_to_base "should have at least one part" if widget_parts.blank?
  end

  #... rest of the model goes here
end

Validations should obey the DRY principle...

Please save my eyeballs from having to read the same thing over and over, and obey the DRY principle. You can write:

# please use:
validates_presence_of :field_one, :field_two, :field_three, :field_four
# instead of:
validates_presence_of :field_one
validates_presence_of :field_two
validates_presence_of :field_three
validates_presence_of :field_four

You can also use "validates_each" if you have several things that perform ostensibly the same validation... but it isn't an included one. eg:

# please use:
validates_each [:field_one, :field_two, 
                :field_three, :field_four]  do |model, attr, val|
  model.errors.add(attr, "should be a positive number") unless val.nil? || 0 <= val
end
# instead of:
def validate
  errors.add(:field_one, " should be a positive number")  unless field_one.nil? || 0 <= field_one
  errors.add(:field_two, " should be a positive number")  unless field_two? || 0 <= field_two
  errors.add(:field_three, " should be a positive number")  unless field_three? || 0 <= field_three
  errors.add(:field_four, " should be a positive number")  unless field_four? || 0 <= field_four
end

Read the section on validations in the book!!!

There are lots of really useful standard validations available in Rails... PLEASE check for whether one exists before writing your own hack into "def validate"... you might be surprised at how many are provided absolutely free of charge!
One of the most useful being "validates_each"... so you don't have to repeat the same thing over and over and over... remember that DRY principle?[1]

Another is "validates_format_of" which allows you to perform several validations at once - and give a consistant error message eg checking if a percentage falls within the range (0-100) with an optional percentage sign:[2]

  validates_inclusion_of [:my_percentage, :your_percentage], :in => 0..100, 
                       :message => "should be a percentage (0-100)"
  validates_format_of [:my_percentage, :your_percentage], :with => /^(\d{1,3})%?/, 
                       :message => "should be a percentage (0-100)"

If a validation doesn't refer to a single field... put it on base.

When it refers to more than one field, feel free to use

errors.add_to_base <whatever>
don't do something silly like: errors.add(nil, ...)
That being said - if you can squeeze it onto a field, please do - as it really helps the user to quickly find where the problem is in their form.

Give the poor users some useful feedback when validating

Don't tell them "Your weights should add up to 100"... tell them "your weights add up to 82%, your weights need to add up to 100% to be valid". It actually helps them see exactly what's missing.

When you update your validations, update your fixture data

This one has got me a couple of times. The problem is that fixtures are loaded into the database without going through validations - so your test data could be wrong - even when rake test passes flawlessly :(

Name your callbacks and declare them as handlers

If you have a number of callbacks (eg before_save/after_create hooks), it's better if you name them as a method, and just refer to them near the top of the model. eg:

# instead of:
class Widget < ActiveRecord::Base
  belongs_to :widget_class
  has_many :widget_parts
  has_one :pricing_plan

  validates_presence_of :widget_class, :pricing_plan
  def before_save
     # adds default pricing if none was specified
     pricing_plan ||= PricingPlan.find_default
  end
  #rest of model here
end
# try using:
class Widget < ActiveRecord::Base
  belongs_to :widget_class
  has_many :widget_parts
  has_one :pricing_plan

  validates_presence_of :widget_class, :pricing_plan
  before_save :add_default_pricing

  # handler to add default pricing if none specified
  def add_default_pricing
     pricing_plan ||= PricingPlan.find_default
  end
  #rest of model here
end

This example is too simple to easily show it, but there are a number of reasons why this is a good idea.

  • Firstly, you've named the method, which gives you an idea of what it's for = easier to maintain.
  • Secondly, if you need to add multiple of the same hook, you don't need to mash them all into the same callback method - which also aids readability.
  • Thirdly, by keeping each call as a single line, you can list them all near the top of the model class without taking up heaps of room (especially annoying in the case that the methods start getting really long). This helps you to quickly see what is and is not being done, without having to read through the details of the methods... or, even worse, seeing the first "after_create" hook and thinking it is the only one, when another is hidden later in the class.
  • Fourthly, if the method ever gets more complicated, or has to change, your method is encapsulated - the call to the method is independant of any other callback handlers of the same type.

Use Rails helpers for simple find calls

Rails comes with lots of helpful convenience functions. Read the book and learn about them! The find helpers are a good example. Something like the following is an excellent candidate.

  # instead of
  thing = Widget.find(:first, :conditions => "name='wodget'")
  # use the find_by_<whatever> method
  thing = Widget.find_by_name('wodget')

It shortens the code and makes it clear exactly what you're trying to find.

View idiom

Use -%>

Make sure you put a minus sign before closing your %>. This reduces whitespace in the resulting html - which makes it easier for us to read when we need to diagnose a display problem. eg use

<%= render :partial => :foo -%>
<% if some_condition -%>
   <div id="bar">
       <%= bar_fields -%>
  </div>
<% end -%>

h( x-site-scripting )

See this topic in the RailsSecurity section under XSiteScripting

Apply the DRY principle

Templates can have common code too - even across resource-boundaries.
Eg pagination links, or a common header/footer. Common view code should be factored out into a partial template and can be shared by a single resource-type, or across multiple resources.

Shared partials go in /app/views/shared

If a partial is to be used in a common layout, or across multiple resources (eg a common header for all resources reached after login), then the partial should be stored under the "/app/views/shared" directory.

Suppose you have a partial named "_common_footer.rhtml" in the shared directory. You can access it by referring to the partial thus:

<%= render :partial => 'shared/common_footer' -%>

Don't write a string inside a code snippet

This is silly: <%= " | " -%>
In a template, you can just type the pipe (or whatever) without the extra FrouFrou around it.

Controller idiom

Never, never, never use @params or @flash

These were deprecated some time ago. Instead use the similarly named methods: params or flash
Incidentally - you should be reading and taking heed of the warnings that the script/server is giving you about these things...

redirect when moving on

When you are moving on to another stage in the process, you should use redirect. eg you have successfully updated a page and are now moving to the index page.
When you are coming back to the same page, you can use render (eg the user entered an error and you are redisplaying the 'edit' page).

The reason we do this is that a redirect will change the URL in the address bar to the new page. If you just render a different page-action, the user will still have the URL of the previous page, even though they are now looking at a different page. This is Bad because if the previous page was, say, a form-POST... and the user clicks "refresh"... Bad Things will happen. Whereas if you redirect, then the user will have the URL of the new page and it's perfectly fine for them to hit refresh to get the page they're on.

Make private methods obvious!

Private controller methods aren't visible as actions. It can be a problem if somebody doesn't notice your little word "private" and starts to add methods below it without realising - and then wonders why Rails can't find their shiny new action.

So make sure that it's obvious and difficult to make the mistake.

Firstly, put private methods only at the bottom of the file!
Why bother going through spaghetti code with continual calls to private then public then private again... etc and having to figure out exactly what level of visibility we are right now? sort them and put the private ones all together at the bottom.

Secondly, use a line like this:

    private ###################################################################################

To make it painfully obvious to all who follow that below this line thar be dragons...


Testing idiom

All testing

Learn your basic assertions

Another "please read the book" request - there are lots of useful assertions already written for you. Using the appropriate ones makes your life easier as the error messages are more appropriate/meaningful for your test case. I especially find it irritating to see things like assert foo == bar when assert_equal not only has a better error message, but also means you're unlikely ever to accidentally use only one '=' - causing the test to silently pass when it should have failed.

Put the expected value first in assert_equals

The error message reads better that way.

Don't cram multiple tests into a single test case

Do not write generic tests with a dozen things to test all in a single test case.
Example shown below. Note - the real code example I took this from actually had another 6 tests crammed into it...

There are (at least) two reasons for not doing this:

Firstly - if the first test here breaks... it doesn't keep running the other tests. If it breaks, you can only tell that the first test isn't working... if you split them out into individual tests, rake will keep running the other tests - this is extremely helpful as you will then know whether all the validations are failing - or just that one.

Secondly - naming tests to mean what they're testing actually helps you find out what's going on when it breaks. A generic "test all sorts of stuff" name doesn't help you much, but a "test that widgets can't be created if the size is negative" does. You won't be thinking about this when you're writing that test case (and are fully immersed in the code you're testing), but six months later when something breaks your test... you will want to know what it's actually breaking and every bit of help (no matter how small) counts.

# don't use:
  def test_create_should_not_accept_invalid_parameters
    assert_no_difference Widget, :count do
        wid = create_widget(:vat_percentage => -1)
        assert wid.errors.on(:vat_percentage)
    end

    # invalid price range
    assert_no_difference Widget, :count do
        wid = create_widget(:price_range_min => -1)
        assert wid.errors.on(:price_range_min)
    end
    assert_no_difference Widget, :count do
        wid = create_widget(:price_range_max => -1)
        assert wid.errors.on(:price_range_max)
    end
    assert_no_difference Widget, :count do
        wid = create_widget(:price_range_max => 100, :price_range_min=>200)
        assert wid.errors.on(:price_range_max)
    end

    # invalid size range
    assert_no_difference Widget, :count do
        wid = create_widget(:size_min => -1)
        assert wid.errors.on(:size_min)
    end
    assert_no_difference Widget, :count do
        wid = create_widget(:size_max => -1)
        assert wid.errors.on(:size_max)
    end
    assert_no_difference Widget, :count do
        wid = create_widget(:size_max => 100, :size_min=>200)
        assert wid.errors.on(:size_max)
    end
  end
# instead try:
  def test_create_should_fail_on_negative_percentage
    assert_no_difference Widget, :count do
        wid = create_widget(:vat_percentage => -1)
        assert wid.errors.on(:vat_percentage)
    end
  end
  def test_create_should_fail_on_negative_min_price
    assert_no_difference Widget, :count do
        wid = create_widget(:price_range_min => -1)
        assert wid.errors.on(:price_range_min)
    end
  end
  def test_create_should_fail_on_negative_max_price
    assert_no_difference Widget, :count do
        wid = create_widget(:price_range_max => -1)
        assert wid.errors.on(:price_range_max)
    end
  end
  def test_create_should_fail_on_backwards_price_range
    assert_no_difference Widget, :count do
        wid = create_widget(:price_range_max => 100, :price_range_min=>200)
        assert wid.errors.on(:price_range_max)
    end
  end
  def test_create_should_fail_on_negative_min_size
    assert_no_difference Widget, :count do
        wid = create_widget(:size_min => -1)
        assert wid.errors.on(:size_min)
    end
  end
  def test_create_should_fail_on_negative_max_size
    assert_no_difference Widget, :count do
        wid = create_widget(:size_max => -1)
        assert wid.errors.on(:size_max)
    end
  end
  def test_create_should_fail_on_backwards_size_range
    assert_no_difference Widget, :count do
        wid = create_widget(:size_max => 100, :size_min=>200)
        assert wid.errors.on(:size_max)
    end
  end

Group tests in a meaningful way

Test files start out with only a handful of tests... but this rapidly changes. Very soon we get vast numbers of tests covering all sorts of actions.
To reduce the potential maintenance nightmare, group related tests together and put in a comment to indicate why they are grouped together. Any new test should be either put with related groupings or a new grouping created for it.

As an example: the most common set of tests revolves around resource CRUD - in which case it makes sense to group tests with related CRUD actions (eg "edit" and "update") near one another.
Another example is for the "user" object. users go through stages (process of creating a new user, the activation process, CRUD for an ordinary/active user, then destroying/reviving a user), so tests in these stages are grouped together.

If you're using rspec or shoulda, you are provided with a natural grouping mechanism (describe and context respectively). Use that to your advantage and make your future life easier.

Use named fixture records

Don't use bare fixture ids in your tests (eg get :edit :id => 1), instead use named references (eg @quentin.id).

This is a bit of a gotcha as the automatically-generated tests often use bare ids. However, these should be avoided where at all possible.

There are three main benefits:
Firstly, it makes the code more readable, with fewer magic numbers and more meaningful variable names.
Secondly, if we must reorder the fixture records (and thus change the ids) - at least we know which record we really wanted and we don't need to scour the tests looking for the ids to change.
Thirdly - if you're using auto-generated ids (rather than specifically passing them in) you won't know the id - and assuming that it's 1 sets you up to fail.

Similarly, don't use magic strings - especially for things like testing login/authentication. This will eventually bite you. It's much better to specify @quentin.login, @quentin.email (or whatever) so you don't have to go through your tests updating at a later time.

Don't forget to reload

The number of times I've been caught out by this... when you change something and are testing that it got saved into the database correctly, make sure you use @foo.reload. It's especially important if you're checking that you get the right validation errors on your ActiveRecord... you then want to test that it *didn't* change in the database too.

Common functions can be made into methods

If you find yourself performing the same action over and over, don't forget that you can DRY it up by putting it into a common function at the bottom of the test file.

  def setup
    @default_widget_hash = {:name => 'My widget', :size => 42, :wodget => wodgets(:my_wodget)}
  end
  def test_should_create_widget
    assert_difference Widget, :count do
      w = create_widget
      assert !w.new_record?, "should not have errors: #{w.errors.full_messages.to_sentence}"
    end
  end
  def test_widget_should_require_name_on_create
    assert_no_difference Widget, :count do
      w = create_widget(:name => nil)
      assert w.errors.on(:name), "Should have required name"
    end
  end
  def create_widget(options = {})
    Widget.create(@default_widget_hash.merge(options))
  end

Really common tests can be made into assertions

If you find yourself testing the same thing in multiple unit/functional tests, you can DRY that up too it up by putting it into the test helper as a new assertion. Don't forget to name it "assert_X" (where X is descriptive).

Make sure you name it starting with "assert_". This convention allows you to see which methods will have assertions in them (and thus possibly cause your test to fail), as opposed to general methods that simply set-up/tear-down information.

  # Assert that the handed object (thing) has no errors on it.
  def assert_no_errors(thing, msg = "")
    fail("should have passed an object") unless thing
    e = thing.errors.full_messages
    assert e.empty?, "Should not have error: #{e.to_sentence} #{msg}"
  end

Unit tests

Test your CRUD actions

It may seem like overkill to test CRUD here *and* in functional tests, but this is the place to test validations and any things that will affect basic CRUD-actions (eg if you have a method that stops your Widget from being deleted after it's been submitted for approval... then you need to try deleting a submitted one and make sure it fails!

If you only leave it up to your controller tests to check this stuff, then you are getting your data back filtered through one more layer of abstraction and will have to sort out whether it's a model-bug or a controller-bug. It's better to be sure of your model before adding an extra layer on top. It really makes your life easier in the long run!

Sadly scaffold_resource no longer writes these tests for you. :P

Test your validations

Each validation should be tested by trying to create your model object with that one set to nil. Trust me, it's worth it in the long run!

Functional tests

Test your validations

Create a functional test for each validation - again, try to create an object, nulling out the attribute-under-test. I know this seems to double-up on the unit tests - but what you're testing here is that the model is correctly getting back an error on the required attribute. This will be what is displayed on the user's screen - so they can fix the error before moving on. You also need to test that the template is still on the "edit" page. If it's not, then all the ActiveRecord Errors will be in vain as your user will never get to see them.

Test authentication-levels

If you have different levels of authentication (eg administrators get to see everyone's stuff, but you can only see your own stuff) then you must test both ways... test that you can see what you should see, but also test that you can't see stuff that you shouldn't see! You don't want your users peeking into your admin accounts just because you missed the "unhappy path" tests.
...and don't forget to test what happens when you're not logged on too.

Tests need more than just "assert_response :success"

Unfortunately, "assert_response :success" only checks that *something* happened - not that it was the right something.

To be more precise, "assert_response :success" asserts that rails successfully returned an "HTTP 200" response (as opposed to a 404 or a redirect). Unfortunately it does not check that you received the response you were expecting... (though I wish it were that simple) - sadly you do have to actually check that it not only returned a response - but that it was the correct response with the correct variables etc.

In the case of a "get" request - you should check that we are rendering the correct template, and that the "assigns" variables have been correctly populated.
In the case of a request that does something - check all the variables afterwards to make sure they changed in the way that you expected (eg that the variables actually were updated).


Notes

[1] Why is it that the DRY principle is the one I seem to have to harp on the most? Repeated endlessly reaching a pinnacle of irony.

[2] check my earlier post on "validates_percentage_of" for this example in context.

Saturday, 6 June 2009

dynamic_parameters

The params hash in Rails is special. It contains not only the query/post parameters that the user asked for, but also a set of special parameters such as controller and action...

but there are times when we want to know *just* what the user has *actually* asked for (without Railsy interference, however helpful).

I was quite surprised to find that there isn't a function in Rails that actually allows me to specify these. So I wrote one.

This helper goes into application_controller.

  # helper to fetch out only the dynamic parameters of the given request (ie
  # anything in the query string or POST parameters - but without the rails
  # "special" routing parameters.
  def dynamic_parameters
    @dyn_params ||= request.request_parameters.merge(request.query_parameters).with_indifferent_access.reject {|k,v| request.path_parameters.has_key? k }.symbolize_keys
  end

arrayify!

Following from my post: Array.acts_like_array?, I can now implement arrayify without using is_a? Array or respond_to? :[] && !is_a? String

class Object
  def arrayify
    self.acts_like?(:array) ? self : [self]
  end
end

So we can now do: thing = thing.arrayify or: do_something(thing.arrayify)

Now, what I really want to do is thing.arrayify!, but I'm not sure how to implement that. Any ideas?

Wednesday, 3 June 2009

Array.acts_like_array?

I've recently discovered the joys of: acts_like?, and was astounded that there hadn't been a simple extension to Array. So here's the monkeypatch:

# extension to allow us to check if an array is an array...
# This feeds the Object.acts_like? method
class Array
  def acts_like_array?
    true
  end
end

Monday, 1 June 2009

Help? ActiveResource common config

So we have a system that communicates with a "local" API - ie all the objects in our system are on another database, which we access via an API[1]. A very large number of our objects are accessed using HyperactiveResource[2].

Because so many of our resources are hosted on the same system - it makes sense to put the common access-information (eg "site", "user" and "password") into a commonly-accessible spot.

So I tried just plonking something like:

module HyperactiveResource
  self.site = API_SITE_ADDRESS
  self.user = API_SITE_USER
  self.password= API+SITE_PASS
end

into config/environment.rb... which sort of worked... but seemed to cause some clashes with the actual HyRes objects later - because they aren't loaded until *after* config.environment.rb

Maybe I'm just being thick, but I wasn't able to figure out how to get this stuff to be pulled into every HyRes object without it happening before I've really defined HyRes... and it would then get really upset when I tried to "redefine" HyRes when the same was loaded afterwards.

My quick fix was to define these using the following instead:

module ActiveResource
  class Base
    self.site = API_SITE_ADDRESS
    self.user = API_SITE_USER
    self.password= API+SITE_PASS
  end
end

Which means we don't get a HyRes conflict, but I'm sure there must be a better way... any suggestions would be welcome!

Notes:
[1] There's various business-reasons for why this is. Mainly - because the existing codebase is very large, and we only want to migrate to Rails one chunk at a time. This allows us to keep the existing systems running on the existing code-base, but add new functionality (or convert existing chunks) without disturbing the operation of that section.
[2] My extension to ActiveResource that actually works more like ActiveRecord (unlike vanilla ARes)

Thursday, 21 May 2009

Testing gotchas: nested logins don't work

*sigh* after a couple of hours futile hacking about I finally realised that my controller tests were failing because login_as cannot be nested. ie. if you have a context with a login_as @user1 in the setup... then put a context inside it where you login_as @user2... it won't actually do the second login.

So just move that nested context outside. Simples neh?

Thursday, 30 April 2009

Should have form fields (macro)

Quickie snippet form-macro to make sure that the page you're on has the given set of form fields. You can optionally pass in a model name for "form_for" style forms (because these are pretty common), but it also lets you just pass in straight field names (for when you check your submits and hand-crafted tags).

Stick this at the bottom of your test_helper.rb and use as below:
should_have_form_fields [:name, :email, :phone, :gender], 'user'

Note: should work for text inputs as well as radio buttons and checkboxes (which is why we check for name not id - also means you don't match the label by accident). This assumes you've used Rails' helper_methods to generate your form fields - or that you have included a name on every field.

class ActionController::TestCase
  # A macro to check that we should now have form fields for the given
  # columns.
  def self.should_have_form_fields(fields, model_name = nil)
    return if fields.blank? # short circuit
    fields = [fields] unless fields.respond_to?(:[]) # arrayify
    fields.each do |f|
      should "have field #{f} on form" do
        assert_select 'form' do
          if model_name.blank?
            assert_select "[name=#{f}]"
          else
            assert_select "[name='#{model_name}[#{f}]']"
          end
        end
      end
    end
  end
end

Dehumanizing Rails

So we have a string that has been "humanized" - and all the underscores have been stripped out and replaced by spaces, and it's been nicely capitalised (sorry, make that "capitalize"d). So how do we go back the other way?

Plonk the following into config/initializers/inflections.rb (or if you're using a legacy Rails, drop it onto the end of config/environments.rb).

module ActiveSupport::Inflector
  # does the opposite of humanize.... mostly. Basically does a
  # space-substituting .underscore
  def dehumanize(the_string)
    result = the_string.to_s.dup
    result.downcase.gsub(/ +/,'_')
  end
end
class String
  def dehumanize
    ActiveSupport::Inflector.dehumanize(self)
  end
end

Note: it will not reverse any speccy stuff like adding back any "_id"s etc, but it does mean you can do something along the lines of:
assert_equal "network_type", "Network type".dehumanize

Tuesday, 28 April 2009

Playing nice with XML and HTTP

So we've got a setup that has a remote API that we're accessing using HyperactiveResource (an extended version of ActiveResource). Now, I'm using Rails to simulate the remote API (for the purposes of testing), and I've come across some annoying behaviour.

One issue is that standard rails routing for a RESTful interface will direct a badly-constructed (or non-existent) URL to a real action... let me demonstrate thus:

Real member path: /users/1.xml Routed to: :controller => 'users', :action => 'show', :id => '1'
Real named collection path: /users/count.xml Routed to: :controller => 'users', :action => 'count'
Non-existent path: /users_party_on.xml Routed to: "Bad Request" handler
Non-existent path2: /users/party_on.xml Routed to: :controller => 'users', :action => 'show', :id => 'party_on'

If I called the last URL with curl, I'd expect to be routed to the "Bad request" handler and receive some sort of error-like http-status and an XML message explaining that no route exists or something similar... what I get instead is a horrible big *html* page telling me it couldn't find the user with an id of "party_on" (unsurprisingly).

So what do I want to have happen? I'd rather this stuff was caught in the router. It'd be nice if there were a way to tell the router that your :controller/:action/:id is only valid for a certain formatting of the :id field. If anybody out there on teh Intarwebs knows how to do that, please tell me now!

Unfortunately, it doesn't seem to do this... and in any case, the router/dispatcher also doesn't seem to return XML to an XML-request... it only seems to know how to handle HTML-based errors (by spitting back the public error pages[1]).

So instead, what I need is to return a "URL not found"-style xml error at the appropriate time.

Most of my controllers have a "find_" function on the member-functions (ie just @thing = Thing.find(paramd[:id])). Now, since the bad URLs tend to converge on the "show" action - this seems as good a place to put a bad-request filter as any. I'll also incorporate it with the 404-code that also seems missing when a doesn't exist (or is not accessible by this person).

So this calls for a helper-method as below, as a hack to fix this lack of proper routing.

  # convenience method for extracting the expected model name from the
  # controller name
  # Note: expects the model to be rails-standard eg "ThingsController"
  # should map to the Thing model
  def model_name
    self.controller_name.singularize.camelize
  end

  # use this to skip out early and return better http status codes for XML
  # requests.
  #
  def find_thing
    the_id = params[:id]
    # ids should be numeric. If they're not - we accidentally got through
    # the router with an unrecognised action - because Railsy named-routes
    # that *don't* exist, look like the "show" action with a bad id.
    if !the_id.blank? && !the_id.to_i.is_a?(Numeric)
      # skip out with a 400 early...
      respond_to do |format|
        format.xml do
          return render :xml => 'Error: URL not recognised', :status => :bad_request 
        end
      end
    end
    begin
      thing = model_name.constantize.find(the_id.to_i)
    rescue ActiveRecord::RecordNotFound
      # skip out with a 404 early...
      respond_to do |format|
        format.xml do
          return render :xml => 'Error: resource not found', :status => :not_found 
        end
      end
    end
    thing
  end

Due to how controller before_filters work[2], you should use this code thus:

class UsersController < ApplicationController
   before_filter :find_user, :except => [:new, :create, :index, :count]
   # actions all go here
   # ...
protected
  def find_user
    @user = find_thing
  end
end

Caveat: if you have non-standard naming of models/controller - the model_name.constantize will not work... so you may want to modify this to pass in an optional klass param.

Notes:
[1] and when exactly are they going to make these into templates so we can use the standard layout rather than hand-coding it for each one?
[2] ie, I'm too slack to figure out exactly how to do: @thingy = in a block passed to the before_filter command. Again - if you know how, let me know.

Monday, 27 April 2009

Testing ActiveResource

I've been banging my head against the wall that is ActiveResource for a while now. One big problem is actually getting the testing correct.

Take 1: HTTPMock

The supposedly sanctioned expectation is that you test it against the provided HTTPmock... unfortunately, this is great for testing that ActiveResource makes the correct remote calls eg, when you update a user that ARes POSTs to /users/123.xml... and that it reacts to a (pre-stubbed) 404 by raising a ResourceNotFound; but it doesn't allow you to actually test your model - eg to assert that when you update your user's login field and reload your model, that your user can now login with the new details...

AFAICS you can't test anything that is actually meaningful or useful to your business logic - which kinda defeats the point IMO.

Even with a lot of hacking, I couldn't get the kind of dynamic mock backend that I needed for any meaningful test-coverage.

Take 2: Mocking with Stump

Next up was to try to mock/stub out the backend functions appropriately. I'd heard some good stuff about the stump plugin and dutifully installed it to have a go.

Stump let me do much more dynamic testing. I could stub out the "create" and "find" functions and make it respond with a mock remote object that I could then further stub out the "save" and "update" functions... and after a while it seemed like I was stubbing out functions to return stubbed out stubs and it all got a little bit circular... and in many cases: really complex.

I also found that I got the code to work in the browser I'd often still have to spend ages getting the tests to work with the right set of mocks and stubs to patch all the ways that rails could leak out and try to call the "real" API. :P

I also noticed a few times where I'd somehow accidentally plastered over a bug with a stub by assuming the return-value. I only noticed this because the tests would run fine - yet trying the same operation in the browser revealed the bug.

Now how can I test if I can't rely on the test to actually test? My safety net suddenly has big gaping holes in it that I'll only find if I fall through them while testing by hand! This is such a step backwards that it's horrifying!

Then I hit a wall.

We have a user and during creation (and update) we need to check that the login is unique - which requires a find on the remote API. So in the unit tests we mock out the User object's "find" method to return an empty set... (ie no users were found that match this login field). But then (immediately after creation), we need to be able to find the user - we we re-stub out User.find to return the given user - otherwise the test fails.
... but when you're doing a functional test, all of the above operations are inside the atomic post :create call. You can't stub out one half, then stub out the other half of the operation partway through... because you can't get to the user object partway through the process... so creates were suddenly failing badly with seemingly no solution.

The code didn't do what I needed, and on top of this was tangled and messy and really too ugly. When it comes to rails, ugly code screams out for another solution...

Take 3: A real API

So, I turned to (what I see as) the lesser of two evils. I have created a test-API project that will simulate the real remote API with the models that are used by the (local) project. the test environment calls this API instead of the "real" one by using a defined constant for the site.

In setup I send the ActiveResource model a delete_all which will clear the remote db[1]. This simulates what Rails testing does anyway (ie clearing out the db before each test).

So now the project tests against a real, live-but-fake API that I can run in another window on my local machine. The tests run - they look pretty much the same as my ActiveRecord tests - which means I can check everything I need to check. It also means I can be assured that the code hasn't plastered over a bug with mocks and stubs.

Conclusion:

Pros:

  • Real end-end tests
  • Dynamic remote object instead of static mocks (means you can test that values are changed and that reloading returns what you expect)
  • Fewer assumptions means the test-code is less likely to be buggy - ie you're more likely to be actually testing what you think you're testing.
  • Easier to test as you don't have to mock out things that will return things that you've mocked...
  • Above leads to a more natural Railsy test syntax.

Cons:

  • Keeping two projects in synch
  • extra development time of the fake API
  • Will need to make sure the API a) exists and b) is running before you can run tests
  • Have to verify that the mock API does what the real API does.
  • Tests run slower as there's the (local) network turnaround time to consider.

In my mind, the pros outweigh the cons. YMMV

Notes:
[1] We're currently not using fixtures anyway, but it we were - I would then send the remote API a "please load up your fixtures" command.

Wednesday, 22 April 2009

should_set_the_flash better

should_set_the_flash_to doesn't do quite what I'd like. I want to: a) specify which level of flash should be set (ie make sure it's a notice and not an error) and b) not care exactly what notice has been set.

IMO tying your tests down too hard to ephemeral strings is annoying... what is somebody changed the wording from "user created" to "thanks for signing up"? You have to change your test cases for that? :P So, being able to just enter "anything" should be possible.

Note - it piggybacks on the existing flash shoulda macro, and thus allows you to use it as before (pass 'nil' for level if you want - but it's better just to use the original if you need it.

So, here's my updated code. Dump it into the bottom of your test_helper file.

class ActionController::TestCase
  # Make sure that a message has been set at the given flash level
  # you can test that a notice has been posted, but no error thus:
  # should_set_flash :notice
  # should_not_set_flash :error
  def self.should_set_the_flash(level = nil, val = :any)
    val = /.*/i if val == :any
    return should_set_the_flash_to val unless level
    if val.blank?
      should "have nothing in the #{level} flash" do
        assert flash[level].blank?, "but had: #{flash[level].inspect}"
      end
    else
      should "have something in the #{level} flash" do
        assert !flash[level].blank?, "No value set of given flash level: #{level}"
      end
      should "have #{val.inspect} in the #{level} flash" do
        assert_contains flash[level], val, ", Flash: #{flash[level].inspect}"
      end
    end
  end
  # convenience method for making sure nothing has been set for the given
  # flash level
  def self.should_not_set_the_flash(level = nil)
    should_set_the_flash level, nil
  end
end

[Edit: 22-Apr-2009] : ADDED a convenience method for "not_set" and updated "set" to take advantage of this. Also updated comments to include an example of use

Thursday, 16 April 2009

rails gotchas: nested transactions

So I've been adding some new plugins for testing that I've never tried before, in this case: shoulda and stump, and found that stump just isn't playing nice with SQLite :P

The problem only occurred when I tried to declare a proxy! function on an existing model. ok, I didn't get around to trying everything, so maybe it occurs on other things too, but it seemed to be happy with stub and mock.

The code would simply fail with the following error:

SQLite3::SQLException: cannot start a transaction within a transaction
<insert useless backtrace here>

It's been driving me crazy for the last few days trying all sorts of ways to get it to work. I almost abandoned transactional fixtures entirely before I finally found this snippet at the very *bottom* of the ActiveRecord::Transactions page in APIdock

"Most databases don’t support true nested transactions. At the time of writing, the only database that we’re aware of that supports true nested transactions, is MS-SQL."

So the problem is not transactions... but SQLite's lack of support of proper nested transactions.

The solution: Install MySQL.

It's annoying to have to go back to creating a MySQL db/user and granting rights before I can run my tests - but if it makes the test code work, then it's worth that extra step.

SQLite is nice, but it's ability to blow away the db by just doing a quick rm isn't worth not being able to properly test my code. Besides - the real db is on MySQL anyway, so I figure I might as well.

Thursday, 9 April 2009

Rails gotchas: shoulda not_allow_values_for

If you're using should_not_allow_values_for and getting a failing test something in the lines of:

Failure:
test: User should not allow email to be set to "b lah". (UserTest)
    [/usr/lib/ruby/gems/1.8/gems/thoughtbot-shoulda-2.10.1/lib/shoulda/assertions.rb:56:in `assert_rejects'
     /usr/lib/ruby/gems/1.8/gems/thoughtbot-shoulda-2.10.1/lib/shoulda/active_record/macros.rb:174:in `__bind_1239266378_671612'
     /usr/lib/ruby/gems/1.8/gems/thoughtbot-shoulda-2.10.1/lib/shoulda/context.rb:253:in `call'
     /usr/lib/ruby/gems/1.8/gems/thoughtbot-shoulda-2.10.1/lib/shoulda/context.rb:253:in `test: User should not allow email to be set to "b lah". ']:
Expected errors to include "is invalid" when email is set to "b lah", got errors: email is too short (minimum is 6 characters) ("b lah")email should look like an email address. ("b lah")

You can see in the above test that the email does have an error on it - but the error message is not the default. shoulda specifically checks for the error message - and if you don't have the default, then you need to pass it in thus:

should_not_allow_values_for :email, "b lah", :message => Authentication.bad_email_message

You shoulda test your plugin!

So, lets assume you have a plugin "acts_as_teapot" that you want to include in some of your model classes. You've written a number of useful methods for teapot-like models to use, and now want to check that the models that are implementing acts_as_teapot actually are able to make use of the full spectrum of teapotly functionality.

You could write a set of asserts that you can tell your plugin's users to "please include these asserts/tests in your model classes"... but it's not DRY, and the users might miss one, and they'll get out of date real quick... what you want is kinda one big assert they can just put in once and that calls some library on the plugin itself (so it keeps up to date with the latest plugin code).

Luckily, shoulda is here to save the day! You can create a big context full of all the right tests and save it in the plugin file itself. This will be drawn in by the model at the time it's included. Then the user just has to call a single "shoulda" and it's all done for you.

Now, I was against shoulda for a long time - mainly wondering why anybody would use a half-arsed version of rSpec if they didn't actually want rSpec... but for me, plugin-testing is the killer-app that forced me to re-evaluate shoulda, and so far it actually looks ok. :)

So, to the code...

Plugin code

module Acts
  module Teapot
    def describe_me
      "short and stout"
    end
    def tip_me_over
      "pour me out"
    end
  end
end
class Test::Unit::TestCase
  def self.should_act_as_a_teapot
    klass = model_class

    context "A #{klass.name}" do
      setup { @new_klass = klass.new }

      should "respond to teapotly functions" do
        [:tip_me_over, :describe_me].each do |f|
          assert @new_klass.respond_to?(f), "#{klass.name} should respond to the function: #{f}"
        end
      end
      should "be short and stout" do
        assert_equal "short and stout", @new_klass.describe_me, "#{klass.name} is a funny-looking teapot."
      end
      should "pour me out" do
        assert_equal "pour me out", @new_klass.tip_me_over, "#{klass.name} doesn't make very good tea!."
      end
    end
  end
end

and testing the model:

class MyTeapot < ActiveRecord::Base
  acts_as_a_teapot
  #...
end

class MyTeapotTest < ActiveSupport::TestCase
  fixtures :my_teapots

  # plugin contexts
  should_act_as_a_teapot

end

There's a real-world example in the paperclip shoulda test

rails gotchas: assert_raises a syntax error

I found that assert_raise was causing a syntax error of the form:
syntax error, unexpected '{', expecting kEND (SyntaxError)
For a fairly simple test:

should "remove it from the database" do
  assert_raise ActiveRecord::RecordNotFound { User.find(@uid)}
end

Looks like it's getting confused about nesting. the solution? parenthesise your Error thus:

should "remove it from the database" do
  assert_raise(ActiveRecord::RecordNotFound) { User.find(@uid)}
end

Monday, 6 April 2009

rails gotchas: restful_authentication not 2.0 compliant?

restful_authentication[1] is a great plugin, but the standard svn version is showing it's age. It's a whole month or so out of date... which is, of course, an eternity in the fast-paced world of Rails.

Luckily it's actually just moved home. You can find the up-to-date version on github

[1] Formerly known as acts_as_authenticated

rails gotchas: HttpMock not enough variables

A quickie for my own remembrance. I'm currently setting up HttpMock to test ActiveResource. I'd set up a few extra "routes" for it to mock out and kept getting the error below:

NoMethodError: undefined method `size' for :not_found: Symbol

For the code line:

mock.get    "/users/#{uid}.xml", {}, :not_found

The fix was pretty simple - I'd just accidentally missed out the "nil" for the body - ie it was actually an ArgumentError - but not getting picked up. ie, the code should have been:

mock.get    "/users/#{uid}.xml", {}, nil, :not_found

Thursday, 2 April 2009

ActiveRecord::Validations in ActiveResource

The holy grail for ActiveResource users is for ARes to actually behave like ActiveRecord. In theory, ARes is just like AR, but in practice it's only kinda, sorta like AR... but missing a few bits that really seem to make all the difference.

ARes is still missing fundamental functionality that we have all grown to know and love... It all looks alright on the surface, but you can't help but notice the giant glowing absence the moment you decide to hide your models away in a Web service and then try to use ARes to implement a Railsy front-end.

Needed functionality includes:

  • Associations (ie has_many/belongs_to)
  • the usual suspects of callbacks (eg before_save)
  • safe-making your attributes (eg attr_accessible)
  • Widget.count
  • Actual conditions in finders (eg :conditions => {:name => 'Joe Bloggs'}
  • and the all-important Validations

I need my validations. The funky way rails handles AR errors is one of the things that makes Rails so special. I love to be able to just type validates_presence_of :foo and for everything else to Just Work.

ARes doesn't bother with them at all - and in that case I hardly see why Rails can call it AR-like when these are missing. Oh yes, sure, you can overload the validate method on your model object, but that seems very crude! Like having to hand-write your database connection code for each model. :P

All I can say is why can't I have validates_presence_of independently to the database connection? It's not really necessary - so why are Validations still locked inside the db-wrapper?

In my opinion, ActiveResource needs a lot of upgrading. Unfortunately, that looks like a fair bit of work... and we don't know how long that will take, and whether it will just be superceded once Merb merges with Rails.

Luckily we have an interim solution, in the form of a plugin called HyperactiveResource. It is fairly crude, but it seems to roughly give us an ActiveRecord-like interface that works fairly well.

Before I hit the code, the plugin already came with a lot of the currently-missing functions - rebuilt with ARes-style processing. They also had a rough implementation of Associations (not entirely AR-like, but getting there). I've been working on adding validations and validation-callbacks.

I can't say it works perfectly as I've just started working on it, as of this morning; but I've already got the basic validations working without falling over completely and I'm using the models auto-generated by restful-authentication to test it.

It's a start...

git gotchas: the wrong forking repository

So I'm pretty new to git, really, and am learning all the places that I can stumble and fall head-over-arse. Today's escapade is entitled "how to recover from forking the wrong repository in github". It's quite a simple recovery, and you'd think it'd be obvious how to recover from it... you'd also think that github would have a "how to" in the main help page...

This solution is based on having just made the mistake and wanting to just delete it and start over straight away. Obviously this doesn't work if, say, you've started making changes and want to keep them - you're on your own there.

The solution... delete the repository and start again

  1. Go to your version of the repository on github
  2. Choose the "Admin" tab (from the list at the top)
  3. Right down the bottom of the page is a section labelled "Administration". At the bottom of the box is a small, blue link labelled "Delete this repository".
  4. Click that link - then click "Yes" a few times to convince github that you really are sure.

Now you're done. The repository may not be deleted for a few minutes (it's a background process) and your dashboard may be cached so it may appear for a little while longer. Once it's gone you can go back and pick the correct fork of the repository that you want.

Tuesday, 31 March 2009

State of authentication

We're using restful_authentication for login - because it's a good standard set of tools. However, we didn't want activation for our system, as it's just a slow-down on the path to user kick-ass-ness, and we wanted out users up and running with a minimum of fuss. However what we *did* want, was statefulness (so we could suspend users).

When I ran the usual generator without the --include-activation flag, I figured that would be the trick...

Unfortunately, AR doesn't seem to grasp that you can have statefulness without activation - so the authentication/authorisation sections have an activation state (and set up an activation code and require that you provide it and activate your user... etc) regardless of your choice of --include-activation.

This is a bit annoying - especially because the Authorization module is now also hidden deep in the bowels of the AR plugin itself (rather than in the user model), which means I can't just delete the bits I don't want.

The plugin itself is not clear about how to go about setting up your own set of states (eg removing activation, or adding a "locked" state), so this is a quickie guide.

Setting up your own states

To begin with, to override your states, you need to copy the appropriate Authorization file into your lib directory eg:
cp vendor/plugins/restful_authentication/lib/authorization/stateful_roles.rb lib/my_stateful_roles.rb

Then require this file at the top of your user model eg:
require 'lib/my_stateful_roles.rb'

Open up this file and change the module name so you don't clash (in the bizarre case you need to use the standard set of states in some other model) eg:

module Authorization
  module MyStatefulRoles

Your user model will then have a reference to this module that you need to update from Authorization::StatefulRoles to Authorization::MyStatefulRoles

You now have a file that you can alter to your heart's content.

Coming soon: how to successfully remove the activation state from this file...

Monday, 30 March 2009

acts_as_state_machine 2.0

I spun up a new rails project and downloaded my favourite authentication plugin restful_authentication, only to find that all out-of-the-box tests break (oh noez!)

Clearly, I thought, restful_authentication is now just a millisecond behind edge rails... and therefore doesn't work properly anymore.

I quickly realised that no, it's just that the old-standard install (via svn) is no longer maintained - you have to move over to the github version.

This solves most of the problems, but I was still having a problem with one of the tests. It seems that acts_as_state_machine is similarly non-Rails 2.0 compliant.

Luckily, acts_as_state_machine has just a one-line fix to get it back up to scratch. It seems that the write_attribute method has been deprecated, and this causes a little bit of internal havoc for the state? style of methods (eg passive?, active? etc as used by restful_authentication).

The test cases kept giving me:

NoMethodError: You have a nil object when you didn't expect it!
The error occurred while evaluating nil.to_sym

To fix it, search for the string: "def set_initial_state", then replace the line (just below it):
write_attribute self.class.state_column, self.class.initial_state.to_s
with:
self.send "#{self.class.state_column}=", self.class.initial_state.to_s

I just want a toaster...

I was buying all my basic housey stuff at Tesco today (it's *that* kind of supermarket/superstore), but feeling overwhelmed... Being in a foreign country means all the brands mean nothing to you, so you don't know which ones do what, and you find there are many brands that do things you've never even heard of.

So you have two options. You can either pick things at random and hope you aren't going to end up with your washing machine spilling a river of frothy soap suds across the floor, or check every single item to make sure it does what you expect it to do.

When you have to read the back of every bottle of washing liquid and read all the dodgy manufacturers claims that mean nothing to you to decide whether you want "unscented" or "hypoallergenic" or "automatic" or "non-biological"... and whether that actually means anything or just another manufacturer's claim (like "full of wholesome goodness")... for every single item in the shopping trolley you find it's quite easy to take several hours just to do a normal shop.

By the end of that you come to a shining wall of toasters and find yourself weighing the benefits of "hi-lift" vs "variable-browning" and whether it's better to have "sneak peek" or "cool wall" technology and you get to the stage where you just want to scream "I want something that takes bread and spits out toast, that's *all* I want!"... :P

...and since when did toasters all become so *huge*. I'm missing the toaster I had in Sydney even despite having a dodgy Singapore plug that required a plug-adapter to use. It took four slices of toast and fit into nearly the same space as my new two-slice toaster that's tall and fat and looks like some kind of shiny metal alien pod. :P

and in case you ask "well why didn't I choose one that doesn't look like that - they all do, of course. Apart from the ones that are even *bigger* and look like some kind of industrial machine that will take your bread and throw it clear across the factory floor to the industrial jam-spreader...

...in the end, of course, I went with the one that had all of the above. Because I can't be sure I won't suddenly need "cool wall" technology (whatever that is)... I can always google it when I get home.

Saturday, 7 February 2009

Some day, my visa will come...

So in case you happen not to have actually spoken to me any time in the past two months, I'm in the process of moving to the UK. Of which, the major step is to get myself a UK work visa.

I'm (frankly) too old for a holiday working visa (and I don't want to go for that short a time anyway), so what I need is the Tier 1 general migrant visa. This will let me work in the UK for any company for up to 3 years before having to reapply, so it's pretty good.

However... apart from some very strict requirements the current wait for a visa of this kind is a bit on the long side.

The website tells us that there is a wait of 25 working days for any kind of visa[1], but this is simply not the case. The wait is currently averaging 45 working days. The following thread is an ongoing conversation by a large number of people frustrated about the current Tier 1 visa waiting times. It's been running constantly since October (when the Tier 1 was first introduced) and now runs at 47 pages of comments... this should give an indication of the current level of feeling surrounding the handling of this class of visa.

One of the chief grumbles is that the official website flat-out lies about the length of time it takes. The visa-office staff are aware of how long the wait times are currently averaging (I called and asked).

Moving internationally is a really large, life-changing event. It takes a lot of planning. It requires packing up your house and quitting your job, and long, tearful farewell parties... If we are told that it takes 25 working days, and then it takes another month... it means there are a lot of people left twiddling their thumbs and watching their bank balance go down - at an alarming rate if we're unlucky enough to have quit our job or are forced to stay in a motel after giving notice or selling the house.

Don't get me wrong - I don't mind that it takes 45 working days, but if I had known this from the start, I wouldn't have planned my life around 25 working days... and things would be going much smoother. As it is, I've been interviewing with several companies in the UK and having to tell them "looks like my visa *still* hasn't come through yet... can you hold off another 2 weeks"... which is a situation that *nobody* likes to be in.

It's not my fault that the visa simply hasn't come through, and that I have no idea of if or when it ever will... but I can't shake the feeling that I'm coming across as being flaky... which is a feeling that I really dislike.

The problem is compounded by the fact that there is literally no way in which one can find out the progress of a visa application[2]. There is no website, there is nobody I can call. I just have to wait and hope... and wait and hope some more... while every friend, relative and innocent bystander is going crazy alongside me and asking me when it's going to come... and I don't have any answer to give them!

Being my mother's daughter, I called anyway and asked - they said there was no way, but they confirmed it was taking 45 working days, so they know that the website is incorrect, and yet have done nothing about it.

That made me mad. Are they deliberately misleading people or are they simply too negligent to bother? We're not talking about the kiddies on their gap year holiday doing a bit of waitressing on the side (incidentally, those visa come through in a jiffy). These are the most highly skilled people in the country who are moving their lives around to get over there, and the official website gives an estimate that is 80% inaccurate... and they *know* it is! :(

The real kicker for me is that, because we can't see our progress, they have no obligation to get anything done at all! Now, I know the system has just changed and they probably have a stack of applications a mile high on their desk - so they're probably *not* just being lazy... but the lack of transparency certainly leaves the system open to abuse.

One final niggling point is that they recently published some world-wide stats for visa processing times... and it looks like countries such as the US have an average turnaround time of only 6 working days. I can imagine a time difference of maybe ten or even twenty days, but this is ludicrous. What could cause a discrepancy of over 600%!

If there's a shortage of staff... hire more?

[1] Apart from a permanent settlement visa, or an annoyingly vague description of a "non straight forward" visa - which apparently means a visa application that is incomplete... by which assumption the Tier one should be straight forward as long as you provide everything they ask for.
[2] at least until after 45 working days have past in which case you can pay $10 to get an SMS "progress report" sent to you (of which the aforementioned thread has several stories of it never arriving) and which, when it arrives, tends to say something vague along the lines of "In progress", and with no indication of how long it will take from there.

Wednesday, 31 December 2008

Link: A signature cadence

Too many websites (and applications) are soul-less, corporate entities that treat you like just another number.

Rands' article: A signature cadence shows how a human tone can make a big difference in how we view a site. Even tiny details like the copyright wording can help make an overall impression that you are actually a bunch of real people. I think we need more of that.

Tuesday, 30 December 2008

Squidoo and ssh tunneling

I've found squidoo - a site where you can set up pages called lenses that help direct people to all the relevant information about a specific topic.

I love it!

I've only just begun, but I think I'll slowly add pages on my favourite topics and see where it goes.

ssh tunnels R Us

To start with, I put up a page on ssh tunneling.

For years I've used ssh to login to a remote linux box to get work done. But recently I had to get through to the http server on my work machine - which was hidden behind a firewall. The best way to do that, it seems, is to set up an ssh tunnel and just port-forward the http port to my local machine!

In the process I found several useful (and less useful) articles on the subject, and it took me ages to crawl through each one to figure out which was which. It seems "ssh tunneling" means different things to different people...

So where does squidoo feature in all this?

In the squidoo lens, I gave an overview of ssh and tunneling - and then listed each of the articles with a brief description of the contents. In this way, the squidoo page acts as a differentiator for any people searching for the same thing in future. Hopefully it'll cut down the time people need to spend scanning all the possible google results they can find. They can zero in on an answer all the more quickly.

[Edit...] More squidoo...

So I've now gone and added a few more pages on my other hobbies.

For starters, I quite like brewing mead - which is a wine made from honey. I've been doing it for about 15 years now, and you can read more about it in my lens on mead, or you can jump straight to my (long) howto lens that will teach you how to make mead in three weekends.

Alternatively you can try your hand at home preserving with vacola - an ingenious system of bottling that creates a vacuum seal in the bottle to hold on the lid.

Stay tuned for more fun stuff as I go along.

Thursday, 25 December 2008

Link: 13 coding nightmares to avoid

Here is a great list of bad RoR practices - make sure you read the comments as there are some interesting exceptions raised.

Merry Christmas!

So I've been pretty quiet for the past month. That's because I'm running around like a mad thing getting set to make a big move to the UK. Won't be for another month yet, but it's consuming all my spare time.

Anyway, today's been quiet enough to sit down and actually write something so - merry christmas everybody!