Link: don't publish your secret keys on github

I probably don't really need to tell you that it's dangerous to publish secret keys on a public website... but which key am I talking about here?

  # this one...
  MyApp::Application.config.secret_key_base = 'fb636477f014943b9cf2d202f9da4349a580fbb9521ef38d8dc0111ae933a10bfec732ce7d563a854db1663ba00af1db21d06d6764d84a88d0a6286bcd2299e2'

It's one of those things that often gets overlooked...

  rails new
  git add .
  git push

Bam!

But if you're on heroku, rather than deploying to your own hosted site (where you can easily drop config files into a shared directory), how do you have a secret key?

Here's a great little gist that will show you how to set it up on an environment variable.

---

Update/gotcha

Don't forget, that if you put more than one of these into your .env file... you need to append to the file or you just overwrote your secret key (oops)

---

Further update

If you're on heroku and putting one of these environment variables into code that will be evaluated at compile-time (eg config/initializers/devise.rb for instance) you will need to switch on heroku's "compile time environment variable" option: user-env-compile or you will not even be able to deploy your site.

This feature allows your environment variables to be available during compilation of the slug (which is when things like bundler, asset-compilation etc are run)

Note that this is an experimental heroku:labs feature - so blah blah use at your own risk blah...

Gotcha: irb, multi-line strings and "Display all 309 possibilities? (y or n)"

Small thing I figured out. I was in irb and trying to copy/paste some text into a multi-line string to play with gsub, and it kept exploding halfway through and showing the message:

Display all 309 possibilities? (y or n)

instead of actually allowing me to create a multi-line string.

Turns out, irb still responds to tab-completion even if you're halfway through creating a multi-line string.

This kinda sucks if you want a string that contains tabs - but don't want to manually type the whole string (including tabs) yourself as you go along.

To fix it, I copy/pasted the text into vim and used :retab to replace the tabs with spaces before copying it back into the string in irb... its a hack, but it worked for my purposes.

Any thoughts on a better solution?

Link: alias vs alias_method

If you ever wondered what the difference was, this quick post by Neeraj Singh will show you the difference between alias and alias_method

TL;DR:

• alias_method is slightly more flexible accepting strings or symbols
• alias_method will redefine local methods at runtime - which means you can override the aliased method in a subclass and alias_method will pick this up
• alias will keep the original scope when aliasing - so even if you override the method in a subclass, the aliased method will refer to the parent's original method
• alias works nicer with rdoc

Don't forget to use a comma for alias_method

Link: ruby-warrior : a fun way to learn a little ruby

I just came across ruby-warrior.

I can thoroughly recommend it as a couple of hours of fun diversion while picking up a few lines of basic ruby.

Its byline is:

"A triumphant quest of adventure, love & destiny all within a few lines of code."

Link: Your boss, the invader from Mars (or how agile transitions can become unhinged by poor buy-in)

If you've ever tried to implement an agile transition in a traditional workplace, you'll know there are many stumbling blocks - one of the greatest being that of the existing stakeholder: Your boss, the invader from Mars explains how change can and should be dealt with by both the SCRUM and kanban approaches to agile development - and how that can easily be messed up by your boss.

The case-study especially strikes a chord with my experiences, describing a shop wanting to "become more agile" but not actually implementing agile processes, instead trying to add the "agile" on top - with a rather apt simile where the agile process are described as "layered over existing dysfunctions as a sort of veneer".

The article then goes on to show various ways in which a boss who is not fully vested in the agile approach can unhinge the transition process... but also lets us learn the various lessons from that so a we can hopefully improve upon it in our own attempts.

Gotcha: rvm and No binary rubies available for: ubuntu/12.04/x86_64/ruby-1.8.7

I'm working with a legacy rails system. We're still stuck on Rails 2.3.2 (yes really) and ruby 1.8.7 (it's better than 1.8.6 which we were on only a few months back).

I've just been given a new computer at work (finally) and it has ubuntu 12.04 - and I was trying to set it up with a rails stack. For which I can strongly recommend Ryan Bigg's howto article: Ubuntu, Ruby, RVM, Rails, and You

I got through the tutorial ok, happily installed ruby 1.9.3... but when I tried to install 1.8.7, I hit a snag.

    triton:[~]% rvm install 1.8.7
    Searching for binary rubies, this might take some time.
    No binary rubies available for: ubuntu/12.04/x86_64/ruby-1.8.7-p374.
    Continuing with compilation. Please read 'rvm help mount' to get more information on binary rubies.
    You requested building with 'gcc-4.4' but it is not in your path.

According to rvm, you can list the set of available rubies using rvm remote which gave me:

    # Rubies available for 'ubuntu/12.04/x86_64':
    
       ruby-1.9.3-p194
       ruby-1.9.3-p286
       ruby-1.9.3-p327
       ruby-1.9.3-p362
       ruby-1.9.3-p374
       ruby-1.9.3-p392
       ruby-1.9.3-p429
       ruby-1.9.3-p448
       ruby-2.0.0-p0
       ruby-2.0.0-p195
       ruby-2.0.0-p247

Ruh-roh!

Now, I know 1.8.7 is ancient news, but I didn't think they'd drop support for it so entirely.

I went wandering about looking for a way to specify an older repository - perhaps the 10.04 repo would still have 1.8.7 in a useable format...

Turns out I'd just made a simple mistake, easily rectified.

The important clue was in the last line of the result for rvm install:

You requested building with 'gcc-4.4' but it is not in your path.

I'd copied a whole bunch of things across to the new 'puter, including my .zshrc file - which is what was specifying the gcc version (can't remember why). And on the new 'puter I had gcc version 4.6 but my .zshrc (copied from my previous computer) was pointing at this older version.

Because of this small mistake, rvm thought I didn't have a c-compiler. Which means that rvm wasn't able to install any rubies from source - only pre-compiled rubies - which means it was restricted to only those already pre-compiled for my platform (ie 64-bit ubuntu 12.04).

oops.

When I fixed the gcc line in my .zshrc - rvm was able to figure out how to install+compile source-code rubies - including v1.8.7

How to shop at Ikea...

Step 1. recognise that you have a problem.
Decide that ikea has the perfect solution.

Step 2. Find a time where you know you have four hours spare (just in case), but convince yourself that you're only likely to be there for an hour or so.

Step 3. struggle for two hours through a labyrinth of perfect mini-houses and dawdling shoppers trying to find the thing you know is there somewhere... you've *seen* it the last time you were here...

Step 4. Along the way discover solutions for five other problems you'd forgotten you had. Write down prices and numbers for ALL the Things!!!

Step 5. You reach the end. Stop for meatballs.

Step 6. Spend a few minutes adding it all up on paper to discover that you'd be spending the next five pay-checks paying back the credit card debt to cover it. Cut it down to only two or three things you most need.

Step 7. Refreshed and ready. Head downstairs to pick up your things... and suddenly re-discover the kitchen section... and remember that you needed some wine glasses, and you loved that metal bowl and could do with another one... and oh! memory-foam pillows, you need a spare! etc etc...

Step 8. Get to the warehouse section and discover there's no flat trolleys, there's a queue of ten people waiting already and they're somehow closing in fifteen minutes. wait... and wait... and wait... plan the order-of-attack for hitting the aisles at a run.

Step 9. Eventually get to the first aisle... and discover you wrote down the wrong number for this item, and without it, almost all the others are useless.
Give up and decide you'll just get the original item *only*. try not to feel bad about the additional two hours of lost time spent choosing these items.

Step 10. Get to the aisle and discover that it comes in a set of *three* huge boxes. Heave them onto the trolley. Think to yourself... holy shit, I don't remember the boxes being that long... how the hell are they going to fit in the hatchback?

Step 11. navigate the trolley towards the checkout... as you try to turn it, remember that these trolleys work on hovercraft-physics, and have about the same leeway... try desperately to stop five times your own bodyweight of flat-packed MDF as it begins to slew into a pair of inattentive shoppers that are drifting about in front of you without looking... discover that the best course of action is to yell maniacally "I can't stop this thing!!!" while they stare at you in horror and clutch small children to their shaking breast.

Step 12. When you're finally waiting in the queue, remember that they do deliveries! You won't have to fit the boxes in the car after all! ... Get to the deliveries desk and re-discover that Ikea assumes that *everybody* has a house-wife willing to be there from 12-noon to 8pm for when they decide to show up... and they will *only* do next day delivery. Decide you want to keep your job, and go fetch your car.

Step 13. break a nail off and bleed profusely getting the first long box in the car... realise that even with the back seats down and the front seat far forward, it's still sticking out by 5cm... of course you knew this was going to happen.

Step 14. use the remaining shreds of your undergrad maths to figure out a topologically viable solution that will allow you to slam the hatchback closed before the carpark closes for the night... When you do, realise you have zero side-vision on your left, it's pouring with rain and pitch dark outside... hope to god that nobody does anything stupid on that side of you tonight. Decide to see the silver lining - if you die, at least you won't see it coming.

Step 15. get to the exit gates and discover that you were in the carpark for your free three hours... and 15 minutes - about the same time you spent struggling to get the boxes into the car... and for which you now owe $8.
Don't forget to drop your credit card on the ground after the *first* time the machine rejects it... apologise to the three cars behind you in time for it to reject you a second time.

Step 16. discover you have to turn left when leaving the carpark. You know this when twice your bodyweight in MDF suddenly slides off the passenger seat onto your left shoulder. Resolve to drive like a granny until you get home...

Step 17. as you continue to drive... try to repress the sinking feeling that accompanies your realisation that you don't have a trolley at the other end...

Gotcha: undefined method `render' for SomeTemplate

I recently had to scratch my head over a strange exception message I'd never seen before:

undefined method `render' for #<SomeTemplate:0xb17da594>

The problem was, it came with *no* application trace at all... just the usual Rails stack-trace.

I had to guess at where the problem came from by grepping for SomeTemplate in the controller-action that had exploded, and came across the following line of code:

  @template = SomeTemplate.new(params[:some_template])

Now SomeTemplate is a model of ours, and prior to this error occurring that line of code had been:

  @temp = SomeTemplate.new(params[:some_template])

I have a strong aversion to calling any variable "temp", because my reaction to seeing a variable called temp is to say "a temporary what?" not "oh, that's an instance of a template", and if I'm getting the wrong idea when I look at a variable name - that's an indication it should be changed.

So I renamed it...

Unfortunately, it turns out that @template is a magic variable in Rails that refers to the template to be rendered in a given controller action.

So Just Don't Do That.

The code has now been rewritten to the following, and works just fine:

  @some_template = SomeTemplate.new(params[:some_template])

Your Open Source Report Card

Have a look at your own Open Source Report Card. It dives into your github commits and generates a description of your coding propensities - skills and habits.

You might even find out something you never knew about yourself - apparently I'm a Tuesday tinkerer!

Here's mine for a comparison taryneast

Backporting "in_array" to older versions of should'a "ensure_inclusion_of"

Here's another shoulda backport I added recently. If you're still stuck using a legacy Rails system, this backport will let you use "in_array" in the "ensures_inclusion_of" Matcher.

Save it into something like: config/initializers/shoulda_monkeypatches.rb, then use it like this:

   should ensure_inclusion_of(:widget_status).in_array(Widget::VALID_STATUSES).allow_blank.with_message(:is_invalid).use_integer_test_value

  # backport the "in_array" method for the ensure_inclusion_of matcher
  # While we're at it, add allow_blank and allow_nil too
  module Shoulda # :nodoc:
    module ActiveRecord # :nodoc:
      module Matchers
        class EnsureInclusionOfMatcher
          ARBITRARY_OUTSIDE_STRING = 'shouldamatchersteststring'
          ARBITRARY_OUTSIDE_INT = -999999999

          # to initialize the options
          def initialize(attribute)
            super(attribute)
            @options = {}
          end

          # add the method we want to allow us to pass in arrays instead of
          # just ranges
          def in_array(array)
            @array = array
            self
          end          

          # might as well also add the allow_blank and allow_nil methods too
          def allow_blank(allow_blank = true)
            @options[:allow_blank] = allow_blank
            self
          end

          def allow_nil(allow_nil = true)
            @options[:allow_nil] = allow_nil
            self
          end

          # This is a method of my own addition to point out that the
          # test-value must be an Int, not a String... because a string can
          # evaluate to 0 which is a valid Int... which will make the test
          # pass where it shouldn't :P
          def use_integer_test_value(only_integer = true)
            @options[:use_integer_test_value] = only_integer
            self
          end

          # override description so it doesn't just try to inspect the range
          def description
            "ensure inclusion of #{@attribute} in #{inspect_message}"
          end

          # override the matches method to allow arrays as well as ranges
          def matches?(subject)
            super(subject)

            if @range
              @low_message ||= :inclusion
              @high_message ||= :inclusion

              disallows_lower_value &&
                allows_minimum_value &&
                disallows_higher_value &&
                allows_maximum_value
            elsif @array
              if allows_all_values_in_array? && allows_blank_value? && allows_nil_value? && disallows_value_outside_of_array?
                true
              else
                @failure_message_for_should = "#{@array} doesn't match array in validation"
                false
              end
            end
          end

 
        private

          # provide the message-inspect method to use either array or range
          def inspect_message
            @range.nil? ? @array.inspect : @range.inspect
          end          

          # array helper methods
          def allows_all_values_in_array?
            @array.all? do |value|
              allows_value_of(value, @low_message)
            end
          end

          def disallows_value_outside_of_array?
            disallows_value_of(value_outside_of_array)
          end

          def value_outside_of_array
            test_val = @options[:use_integer_test_value] ? ARBITRARY_OUTSIDE_INT : ARBITRARY_OUTSIDE_STRING
            if @array.include?(test_val)
              raise CouldNotDetermineValueOutsideOfArray
            else
              test_val
            end
          end

          # blank and nil helper methods

          def allows_blank_value?
            if @options.key?(:allow_blank)
              blank_values = ['', ' ', "\n", "\r", "\t", "\f"]
              @options[:allow_blank] == blank_values.all? { |value| allows_value_of(value) }
            else
              true
            end
          end

          def allows_nil_value?
            if  @options.key?(:allow_nil)
              @options[:allow_nil] == allows_value_of(nil)
            else
              true
            end
          end         




        end

      end
    end
  end

Enforced computer-free time

Gah - I've been computer-free for the past week, and not by choice.

My ageing macbook suddenly started to switch off... at random. This was ok for a while... but when it took four times for it to even get through the boot process - I had to take it into the Apple shop. meanwhile I finally set about getting a new desktop, and work gave me a new macbook to try and use.

I couldn't get the monitor to work on the desktop, and the macbook is too new to run the current ubuntu LTS (though I somehow managed to get 9.10 to install... but it won't find the ethernet port)...

I've finally got the desktop working (I was using the wrong video connection) but I still can't get mail to work properly (the joys of mail on a linux machine...), and my macbook is back from the shop... but the ethernet isn't working.

soooo.... still no email and it's been a week. But at least I have google now.

You realise just how dependant on something you've become when you suddenly don't have it anymore.

You also find just how stuck you can be. How do you find the location of a good internet shop if you don't have access to google to look one up? There used to be a directory-services phone number, but I have no idea what that is anymore, and of course I don't have a copy of the yellow pages anymore (who bothers with those when you have google)?

Ubuntu without the Unity!

For some time now my ageing (and now quite flaky) Macbook Pro has been running Lucid Lynx (an older version of Ubuntu). I've been biding my time and avoiding upgrading since, well, April 2010, I guess.

For most of the time it was because upgrading your only machine that you rely on for 100% of your income, to a potentially unstable, non long-term release version is generally a bad idea; but then, when I went to a new employer, I tried a new install of the new long term release... and slammed headlong into the weirdness that is the Unity Interface.

To sum up my experience: when I buy a desktop, I want it to *be* a desktop... not a iPad.

Now - maybe I'm just being backwards and should embrace the new unified interface which I'm told is meant to be amazing... if a little bloated.. and sometimes corrupts your file system if something goes wrong... but this is still my only machine (for now) and I need to be sure it's ok. I don't want to upgrade to Unity.

Of course, there is now a new long-term-release version, and sadly, the old one is now no longer supported, so I'm feeling a bit like a sprite in an old platform game, being relentlessly pushed towards the future by the scrolling screen...

Thankfully, I've malingered just long enough for the Ubuntu Gnome project to have been officially blessed by . it's not an LTR version yet, but it will be.

Until then - it's also possible to upgrade to the LTR but keep gnome. I've been putting it off because I'm ultra-busy and kept finding other, much more important things to do than figuring out how to do it myself... but I've just found a neat article on how to change your ubuntu interface to gnome.

Just what the doctor ordered

Huge medieval camping event - come and join in!

The reason it's been so quiet around here this month is that I've been crazy busy in my main hobby due to our major event for the year (run by our local group).

Rowany Festival is the largest Medieval re-enactment event in the Southern Hemisphere. I've heard it described as "kind of like Burning Man but with a medieval theme" - we prepare all year for this!

This year 800 of us all got together at Peats Ridge and camped medieval-style, fought wars, sang, danced, drank a *lot* of booze (some of it made by me) and generally had a great time...

Needless to say not a lot of Ruby got written... but that'll pick up again now I've got most of my washing done and camp kit packed away again.

If you'd like to join me on this stuff - I'm the president the NSW branch of the SCA, known as Rowany. The Australian kingdom is called Lochac (and also covers NZ too).

The whole Society is huge: 100,000 worldwide, and we all run regular events (camping roughly once a quarter, and feasts/tourneys once a month and just general hanging-out and learning fun stuff every week) and it's a great deal of fun.

Come along to our casual fighter-practice which is held on Tuesday nights at Petersham Town Hall on Crystal St (near to Petersham Station). We regularly get 30-50 people every week and they're running beginners fighting classes right now*. We also have regular singing, Calligraphy fibre-craft and costuming days every month.

---

* its $5 a night for non-members, or you can join for a year for $30 - but you have to do your membership online here (we're not allowed to take payment directly for that)

Picture Credit: Enough Wealth: Rowany Festival

Link: Why Yammer believes the traditional engineering organizational structure is dead

This is a great article describing the way that Yammer has set up its organisational structure to be nimble and effective. Why Yammer believes the traditional engineering organizational structure is dead

Highlights for me:

Yammer’s biggest rule of thumb is “2 to 10 people, 2 to 10 weeks,” ... If you employ the “2 to 10″ rule, it’ll also force you to release often, test your assumptions, and not over-invest in mistakes.

and

While everybody knows how expensive context switching is, it’s staggering that nobody builds that into their organization as a constraint. With total focus, you build one thing, ship it, and then are able to move onto something else.

rubyconf was awesome

that is all...

Quick and dirty facebook feed parsing

So, there's this big discussion going on for my hobby group at the moment, and the main discussion has been going on in facebook - mainly because that's where I was first asked to set up a discussion and it took all of five minutes to get the page up and running.

However - now that discussions are progressing, there are a number of people *outside* the facebookiverse who have raised quite reasonable objections to the discussion happening there. not everyone is on facebook, not everyone *wants* to be on facebook, and to be honest, a facebook group kinda sucks for searching and archiving really important discussions.

thus it has been requested that I copy all the posts and comments to Someplace Else, to make them available for more general consumption.

At first I balked at this request - 24 posts and 250 comments to be individually copied/pasted??? Who has that kind of time?

Of course when I actually sat down to think about the problem seriously, it took far less time than I though to solve it. So here's what I did, including the quick-and-dirty ruby script that will massage the facebook feed into something that resembles readable format. It aint pretty - but it'll pass for government work.

Step 1: get the feed from the API

I'll assume you are actually a member of the group that you're after. You will need to be.

You need to go to your group and get the group's ID from the URL.

The facebook API page is here: Facebook Graph API explorer

First you need to create an access token to get the data out of facebook. This is essentially the same as doing one of those "allow application to access my data" things that you click on when you add a new app. In this stage, you need to allow the Graph-API application to access *your* group-data, to prove that you have access to he feed of the group - and to allow it to fetch out all the posts for you.

1. click the "Get access token" button
2. Select the "user_groups" checkbox
3. click the new "get access token" button
4. follow any prompts (if this is the first time using this API, you'll get the "allow access for this application" process - but it may not happen for subsequent attempts

You should now have a long encoded token in the box at the top of the page.

Next up we need to tell it what feed to use. There's a drop-down labelled "GET" which you should leave as-is. In the text-box next to that, type in the ID of the group in a URL-format like this: /1234567890?fields=feed and then click "Submit". The "fields=feed" tells the API to actually go and fetch the feed of posts and comments.

At this stage, you should be able to see a huge hash full of posts and comments in the box to the right hand side of the screen. Copy and paste that into a file.

Step 2: massage it into shape

Now you've got your feed data, you just need to play with it and spit it out into a nicer format. In our case, I decided to go for just a rough html format that showed what the posts were, what comments were attached, and who said what. My script is posted below - which can serve as a starting point for whatever you'd like to see done.

This script accepts the input filename and an optional output filename (or it just jams '.html' on the end of the input filename). It'll generate a really rough-and-ready html file that contains the posts and comments (with names and datetimes) plus some of the links (if present).

Enjoy...

---

#!/usr/bin/env ruby

DATE_FORMAT = "%H:%M:%S %d-%m-%Y"

class Object
  def blank?
    self.nil? || (self.respond_to?(:empty?) && self.empty?)
  end
  def present?
    !self.blank?
  end
end

new_file_name = nil

# if they've passed in the filename, use it
if ARGV && ARGV.length >= 1
  file_name = ARGV[0]
  if ARGV.length > 2
    new_file_name = ARGV[1]
  end
end
if file_name.blank?
  p "usage: facebookfeed <file_name> [<outfile_name>]"
  exit(1)
end
p "got file_name of: #{file_name}"

unless File.exists?(file_name)
  p "file: #{file_name} does not exist"
  exit(1)
end

# munge up an html filename for the output file
new_file_name ||= file_name.split('.').first + '.html'


if File.exists?(new_file_name)
  p "output file: #{new_file_name} already exists, please supply another"
  p "usage: facebookfeed <file_name> [<outfile_name>]"
  exit(1)
end

# parse json in file into ruby - preferably a hash
require 'rubygems'
require 'json'
require 'date'
facebook_hash = JSON.parse(IO.read(file_name))

feed_data = facebook_hash["feed"]["data"]



if feed_data.present?
  File.open(new_file_name,'w') do |outfile|
    puts "parsing #{feed_data.count} posts"
    sum = 0
    feed_data.each {|post| sum += post["comments"]["count"].to_i }
    puts "with: #{sum} total comments"

    feed_name = feed_data.first["to"]["data"]["name"]

    # html headers go here
    outfile.puts "<html>"
    outfile.puts "<head><title>#{feed_name}</title></head>"
    outfile.puts "<body>"
    outfile.puts "<h1>#{feed_name}</h1>"

    feed_data.each do |post|
      outfile.puts "<p>by <b>#{post["from"]["name"]}</b> at: <b>#{DateTime.parse(post["created_time"]).strftime(DATE_FORMAT)}</b></p>"

      if post["picture"].present?
        outfile.puts "<div style=\"float:left;\"><img src=\"#{post["picture"]}\" /></div>"
      end
      name_str = post["name"]
      name_str = "<a href=\"#{post["link"]}\">#{name_str}</a>" if post["link"].present?
      outfile.puts "<h2>#{name_str}</h2>"

      message = post["message"]
      message.each do |para|
        outfile.puts "<p style=\"clear:both;\">#{para}</p>"
      end

      comments = post["comments"]

      if comments.present? && comments["count"].present? && comments["count"].to_i > 0
        outfile.puts "<h3>Comments</h3>"
        outfile.puts "<dl>"

        comments["data"].each do |comment|
          outfile.puts "<dt>by <b>#{comment["from"]["name"]}</b> at: <b>#{DateTime.parse(comment["created_time"]).strftime(DATE_FORMAT)}</b></dt>"
          outfile.puts "<dd>#{comment["message"]}</dd>"
        end
        outfile.puts "</dl>"
      end #any comments present
      outfile.puts "<hr />"
    end # each post

    # html footers go here
    outfile.puts "</body>"
    outfile.puts "</html>"
  end # with open outfile
end #feed data present

Link: What rails security means for your startup

If you hadn't already heard, Rails has a security vulnerability that affects all versions of Rails. This one is about XML-parsing of YAML strings.

This was followed by a second vulnerability in the JSON parser - again of YAML-parsed code.

So what does this all mean for all of us running Rails-based systems? Is this just a flash-in-the-pan issue that will fade away the moment it's out of the public eye? or is it a herald of the coming apocalypse?

A really cogent overview of what the rails security issue means for your startup has been written by Patrick, and I strongly recommend you read it, and pass it on.

Amongst a number of useful overviews, it covers such things as "yeah, but we're not a high-profile site, nobody's going to attack us right?" and concludes that the worst may not yet be past, and that:

You Should Be At Defcon 2 For Most Of February

Link: Help Vampires: A Spotter’s Guide

Here's a great post on the ubiquitous "Help Vampire" who drains the life out of helpful communities...

Help Vampires: A Spotter’s Guide gives tips on how to spot, avoid and reform them for the future benefit of humanity...

XML-YAML-parsing security fix for older versions of rails

Earlier I mentioned the Serious Rails vulnerability that affects all versions of Rails for the last six years.

A fix has been put into the latest versions of Rails 2 and 3. but it requires you to upgrade to the latest version.

If you have an older version of rails and can't upgrade for various reasons (eg we are still stuck on v 2.3.2 due to some legacy code), there's a better fix for the *link* xml parsing error than the workarounds on offer (which tend to switch off your ability to parse XML).

The fix that we've done requires that you use bundler, though you could equally-well freeze rails into vendor/gems and make the same patch there. We chose the bundler/github approach because it reduces the size of our repository.

Step 1: fork a copy of rails for yourself

1. fork rails
2. git clone it into a local directory.
3. checkout the *tag* that corresponds with the version you are on (eg for us: v2.3.2.1) - you can see what tags there are by running: "git tag -l"
4. Don't worry about the big scary message it gives you about a detached HEAD - that just means you've got a specific commit checked out instead of a branch.
5. create a branch for yourself eg for us: git branch v2.3.2_xmlfixes
6. checkout that branch eg git checkout v2.3.2_xmlfixes
7. push that to your repo on github with eg: git push -u origin v2.3.2_xmlfixes

Now you have a forked copy of the rails repo with a branch at the rails-version you are using that you can refer to in your Gemfile.

Step 2: actually apply the patch...

cherry-pick this commit (which if you look at github is is the one from v 2.3.15 that fixes this very error). Using:

git cherry-pick 70adb9613e4a40c5645c99da37

Note: You are likely to get conflicts with the CHANGELOG - you can keep them or just throw them away as you wish (it's just the changelog which describes the latest changes).

commit and push to your repository.

Now you have a patched version of rails in your git repository.

Step 3: update your Gemfile

Your Gemfile is likely to have a line that includes rails such as:

gem 'rails', '2.3.2'

You need to update that line to point at *your* git repository and your new branch.

The following *should* Just Work:

gem 'rails', '2.3.2', :branch => "v2.3.2_xmlfixes", 
:git =>l 'git://github.com/<your_git_repo>/rails.git'

To find the git url, you can go to your forked copy of git, look near the top of the page where it has a text-box with a git-link. Make sure you click on the "Git Read-only" button, and copy what's in the box on the right.

The branch to use is whatever you named your branch in step 5 above.

You should now be able to run bundle install to regenerate your copy of rails - and it will pull the details from your forked-and-patched copy

Troubleshooting

Unfortunately, when I used the above, it bundled correctly, but any attempt to spin up the server just caused the following error:

./script/../config/boot.rb:61:in `require': no such file to load -- initializer (LoadError)
 from ./script/../config/boot.rb:61:in `load_initializer'
 from ./script/../config/boot.rb:117:in `run'
 from ./script/../config/boot.rb:17:in `boot!'
 from ./script/../config/boot.rb:130
 from script/server:2:in `require'
 from script/server:2

Luckily the answer is in the StackOverflow question: how to use a branch in a fork of rails in a project with bundler.

First, you need to add .gemspec files into your patched copy of rails. If you're forking 2.3.10, you can copy the gemspec files from the commit Adding .gemspec files for all gems in the 2-3-stable version of rails created by the author of the above stackoverflow issue.

Otherwise you'll need ones correct for your own version. The commit above talks about generating them from the associated Rakefiles. I created them by copying the gemspec files listed in the commit above, and then just copying over the spec = Gem::Specification block with the equivalent block that is in the Rakefiles.

eg for actionpack.gemspec, I copied the actionpack.gemspec from the commit into the rails/actionpack directory in my forked copy of rails. Then I opened up rails/actionpack/Rakefile and copied the whole block of code that begins with spec = Gem::Specification into the actionpack.gemspec file, deleting the previous block from that file first.

You will know if you got the gem-dependencies wrong if you get an error like the following:

Bundler could not find compatible versions for gem "activesupport":
  In Gemfile:
    actionpack (>= 0) ruby depends on
      activesupport (= 2.3.10) ruby

    activesupport (2.3.2)

Now you have generated the gemspecs, add them to your forked copy of Rails, commit them and push the commit to your github repo.

Then you can put the following in your Gemfile:

:git => 'git://github.com/<your_git_repo>/rails.git', :branch => "v2.3.2_xmlfixes" do
  # Note: load-order is essential for dependencies
  gem 'activesupport',  :branch => "v2.3.2_xmlfixes" # this must go first
  gem 'actionpack',     :branch => "v2.3.2_xmlfixes" # this must go second
  gem 'actionmailer',   :branch => "v2.3.2_xmlfixes"
  gem 'activerecord',   :branch => "v2.3.2_xmlfixes"
  gem 'activeresource', :branch => "v2.3.2_xmlfixes"
  gem 'rails',          :branch => "v2.3.2_xmlfixes" # this must go last
end 

Note: make sure the gems are in the order above, with rails last - otherwise you'll get something like:

Could not find gem 'activesupport (= 2.3.10) ruby', 
   which is required by gem 'activerecord (>= 0) ruby', in any of the sources.

Also note: you *must* explicitly set the branch on the git-repo line and *also8 on all the gem-lines (and they must match) otherwise bundle install will work fine, but if you try anything else you'll get the infuriating error:

git://github.com/<your_git_repo>/rails.git (at v2.3.2_xmlfixes) is not checked out. Please run `bundle install`

Finally

you should now be able to run bundle update and bundle install and it should now work.

This has been tricky to explain, and the steps are complex - if something's not clear, let me know and I'll try and make it more plain.

Link: DHH - testing like the TSA

Learning how to test is an important skill, learning what *not* to test is also an important skill. DHH shares his thoughts on this subject, with a brief article: Testing like the TSA which cuts through the "security theatre" aspect that can sometimes begin to surround our testing efforts.

There's also a long discussion on Y-combinator about the article here Hacker News: Testing like the TSA