Link: run in with turbolinks

A quick link today to an article on turbolinks. It includes a couple of different levels of work-around snippets when adding javascripty functionality to your own links so it's pretty useful.

Rails 4: My First Run-in with Turbolinks

Generate a subset of pseudo-random permutations in non mega-huge time

One of the neat things about sending packages via sendle is that you don't have to print out a label or even know the address of your recipient (this is done so the recipient can keep their address private if they want). Instead, we give the sender a pair of unique code-names to put onto the package. So instead of J Random Shop-assistant at EvilCorp getting your full address, instead you write something like "From: Golden Lion To: Red Tiger".

It's fun, as well as effective - like secret-agent names.

but it does present us with an interesting problem - how do we generate unique codenames for potentially many, many packages.

Codenames are assigned for every package, not just for every person - that way your humble shop assistant at EvilCorp can't get any info on your shopping habits based on your codename cropping up time and again either. So that means we need lots and lots of codenames, and they must be pretty randomised too.

As mentioned, our codenames are basically of the form "<adjective> <noun>", and we have a medium-sized stock of adjectives and nouns to build from.

With 100 adjectives and 50 nouns, that's a total of 5000 possible codephrases... but the codephrase generated could be used as either the "from" or "to" (as long as it's not reused for a while), so choosing a pair of such codephrases from a set of 5000 (without repetition) gives us 5000*4999 (or 24995000) possible permutations.

That's a lot.

It takes up a fair bit of db space, and it would take quite some time to generate all the possibilities and randomise them too.

So we decided not to do that... but instead, to generate a much smaller sub-set, on the fly (ie when we got close to running out).

Which leaves us with a different problem... how to do that effectively.

The solution that was in place before I arrived, was done in SQL (because it's always faster to let the db do it, right?), and involved doing a four-way cross-join on the adjectives+nouns... and then ordering them randomly... and using a limit to only actually keep a set amount... and then only inserting into our db those that weren't already there (done using an insert where select = 0)

Sadly, when performance testing, this was discovered to be exceedingly slow... ie when it was triggered, it would take 15min or so to run, causing all other "create a package" processes to block waiting for a new code to be generated... which was unacceptable. Sure, we could run this overnight instead, but hey, what if a Good Problem occurred and we had so many orders one day that we ran out in between overnight-runs?

So instead I came up with a solution that would allow us to quickly generate a small subset of new, randomised permutations without killing throughput as badly. The solution takes about 4 seconds to generate 2000 new permutations, which is enough to get by until it's needed again.

In order to explain what it is, I first need to explain a few non-solutions we hit upon to begin with and why they didn't work for us

Solution 1: random indexing into the arrays

It seems like such a simple problem - we have a small set of adjectives and a small set of nouns, just randomly pick one of each and see if it's not already one that we have... create it if not. so the first solution was just to use "rand" on the size of each array to pick one of each by index.

This works reasonably well when the data are sparse... but when you start filling in those gaps, a lot of time is spent thrashing about randomly trying to find a pair that hasn't already been chosen. And there was no way of simply saying "you've already picked that pair" so it would pick the same pairs over and over and over and keep trying them against the set that had already been created.

This became obviously ridiculous in the specs where I chose two nouns and two adjectives... and it spent roughly 16min thrashing back and forth between "blue dog" and "red cat" trying to randomly guess the last remaining pair ("red dog").

I didn't want that to happen while angry users were waiting to post their parcel in the middle of the xmas rush... we needed a solution that would guarantee that it would not endlessly repeat useless, already-tried pairs.

Solution 2: random ordering of the arrays

The next obvious idea was: shuffle the two arrays so they are (pseudo)randomly sorted.... then just loop through them grabbing the next one in line to try and find_or_create.

eg, using a simple nested loop like:

   adjectives.shuffle.each do |adj|
      nouns.shuffle.each do |noun|
         Code.find_or_create(:code => "#{adj} #{noun}")
      end
   end

This solution works. It guarantees that you only try each option once, and it's very quick. The downside is that the randomness isn't random enough.

Now lets say that after shuffling, your first adjective is "grey"... you now loop through every noun and attach the adjective "grey" to it... and end up with a run of very similar code-phrases like: "grey dog", "grey cat", "grey bird", "grey elephant", "grey tiger", "grey wolf", "grey hippo"... not really very random. Plus, don't forget I'm using pairs of code-phrases. So we'd be getting pairs of code-phrases such as "from: grey dog to: grey cat"

Now right now, we don't have anything set up where the user could potentially abuse this obviously guessable sequence to get ahold of somebody else's package... however someday we may...

but what we were more concerned about was that if all the packages have code words that are very similar... there is grand scope for confusion and mis-labelling - which has happened to some of our customers before, and causes everyone no end of heartache and pain.

What we want is not only for individual code-phrases to be random, but for the sequence to also be kind of random too...

Solution 3: rolling loops

So I came up with a solution I've called "rolling loops".[*]

Let i and j be the loop counters. Normally, you'd match i against every value of j, before then incrementing i again.... but we want both i and j to change every single iteration of j... but if we increment both i and j for each step of the nested loop, how do we roll on to the next loop?

Well, what we *really* want is to loop through every possible permutation of the two arrays... so how about I expand out some of those and show you how to get to the solution.

Lets suppose you have two arrays that are just the numbers 0..2 The normal way to loop gives you a complete set of permutations:

[[0,0] [0,1] [0,2]] [[1,0] [1,1] [1,2]] [[2,0] [2,1] [2,2]]

If we used these pairs of numbers as indexes into the two arrays of words.. we'd get every single pairing of words, but as you can see, in the first set, every word-pair would begin with the first adjective, and in the second set, every code-pair would begin with the section adjective and so on - just as described as a problem above.

But there's another way of grouping the permutations:

[[0,0] [1,1] [2,2]] [[0,1] [1,2] [2,0]] [[0,2] [1,0] [2,1]]

In the first subset, we have numbers that are the same as each other, in the second set, we have numbers that have a difference of 1 (mod the size of the array), and the third set are of diff 2 (mod size)

This guarantees to roll through every permutation, and for every step in the loop it changes both i and j - it's only the *difference between i and j* that changes over the longer timeframe.

That may seem complex when I describe it, but it's really not. If we draw the permutations as a square:

   0,0  1,1  2,2
   0,1  1,2  2,0
   0,2  1,0  2,1

The usual simple nested loop starts at the top left of the square and moves down the column before hopping onto the next column... but the rolling loops simply goes across the rows first instead.

Now at first glance, this may not seem very much more random... but in practice (with the nouns and adjectives all randomised before beginning) it is much better. It will eventually loop around again and the adjectives will start to come in the same order - but we have around 100 adjectives - and it's extremely rare for us to have one customer send 100 parcels all at once.

Benefits of this solution:

• it's quick (ie it doesn't first enumerate all permutations before randomising
• it touches every permutation eventually (rather than random-search and maybe it never finds the final one)
• you can stop the loop at any time (eg when you have populated your next 1000 code-pairs) and restart by reshuffling the arrays into a new random arrangement
• each code-pair is only checked *once* for whether or not it already exists - rather than randomly finding the same codepair over and over forever)
• it produces a different i and j for every single iteration even of the inner loop

There's only one more tweak that my colleague suggested he said that to create a product of each of the adj/noun arrays is actually relatively quick - so in *our* project the final rolling-loop is only applied to two copies of the full product of all nouns and adjectives (randomised separately)

Here's my implementation which will keep generating pairs of code-phrases until we have the specified number.

   def populate_codes_until(final_count)
    current_count = self.count

    # generate every permutation of nouns and adjs 
    # then shuffle and duplicate
    froms = adjs.product(nouns).shuffle
    tos = froms.shuffle
    
    max_val = froms.size - 1
    
    begin
      # here we are iterating the *step size* 
      0.upto(max_val) do |step_size|
        # here we simply step through both arrays
        0.upto(max_val) do |i|

          # figure out j based on step-size and mod
          j = (i + step_size) % (max_val + 1)

          self.find_or_create_by(:from => froms[i].join(" "), :to => tos[j].join(" ")) do
            current_count += 1 # actually created a new one
          end

          return if current_count == final_count
        end
      end
    end
  end

---

[*]

yes, it's quite possible that this solution already exists somewhere and has a snazzier, and more widely-accepted, name... but I couldn't find it when I went googling, therefore it didn't exist :)

If you do happen to find it, let me know!

Link: Fundamental Guidelines Of E-Commerce Checkout Design

"Here is the harsh reality of e-commerce websites: according to recent e-commerce studies, at least 59.8% of potential customers abandon their shopping cart (MarketingSherpa puts it at 59.8%, SeeWhy at 83% and MarketLive at 62.14%). The main question is why do customers abandon their shopping cart so often?"

Fundamental Guidelines Of E-Commerce Checkout Design condenses a large study into usability into 11 really useful guidelines to make your checkout experience much more likely to convert.

Here's a condensed, tl;dr version:

But I totally recommend reading the whole thing - it's not that long and chock full of useful insights

1. Your Checkout Process Should Be Completely Linear

Non-linear "steps within steps" confuse people

2. Add Descriptions To Form Field Labels

Just because it's obvious to you what it means, doesn't mean it is to somebody else. be unambiguous

3. Avoid Contextual Words Like “Continue”

continue... shopping? or continue... on to the next step. Also don't use "back" or "proceed"

4. Visually Reinforce All Sensitive Fields On The Payment Page

People are leery of putting their CC details into a page that looks thrown together

5. Don’t Use An “Apply” Button In Your Form

it's often not pressed, even when necessary, or mistaken for the main form-submit. use AJAX instead

6. Format Field For Expiration Date Exactly As It Appears On Credit Card

needs no explanation

7. Use Only One Column For Form Fields

multiple, competing interpretations are present for multi-column pages -> all necessary or alternatives?

8. Use Shipping Address As Billing Address By Default

dammit, make people's lives easier, not more difficult. most people ship and bill to the same address

9. Use Clear Error Indications

you'll just have to see the article to get a plethora of ideas of what not to to here

10. Registration Should Be Optional

yes, a thousand times yes! I don't need to register with your company if I buy something from a bricks+mortar store, why force me to online?

11. Don’t Require Seemingly Unnecessary Information

I don't want to be spammed by you, don't force me to give you fake data it'll only hurt both of us

The study found that all of these things led to people abandoning their shopping carts before converting... so they're worth looking into even if you don't care about people and just want to improve your bottom line. but hey, you could also improve people's lives and make their experiences less confusing, less frustrating and smoother. and isn't that what technology is meant to be for?

Link: Advisory locks in postgres

Advisory locks in postgres

"PostgreSQL provides various lock modes to control concurrent access to data in tables. Advisory locks provide a convenient way to obtain a lock from PostgreSQL that is completely application enforced, and will not block writes to the table."

This is not like row-locking your tables, this is to help you ensure concurrency for certain parts of your application. eg that you only enter a complex, resource-intensive part of your codebase once across all your servers. Given that ruby basically isn't very threadsafe - this provides a mechanism for concurrency-locking that is available across your plausibly widely-distributed servers by basically storing the semaphore in the mutually-accessible database.

gotcha: redirecting through root-url fails to preserve anchor tag

Interesting (and annoying).

If you redirect to your root-path and pass it an anchor tag like "show-help" (in some controller) eg with

redirect_to root_path(:anchor => 'show-help')

It first redirects correctly to /#show-help but it then further redirects to your actual root url... and removes the anchor tag eg /my_dashboard

My expectation would be that it would preserve the anchor tag... eg the final URL would be: /my_dashboard#show-help

But of course, the browser doesn't actually send anchor tags to the server at all... which means that after the first request (to '/'), Rails never gets the anchor tag back again to pass onto the redirected url, which is why it gets dropped.

This is an issue if you really want to open the url with an anchor tag.

The main option you have is to not use root_url, but directly use the actual url you need eg: my_dashboard_url(:anchor => 'show-help'). It's not the best, but at least it works.

gotcha: running `rake db:migrate:redo` and testing

So... rails *tells* me that rake db:test:prepare is deprecated, and that just running the specs will keep the db up-to-date... but it isn't always so.

If you jigger with a brand new migration and run a rake db:migate:redo - the test db can't deal with that by itself. You either have to run rake db:migate:redo RAILS_ENV=test or rake db:test:prepare to propagate those changes.

Link: Asshole culture at Uber

Uber, darlings of the startup world, have displayed particularly poor taste and utter thoughtlessness towards 50% of the population with a promotion that they have quietly removed (without comment) since it was recently denounced.

The article: The horrific trickle down of Asshole culture: Why I’ve just deleted Uber from my phone describes what happened.

Link: Good Ruby-gems patterns

RubyGems patterns Gives you five good tips to follow to write better ruby gems:

• Consistent naming
• Semantic versioning
• Declaring dependencies
• Loading code
• Prerelease gems

Link: schedule your sidetiq jobs for a specific timezone

Currently, "UTC" and "not UTC" are the only options available for scheduling your sidetiq workers...

Now, UTC is great if you're lucky enough to live in London - in which case, you can stop reading now. Or if all your systems happen to be set that way... though it doesn't let you accommodate local things such as daylight savings time-shifts.

So the only other option provided by sidetiq is "not UTC" which means you default to whatever timezone is set on your server... which is great if your server's local time is set to the timezone you actually live in... but, if, like most of us, your server is in an amazon cluster somewhere in Randomville America or somewhere on the sub-continent, you may not have that luxury either.

Then you need some way of making sure that your sidetiq workers run at 2am *for you*... instead during the middle of the midday-rush, slowing your servers to a crawl right when your users need you most...

Thankfully, here is a useful gist by toolmantim that lets you set the sidetiq time zone to whatever you'd like. eg it lets you do something like the following

  schedule_time_zone "Australia/Sydney"
  recurrence { daily.hour_of_day(2) }

Change your dropbox passwords...

You should: Change Your Password: Hackers Are Leaking Dropbox User Info

Classic: Beating the Averages by Paul Graham

Beating the averages by Paul Graham

It's a classic. It's about lisp as a secret weapon, the building of viaweb and the blub paradox.

Sidekiq.❨╯°□°❩╯︵┻━┻

No, that's not me expressing my current frustration with Sidekiq not playing nice with mandrill-mailer... It's a real method on Sidekiq...

[1] pry(main)> Sidekiq.❨╯°□°❩╯︵┻━┻
Calm down, bro
=> nil

I found it while scanning the sidekiq test suite

  describe "❨╯°□°❩╯︵┻━┻" do
    before { $stdout = StringIO.new }
    after  { $stdout = STDOUT }

    it "allows angry developers to express their emotional constitution and remedies it" do
      Sidekiq.❨╯°□°❩╯︵┻━┻
      assert_equal "Calm down, bro\n", $stdout.string
    end
  end

<3 <3 <3

Shellshocked vulnerability bigger than heartbleed - run the fix right now!

TL;DR: run apt-get update; apt-get upgrade

Right now

Then keep doing often over the next few days...

If you're on MacOS, here's a good guide on How to patch OSX in the wake of shellshock

---

"Remember Heartbleed? If you believe the hype today, Shellshock is in that league and with an equally awesome name albeit bereft of a cool logo (someone in the marketing department of these vulns needs to get on that). But in all seriousness, it does have the potential to be a biggie and as I did with Heartbleed, I wanted to put together something definitive both for me to get to grips with the situation and for others to dissect the hype from the true underlying risk."

Everything you need to know about the Shellshock Bash bug

"Which versions of Bash are affected?

The headlines state everything through 4.3 or in other words, about 25 years’ worth of Bash versions. Given everyone keeps comparing this to Heartbleed, consider that the impacted versions of OpenSSL spanned a mere two years which is a drop in the ocean compared to Shellshock. Yes people upgrade their versions, but no they don’t do it consistently and whichever way you cut it, the breadth of at-risk machines is going to be significantly higher with Shellshock than what it was with Heartbleed.

But the risk may well extend beyond 4.3 as well. Already we’re seeing reports of patches not being entirely effective and given the speed with which they’re being rolled out, that’s not all that surprising. This is the sort of thing those impacted by it want to keep a very close eye on, not just “patch and forget”."

---

heroku are (at this time of writing) working to resolve it on their servers: heroku shellshocked status page

Classic: Why nerds are unpopular By Paul Graham

Why nerds are unpopular is one of Paul Graham's classic essays and well worth reading again. This essay made me re-evaluate my terrible memories of high-school (I think now I wouldn't have changed a thing); and will no doubt shape any recommendations I have for other young nerds who suffer the same terrible problems that come with massive unpopularity at school.

"When we were in junior high school, my friend Rich and I made a map of the school lunch tables according to popularity. This was easy to do, because kids only ate lunch with others of about the same popularity. We graded them from A to E. A tables were full of football players and cheerleaders and so on. E tables contained the kids with mild cases of Down's Syndrome, what in the language of the time we called "retards."

"We sat at a D table, as low as you could get without looking physically different. We were not being especially candid to grade ourselves as D. It would have taken a deliberate lie to say otherwise. Everyone in the school knew exactly how popular everyone else was, including us."

Stone soup with elephants

Motivation can be a problem sometimes.

For example - some days I really just *don't* want to cook. I come home exhausted, and just wish that somebody else would make dinner for me instead... but if I don't do it... I'll be nibbling raw rice and feeling miserable.

Sometimes that calls for pizza... but too much of the unhealthy stuff and I could end up in a downward spiral that makes me feel even worse.

and sometimes I can use one of my motivation tricks to kickstart myself into something more healthy.

I'm sure you've heard the story of stone soup - where a hungry traveller starts with some water and stone, telling the villagers that she's making soup with the stone. They don't believe her, but she asks if they are willing to contribute a carrot here, some parsley there to add to this stone soup - in order to make it the best stone-soup possible, of course... and eventually they all end up eating a thick and tasty soup.

I like to use that story to tempt myself. In my case - I'll pull a jar of pasta sauce out of the cupboard - but its not quite enough on its own, so I figure "well, I'll just add a few vegies"... before I know it, I've cut up enough veggies to make another two meals... and put the pasta sauce bottle back in the cupboard unopened. Stone soup well in hand.

You see, really the trick is just to get started. I generally then find it's not as hard as I first thought.

So where's the IT angle? Well I have found that a lot of my past clients have let things slip... UI-bugs, code reviews, test suites... the usual.

of course, the first step is to recognise that there is a problem... but often they end up doing little more than that, and promising themselves that "we really need to do the Thing"... but it never goes beyond that.

Then they have a big meeting and tell all the developers that "the Thing is really number one priority now". But after the meeting, nothing on their end changes - all the other priorities are still there. Clients need to "just get this one thing fixed, and then you can start on the Thing"... "we just need a few more features to go out the door to keep our customers happy while we work on the Thing"....

and somehow no time actually gets allocated to the Thing.

Sometimes there's a big meeting where blame gets slung like mud "why didn't you work on the Thing? when we said it was priority number 1?"... but that doesn't change the fact that the Thing... really isn't a priority at all. if it was, you'd be allowed time to work on it, instead of all the other urgent things.

Luckily, the stone soup kicker can help here too. Say your Thing is to build a test suite. Your app has 100,000 lines of code... that's a big Thing. But it's not too hard to promise yourself "just write one test for one model"... and you get caught up in it and before you know it, you've cleared out the tests for a whole section of your app.

I think oftentimes the motivation problem is really that you're looking at a whole elephant, wondering how the hell you're going to eat it all... and have forgotten that the way to do it is "one bite at a time". Once you're elbow deep in solving a small piece - it's often easy to get so caught up in the details that they stop bothering you any more, and you can just get on with it.

So - whatever it is, however big it is... just get started. It doesn't have to be a large piece - just one wafer-thin bite...

Gotchas: rspec expect change gives: "nil is not a symbol"

*sigh* There's a small, but meaningful difference between:

   expect {
     my_thing.do_stuff
   }.to change(my_thing.my_value).to(some_value)

and

   expect {
     my_thing.do_stuff
   }.to change{my_thing.my_value}.to(some_value)

(note the braces on the last line...)

gotchas: require/permitting an array of ids

This one got me the other day... it seems that sometimes the following won't work:

params.require(:my_thing).permit(:widget_ids)

When the set of ids is expected to be an array, instead use

params.require(:my_thing).permit(:widget_ids => [])

String#gollumize

 class String
   def gollumize 
     self.pluralize + "es" 
   end  
 end  

"pocket".gollumize
=> "pocketses"

...just because :D

Life at 38

When I was 5 I liked to do jigsaw puzzles upside down (to make it harder) and blow bubbles off the balcony -- watching them drift over the street. I liked to walk to school by myself (one block) and learn Origami from the lady in the flats behind us.

When I was 7 I wished on every star that I could have a baby sister. When I was 8, I got one.

When I was 10 I liked to explore the backways of Oyster Bay, picking flowers to make perfume (which smelled terrible). I played fantasy make-believe games with my cousins - involving magic and unicorns, where we saved the world.

When I was 12 I got another sister.... I stopped wishing. :)

When I was 13 I liked to play make-believe with my sisters and all the younger cousins. Gordy and I plotted adventures for us all, in-amongst the bamboo.

When I was 15 I liked to climb up on the roof and watch the clouds drift by. I liked to ride my bike home from school and play LARP in the park across the road.

When I was 17 I liked to swim in the backyard pool, drifting underwater with my hair floating loose around me. I liked to frantically scribble in my diary and day-dream about movies; making up adventure stories or inserting myself as a character in my favourites.

When I was 20 I loved the freedom of being independent, waking up in my own flat to the sound of Cockatoos in the pine-trees. I liked being free to wake up in the afternoon and go for a walk in the twilight, or in the quiet time after the curfew painted the streets with night. I liked staying up all night, having coffee with friends, as television got progressively more idiotic. As the sky began to warm with first light - I went out again. I liked feeling the expectant hush of the cool dawn, then retiring before the hustle woke up.

When I was 22 I loved my writing diary - pouring out my heart or playing with words, crafting new worlds on a page. I liked learning new things -- drifting from subject to subject. I liked psychology, programming and the occult. I loved my cats. I liked it that I got married and was somebody's wife. I liked meditating with my husband -- humming toneful chords without meaning. I liked learning martial arts with him.

When I was 24 I loved my garden. I spent days drifting through it and tending to the plants. I liked picking silver-beet and herbs and cooking up a meal that I'd taken from seed to table. I liked shopping at Bunnings for fruit trees. I liked spending Saturday nights with the Druids, singing, meditating, drinking mead and telling stories. Then the morning-afters, skinny-dipping in the wading-pool as the sun climbed the sky.

When I was 26 I moved interstate by myself, to see the Big City. I liked exploring the back-areas of Chatswood in search of great food. I loved hacking together brilliant solutions for colleagues desperately late on their projects. I liked rock-climbing with my work-mates and playing network games in the training room until late at night. I wrote a dictionary of elvish (quenya).

When I was 28 I loved freedom. The freedom to choose my own time, to choose what to learn, to work on my own projects. I liked smiling at my young class-mates who complained about the work - knowing this was easy compared with Real Work. I liked the meditation of archery. I liked spending my time reading while sipping coffee, or eating noodles at the local hole-in-the-wall. I liked long walks in the evening, scoping out my local territory.

When I was 30 I liked building my own small business, knowing I owned it and I could achieve whatever I wanted. I liked learning medieval crafts alongside eager students, and feasting with friends in a medieval campsite. I liked reading books, sipping coffee after a good workout at the gym. I liked watching myself learn how to walk again.

When I was 32 I enjoyed being a senior developer, being at the top of my form as well as earning high pay. I enjoyed choosing my first major investments in shares and buying my first property. I loved planning what to do with my life, and choosing to gather interesting experiences around me - New Zealand, the Northern Territory, Thailand. I learned photography, spanish and investing. I watched a *lot* of DVDs.

When I was 34 I moved internationally by myself. I loved exploring in a new country: visiting ancient ruins and watching Shakespeare at the Globe. I enjoyed getting better at photography, meeting new people and seeing new places. I built my own startup with friends. I enjoyed my morning coffee at the local in Windsor, and walking past a castle on my way to work in the morning. I loved pottering around in my allotment late into the long, english summer nights.

When I was 36 I loved getting about europe in my first car; learning French, and then eating my way around the French countryside. I enjoyed picking fresh fruit/veg from my allotment and making blackberry pie. I loved the lazy schedule allowed by working from home, and lazy Sunday afternoons drinking velvety latte with a Cranberry-and-orange muffin at my local cafe in Windsor, in view of the castle.

When I was 38 I loved being in the thick of things in the medieval group - I was president of the NSW branch (200 paid members, 10 direct and 8 indirect reports). I helped coordinate others who were running feasts and balls and camps and craft classes. I taught a class of newbies to build websites in Rails in 12 intense-but-rewarding full-time weeks. I enjoyed dinners with my cousins, and weekends going to the markets with them and then spending the day cooking preserves, jams and soups to feast upon in the weeks to come.

---

This post began as an exercise: "for every five years, write what you enjoyed doing". It helps you find out what you most enjoy - and how your tastes change over time. I also like to do it to remind me of the good things in life - which can be so easy to forget sometimes.

STOP YELLING! or how to validate a field isn't in ALLCAPS

TuShare lets people give away things they no longer need. It turns out that some people think that putting their headings in ALLCAPS will get more attention... but we think it just annoys people (nobody likes to BE YELLED AT!). Here's a quick snippet that lets you validate that your users aren't YELLING AT YOU! in any given field.

1) create a custom validator in app/validators/not_allcaps_validator.rb with the following code:

# we don't want our users YELLING AT US!
class NotAllcapsValidator < ActiveModel::EachValidator
  def validate_each(record, attribute, value)
    if value.present? && (value.upcase == value)
      record.errors[attribute] << (options[:message] || ""is in ALLCAPS. Please don't YELL AT US (we're sensitive types). Instead, just use normal sentence case. I promise we'll still hear you from over here. :)")
    end
  end
end

2) use it in your model like this:

   validate :name, :description, :not_allcaps => true

3) Add a custom matcher for rspec:

# validates that this field has the "not_allcaps" validator
RSpec::Matchers.define :validate_not_allcaps do |attribute|
  match_for_should do
    match do |model|
      model.send("#{attribute}=", "AN ALLCAPS VALUE")
      !model.valid? && model.errors.has_key?(attribute)
    end
 
    failure_message_for_should do |model|
      "#{model.class} should validate that #{attribute.inspect} cannot accept ALLCAPS VALUES"
    end
 
    failure_message_for_should_not do |model|
      "#{model.class} should validate that #{attribute.inspect} can accept ALLCAPS VALUES"
    end 
  end
end

4) test each field in your model specs with:

    it { should validate_not_allcaps(:name) }
    it { should validate_not_allcaps(:description) }

Link: Why I am leaving the best job I ever had

an awesome post by Max Schireson to remind us of our priorities in life: Why I am leaving the best job I ever had

"Earlier this summer, Matt Lauer asked Mary Barra, the CEO of GM, whether she could balance the demands of being a mom and being a CEO. The Atlantic asked similar questions of PepsiCo’s female CEO Indra Nooyi. As a male CEO, I have been asked what kind of car I drive and what type of music I like, but never how I balance the demands of being both a dad and a CEO..."

Link: Guidelines for growing a stable codebase

"Software projects need to evolve over time, but they also need to avoid collapsing under their own weight. This balancing act is something that most programmers understand, but it is often hard to communicate its importance to nontechnical stakeholders. Because of this disconnect, many projects operate under the false assumption that they must stagnate in order to stabilize.

"This fundamental misconception about how to maintain a stable codebase has some disastrous effects: it causes risk-averse organizations to produce stale software that quickly becomes irrelevant, while risk-seeking organizations ship buggy code in order to rush features out the door faster than their competitors. In either case, the people who depend on the software produced by these teams give up something they shouldn't have to."

Guidelines for growing a stable codebase covers six useful techniques for keeping your codebase stable while it increases in size. Well worth a read.

10K on Stack Overflow

w00t!

This morning I finally got the 10k moderator tools privilege on StackOverflow. You can see it here: Taryn East on Stack Overflow

Pardon me while I blow my horn for a moment, but it's taken me nearly five years to get here...

Link: Getting rid of intermittent feature spec failures

Intermittent feature spec failures are the bane of testing experience.

They cause builds to fail at random (which really screws with you when you have continuous deployment based on master-branch merges). But more insidiously, they lead to bit-rot in your test suite - because once you get used to tests failing at random, you tend to overlook it when they do... which leads to bad code falling through the cracks at the worst times.

Patrick Bacon's article: These Intermittent Test Failures Will Not Stand, Man has a small selection of useful things you can do to address these failures and root them out for good.

Rails migrations for zero-downtime sites

At TuShare we have a zero-downtime site with continuous deployment (ie, you merge to master, and away it goes to production...)

The zero-downtime part means that as the new code goes out, the old servers are still up and running until the new code is fully loaded and ready to roll... and they only then get pulled down.

This is great for uptime, but does mean you have to be extra careful about any migrations. The database must work with both the old code and the new code.

My colleague pointed me at this article: Rails migrations with no downtime. which has some in-depth coverage of the topic. What kinds of things cause errors, what you can do to reduce the pain of change-over.

Working with TuShare

So, I've got a job at TuShare.

TuShare are a little like FreeCycle - people login to either give away their excess clutter, or get some free stuff from others.

But there is one main difference - instead of having to pick the items up yourself, TuShare offers seamless integration with a courier system for any items smaller than 5kg. You still have the option of going out to pick the item(s) up (which you still have to do for furniture and large objects), but you also have the option of a door-to-door courier service for only $10 within your capital city, or $25 for city-city service.

Personally, that extra convenience is extremely appealing to me, and I used it about a month ago to get myself a brand new-to-me juicer. Since then I've put up 6 items of my own, from random old computer bits to a futon left with me by a friend who moved house.

TuShare has a small dev team, but there's a lot to work on - especially after the boom in users following the Global Sharing Day events held last week.

The people are lovely and I'm really looking forward to working with them.

Link: Open-plan offices don't work

Just read an interesting article by Maria Konnikova called The open-office trap

The TL;DR version is that while open-plan offices give employees a feeling of belonging to an open, laid-back company. The actual measured performance of employees shows that they are "damaging to the workers’ attention spans, productivity, creative thinking, and satisfaction."

Link: Service Objects in Rails

Steve Lorek has written a nice overview of Service Objects: What They Are, and When to Use Them.

"There has been a lot of discussion within the Ruby community around the service objects pattern. The term "service objects" may be unfamiliar to even seasoned Rails developers, because Rails itself implements the Model-View-Controller design pattern. There is no concept of a 'service' in a new Rails application. So it's understandable that this concept may seem foreign, and indeed non-obvious. However, I believe that the service objects pattern is well worth considering for any non-trivial Rails application."

It's a quick read, and gives a very good introduction/overview to when and why to use service objects in Rails.

---

Further to the above. If you have a Railscasts-pro account, you can watch this video on Service Objects.

It starts with an example of modularising out the functionality of a FAT model, using concerns. He then describes why he doesn't prefer that approach (my first introduction to the brilliant phrase "grep-driven-development") and finishes up with an alternative using service objects instead.

It's Rails-3 oriented - so you know you don't have to do anything special to include the concerns directory anymore... but otherwise his examples are still quite relevant and a great intro to the practicalities of how to actually go about a simple service-object and/or modularisation refactor.

---

Finally, don't forget some of the classic articles on this subject from DHH's Put chubby models on a diet with concerns to Bryan Helmkamp's 7 Patterns to Refactor Fat ActiveRecord Models

Link: Up goer five text editor

XKCD has a comic called Up Goer Five - which is a comic that describes all the details of what I'm guessing is a Saturn V rocket... but only using the set of 1000 most commonly used words of the english language... it ends up pretty funny :)

and now he's also released an upgoer5 text editor that lets you try to write anything, so long as it is using the "ten hundred" most commonly-used words.

Here's my attempt at describing what I do for a living:

There is a place where lots of computers are joined to lots of other computers. You can put words and pictures there that other people can read. There are groups of people that like to make things that other people can buy. These groups get me to make words and pictures that look pretty, to let other people know about the things their group makes. Then, people with computers can come to the place with lots of computers joined together. They come and look at the words I wrote and the pictures I put with the words, and decide if they want to buy the things the group made.

It's more difficult to be clear than you might think...

Link: programming sucks

Programming sucks

"Every friend I have with a job that involves picking up something heavier than a laptop more than twice a week eventually finds a way to slip something like this into conversation: "Bro, you don't work hard. I just worked a 4700-hour week digging a tunnel under Mordor with a screwdriver."

"They have a point. Mordor sucks, and it's certainly more physically taxing to dig a tunnel than poke at a keyboard unless you're an ant. But, for the sake of the argument, can we agree that stress and insanity are bad things? Awesome. Welcome to programming."

:)

Classic: The Law of Leaky Abstractions by Joel Spolsky

Joel Spolsky's classic: The Law of Leaky Abstractions is still well worth a read today.

Many of today's high-level tools abstract away the low-level details to such an extent that 99% of the time, you don't even need to know they're there. But it's that 1% of the time that's the doozy. When everything suddenly blows up because you accidentally crossed an invisible line you didn't know was there... how can you figure out what went wrong and how to fix it?

The low-level detail leaks through.

So often, even though you're using an abstracted, high-level tool, you often still have to learn all the lower-level stuff it was helpfully abstracting way.

An example would be a need to learn sql in order to understand joins and includes (and how they go wrong) in Rails' Active Relation

Link: Female Founders by Paul Graham

Paul Graham (of Y-Combinator) writes some amazingly worthwhile essays. One of his recent ones is about what he's learned about Female founders

Read to discover:
Is YC anti-female (spoiler: no)
Do female founders fare differently in startup culture (spoiler: sometimes, but that should rapidly improve)
What can we do to encourage more females to be founders?

Heartbleed: openSSL has been compromised test your site!

So, you may have read that there's a security vulnerability in OpenSSL called Heartbleed. It's pretty serious and potentially affects everyone. You should change all your passwords right now.

Read more about it here: Here's How To Protect Yourself From The Massive Security Flaw That's Taken Over The Internet

You can use this site to test any site you care to try: Heartbleed test

Quit being so negative!

Interacting with other people is all about perceptions. For better or worse, we can't see inside of other peoples' heads and have to infer what's inside of them based on their actions... and also on (get this) *our* past behaviour.

You thought it would be *their* past behaviour, yes? but actually unless you've known somebody a really long time - your perceptions of another person are more based on what you have done in the past, than on what they have.

It's way too easy to totally miss this point when it comes up in reality - because what you think another person is doing... just seems so obvious. Surely they're doing X, it's just obvious they are...

What you don't see are all the subconscious micro-decisions that got you to that point. What exactly did you base your observation on?

Could you be mis-interpreting something? Just because you (or your sister, your boyfriend or your old teacher back in high school) used to do X and that meant they thought Y - doesn't mean that this person you're talking to right now... thinks Y - even if they're quite clearly doing X.

This all stems from a bit of an example from my own life (I'm T):

P comes up to T and tells her all about a friend of his who's developing a new javascript library.
"it's gonna be awesome!" P enthuses.
"interesting," T says. "so what does it offer over jquery?"
"what do you mean?", P asks, looking taken aback.
"well, jquery is pretty bog-standard in the web-world right now. What does your friend's library do, that would make me choose to use it over jQuery?" T replies.
"why are you asking me that? It's a great library, it's going to be the best!", P asks

We'll stop here and see where everybody stands, because I know from later discussion that P is thinking that I'm not interested in hearing about this new project. He thinks that *I* think it's only worth doing a project if it's better than jQuery. Why else would I demand what this new project had to offer and whether or not it was worth pursuing? How dismissive of me!

Hang on a tick... what did I *actually* say?

None of the above, it turns out. From my perspective, I was curious about the project and wondering what it might have to offer.

It think it all hinges on what part we both assume the role of questions plays.

I assume that questions are to elicit information on something of interest.
P assumed that questions were a way of poking holes in an idea and dismissing it.

Both of these are assumptions based on our own past history of using questions.

Perhaps when P questions something it's because he has strong reservations about it, or because he's decided it's a bad idea and he wants to highlight all the flaws in it... but assuming that that's the reason why *another* person asks questions... will lead to very great misunderstandings.

No doubt there are downsides to my own assumption - obviously that other people will get the wrong idea about my reasons for doing so... perhaps thinking that I'm attacking their idea instead of finding out more about it.

The biggest part of the problem is that that these assumptions are pretty-much invisible to us. To us it just feels like "the way it is"... not "my perspective". it's only when we brush up against somebody else's different perspective that we even notice they're there at all (like two people with different-coloured sun-glasses finally swapping notes about the colour of objects they see).

I haven't got a solution to this problem yet... I'd be curious if any of you readers out there do. feel free to comment and let me know.

Speaking Eloquent Javascript - learn javascript (and programming) for free

Eloquent javascript seems a nice site for learning programming for free, by working your way through a free textbook.

I've only skimmed it myself - I've been working my way through Code School's Javascript path.

Another that's just come out is Speaking javascript (an O'Reilly book that's also online for free)

I've heard good things about both of these.

Have you tried either one? If so, tell me what you think, or if you have a better (free) source, let me know in the comments.

Link: Australia's top female programmers

Pollenizer has posted a list of what they call Australia's top female programmers.

I apparently made the grade... but it's a little scary to see my name amongst some of the most awesome women coders I've met over the years...

The failure of the ruby community - and what we can do about it

Sam Peters holds up a dark mirror in which our community reflects more poorly than I'd have hoped to see.

Her post on ruby meetups and the developer community is particularly worth a read for those of us happily ensconced inside that echo chamber.

Some representative quotes:

"[at ruby meetups] people who knew one another would connect, catch up, and spend the time excitedly discussing the latest news. To one another. Exclusively."

"Upon my attendance [at ROROSyd] I was immediately struck by the small, tight groups of people scattered across the venue – something that felt quite intimidating as a new learner"

"My more unorthodox theory implies that this community is merely an illusion"

The words "scathing indictment" spring to mind.

Clearly this is not the kind of community that we want.

As a mob, we can be pretty intimidating - and we need to work on better outreach and inclusiveness at our meetups.

I remember the earliest days of the rails community in Sydney. I remember when everybody was a beginner, and we came together to share our solutions to the hard problems we faced, to welcome newbies and help each other grow into skilled rubyists.

One particular observation of hers at rubyconf gives an example of what we can do better:

" I also experienced a bit of community on a personal level too – one particular lunch time, a pair of developers invited me to their table with no pretence or expectation. It resulted in a rather pleasant lunch, and opened my eyes a little. I felt a little more welcome."

In other words - it doesn't take much.

If there are people at your local meetup looking new and alone, go over and say hi. Ask them about their experience in ruby, invite them to sit at your table. Just start the conversation and include them.

I'm personally hoping to help break down that barrier-to-entry with Ruby Women - to provide at lest one extra avenue into the scene via a smaller, more beginner-friendly meetup and more open network of friend

The full article is well worth a read and some careful self-reflection. We should take note and work hard to build something better.

My Students

So, I've been teaching the General Assembly Immersive Web Development course for the past two months now.

It's going great and has been a really rewarding (if highly challenging) experience for me

They've recently finished their second (of three) projects, and the results are pretty amazing - especially considering that most of them had never touched a line of code, two months ago.

Here are links for some of their blogs to see how they've been coping over the past two months...

Sam Peters

Sam is a creative developer and her humour punctuates her work. Her blog showcases her particular mix of creativity and humour, with quite pointed insight into the development process and community

Her first project was: Forage, a website built to let gardeners swap their excess produce with others.

Alberto Forni

Alberto moved here all the way from Italy with his girlfriend in order to learn development. He's studying full-stack development, and his girlfriend is in the classroom next door studying User Interface Design. So I expect them to make a formidable development team when they're done.

His first project was: Alfred lets you keep track of expenses and wishlists - and share your wishlists with your friends.

Luke Schoen

Luke is always overflowing with ideas. No sooner do I describe some new topic and he thinks of five new applications for it in various combinations with other things he's learned.

His first project was TrendMyHunch, a site to share your ideas, pick your favourites and see if they're currently trending on google.

I have plenty more students, but they haven't shared their blog-urls with me yet... I'll come back and add them if/when they do :)

Here are the links to their recent, group-projects:

Expense Tracker

Split and track expenses with your friends or housemates

Autism Connect

A community for carers of the Autistic, allowing sharing of autism-related events, documents, links etc

Rick Rampage

A javascript, scrolling shooter game (the scores are stored in the rails backend)

Classic: Making Wrong Code Look Wrong by Joel Spolsky

The article Making Wrong Code Look Wrong by Joel Spolsky is a classic take on a programming concept that you should still, always keep in mind when writing your code:

You should try to write your code so that it's more difficult for things to go wrong...or at least more obvious when things do.

It was written Some Time Ago - so take his prescriptions with a pinch of salt and apply your intelligence and ingenuity.

It's the vibe of the thing that matters here :)

Ruby Women - a new meetup group

TL;DR: come to the Ruby women Sydney meetup (if you're a woman in Sydney and into ruby and/or rails), and please spread the word

---

I've been kicking around this idea for a while now.

You see, once upon a time I was a regular at the Sydney Linux Users Group - a formidable group of uberhackers. I knew almost nobody there, and due to the high skill-level of the members found it overwhelmingly scary to even think about presenting.

I was simultaneously looking into issues that women face in the workplace, and came across a couple of different books that pointed out that women, in general, tend to be more self-critical and less self-confident than their male counterparts.

It helps to have a network of other women to support you... but where can you get one in the traditionally male-dominated arena of IT uber-geekery?

Well - shortly thereafter, a number of us women in linux geekery began organising a local version of Linux chix. We just got together about half an hour before the SLUG meetup - and chatted and gave mini talks on interesting and basic linuxy topics... (nothing special)... and then all walked en masse to the SLUG meetup.

It doesn't seem like much, but it made a huge difference.

It provided a safe, conducive environment to women-geeks to introduce them to the linux community, without diving right into the deep end. It also provided a network of other women to serve as colleagues and role-models.

A couple of short years later, Sydney hosted a Linuxconf(.au) that had a remarkably large percentage of women for conferences of the kind - along with a whole mini-conf on topics relating to women in IT.

Linux-geekery was now something that women participated in too - in large (if not yet equal) numbers, and I'm quite certain that the strong networks formed at linuxchix helped with this

then I went away to the UK for three years...

Coming back to Sydney again, now part of the ruby community... I went to my first rubyconf.au and was overjoyed to hear of RailsGirls - a hackday-style initiative designed to help non-dev women get their first Rails site up and running.

...but it has come to the attention of many, that there's not much to keep interested women going after this. RailsGirls is currently working on RailsGirls-Next - to help Rails Girls participants continue working on their apps and slowly improve their skills... and alongside that - I've been working on something much more akin to the linuxchix concept:

Ruby women

It will be a group dedicated to women in the ruby/rails community - providing exactly the kind of supportive network and easy transition into the ruby/rails community that I think women still need.

Our first dinner meetup/discussion will be held on Wednesday 19th of March in Sydney

Please feel free to publicise the meetup far and wide, and to invite any and all women you may know.

---

To head off a couple of obvious responses:

What about a ruby men group?

Feel free to start one. There's nobody stopping you, and I'd certainly welcome that too.

I'd be really interested in hearing about the issues facing men in the IT industry too. I've heard especially about issues regarding fatherhood and balancing concerns of needing to appear dedicated to your job, as well spending time with your children... and also tales of unfair paternity laws (ie women are more likely to get time off for children than men are).

Can men come too?

Good question. We probably won't drive you away with pitchforks...

Personally, I'd be curious why you wanted to come. If you're there to pick up chicks - I'd be strongly against you coming. If you're there to support the needs of women in another group you run I'd happily welcome you...

but this won't be just *my* group, so others may have differing ideas about that.

Isn't it kind of exclusionary?

Yes. So is a group catering only to the interests of ruby programmers (as opposed to, say, python programmers)

Meetup is being DDOSed

I just checked meetup.com - it appears to be under a DDOS attack right now.

Check the meetup blog for ongoing details

Startups for the unexotic underclass

If you're an entrepreneur looking for ideas, consider looking beyond the city-centric, navel-gazing, youth-obsessed mainstream.

The unexotic underclass in a fascinating read for startups looking for meaningful ideas for new apps.

It's a call to action for us to stop wasting our time developing the next fashion-hunting twitter-app for our manicured dogs...
and instead start looking at solving some real problems that affect real people that we tend to mostly overlook, because they're just not treated as *interesting* enough by our society.

Really worth a read

Vale David E.

There's too much death going around right now.

The ruby world has been rocked by the death of of Jim Weirich (kind man, great teacher and original writer of rake)...

But for me, I have to say this is overshadowed by two deaths much closer to home.

About 3 weeks ago, my grandfather died.

I was never really close to him, but his passing sent ripples through my family. Even my close family came back from the states to attend the funeral and be with my grandmother.

But my grandfather had made it to 96. He was well-travelled and had managed to complete his book on lay evangelism - which was the end result of his life's work in his church. He lived a long, full life.

This morning, my uncle gave up his struggle with cancer.

He was only 70, and the world is that much less bright without him in it.

He was kind and generous, and always made me laugh. He brought joy and love back to my aunt - whom he married only a few short months ago (right after the diagnosis).

He was way too young to die.

Link: Stack Overflow Goes Down, Programmers Around The World Panic

StackOverflow went down on Sunday. It's back up now, but it sure serves as a reminder of how much we've come to rely on this truly awesome resource.

The article above shows some of the (inevitable) humour that flowed from the event... but it does leave me with that feeling that we're whistling nervously in the dark - what if it went away for reals?

Link: a deeper look at ruby's enumerable

Have you ever needed to load a VERY large file in ruby? No I don't mean your 500 line rails model. I mean a several gig binary or text file.

Try it. Watch your machine become unresponsive. Cry a little inside. Then reach for enumerable.

Ruby's Enumerable module gives you a way of iterating over collections in a lazy manner, loading only what you need, when you need it. But it gives us so much more than that.

rubylove has written a great, in-depth look into enumerable with: A Deeper Look at Ruby's Enumerable

Gamification on coursera

Kevin Werbach's coursera course on gamification is restarting again!

He's the bloke who wrote For the Win: How Game Thinking Can Revolutionize Your Business - and the gamification course had amazing reviews.

I started it too late last time to fully participate - and now I get another chance!

it's only just begun - so there's still time to sign up without missing out on any of the assignments or the option to get a certification.

Teaching at fishburners

If you're wondering why I've been so quiet... I'm currently teaching the "Immersive web development" course for General Assembly at fishburners, in Ultimo

This is a 12-week course covering the full stack of web development. Students learn ruby, HTML, CSS, SQL, sinatra, rails, javascript and several javascript frameworks.They complete three personal projects as well as many small homework projects. They also get an introduction to the ruby/rails and startup communities (especially via fishburners).

I'm teaching the course - and teaching is, for me, a brand new skill. While they learn rails, I get to learn how to structure and plan lessons, and how to deal with multiple levels of understanding.

It's tough - which is what I expected. I have to plan 40 hours of talking... every weekend - and brush up on every aspect of what I'm teaching so I can answer the tricky questions.

This essentially means I'm working 6 days a week, and the teaching itself drains a fair bit out of me.

So if you're wondering why I'm a bit quiet atm... that's why