Driving home the other night, I was teasing my 4-yr old daughter (as I often do).  We were talking about what sorts of games to play when we got home and I suggested that we could spend our time poking Lucy’s baby brother Daniel in the eye.  I also pointed out that we’d have to take turns and asked Lucy which of us should go first.

Lucy protested.  “Daddy”, she said excitedly, “we don’t have to take turns–Daniel has two eyeballs”!

After I stopped laughing at this, I realized that Lucy had seen something very important, which hadn’t even occurred to me:

Poking a baby in the eye is something that can be done in parallel!

Yes, yes, of course–you should never poke a baby in the eye.  And if you have a pre-schooler, you should never suggest eye-poking as a legitimate game to play, even jokingly.  Of course I did explain to Lucy that we really can’t poke Daniel in the eye(s).

But Lucy’s observation pointed out to me how limited I’d been in my thinking.  I’d just assumed that poking Daniel in the eye was something that we’d do serially.  First I would poke Dan in the eye.  And then, once finished, Lucy would step up and take her turn poking Dan in the eye.  Lucy hadn’t made this assumption.  She immediately realized that we could make use of both eyeballs at the same time.

There’s an obvious parallel here to how we think about writing software.  We’ve been programming on single-CPU systems for so long, that we automatically think of an algorithm as a series of steps to be performed one at a time.  We all remember the first programs that we wrote and how we learned about how computers work.  The computer takes our list of instructions and patiently executes one at a time, from top to bottom.

But, of course, we no longer program in a single-CPU environment.  Most of today’s desktop (and even laptop) machines come with dual-core CPUs.  We’re also seeing more and more quad-core machines appear on desktops, even in the office.  We’ll likely see an 8-core system from Intel in early 2010 and even 16-core machines before too long.

So what does this mean to the average software developer?

We need to stop thinking serially when writing new software.  It just doesn’t cut it anymore to write an application that does all of its work in a single thread, from start to finish.  Customers are going to be buying “faster” machines which are not technically faster, but have more processors.  And they’ll want to know why your software doesn’t run any faster.

As developers, we need to start thinking in parallel.  We have to learn how to decompose our algorithms into groups of related tasks, many of which can be done in parallel.

This is a paradigm shift in how we design software.  In the same way that we’ve been going through a multiprocessing hardware revolution, we need to embark on a similar revolution in the world of software design.

There are plenty of resources out there to help us dive into the world of parallel computing.  A quick search for books reveals:

Introduction to Parallel Computing – Grama, Karypis, Kumar & Gupta, 2003.

The Art of Multiprocessor Programming – Herlihy & Shavit, 2008.

Principles of Parallel Programming – Lin & Snyder, 2008.

Patterns for Parallel Programming – Mattson, Sanders & Massengill, 2004.

The following paper is also a good overview of hardware and software issues:

The Landscape of Parallel Computing Research: A View from Berkeley – Asanovic et al, 18 Dec 2006.

My point here is this:  As a software developer, it’s critical that you start thinking about parallel computing–not just as some specialized set of techniques that you might use someday, but as one of the primary tools in your toolbox.

After all, today’s two-eyed baby will be sporting 16 eyes before we know it.  Are you ready to do some serious eye-poking?