What do you do when you have LOTS of options?  What about setting priority quickly if you have some ambiguity?  Pairwise Comparison is a method of scaling your options in relation to each other.  It works best when you have 5 to 12 options that you need to rank in order of importance. It also helps you to reduce the number of things you’re trying to accomplish at the same time.  Often times this is best used when ranking ambiguous customer needs and wants such as:

• “What features of a small business blogging service will my readers value the most?”
• “When I’m talking to my customer, what are the highest priority items we should spend the most time discussing?”
• “When a customer visits my website, what information is the most important for the front page?”
• “When I place an ad, what message is likely to get the most engaged response?”

The 5 Sentence Splash: Quick Summary of the Tool

In pairwise comparison, you start with a list of options, preferably between 5 and 12 options for best use of the tool.  Each option is compared to every other option on the list and you vote for the one which is more important.  The option with the most votes is the highest priority and so on.  While using this method you are often forced to think critically of why one option is better than another option, driving sometimes surprising results. The value of the tool is not just in the results, but the critical thinking that execution of the tool drives.

Why Does this Matter?

I used this tool while I was setting up my Small Business Decisions blog.  It does an incredible job of making you think through why you’re choosing one option over another. During the ranking process, I started to realize that my original approach of focusing on hard math and equations that are more of a “Behind the Curtain” approach was not really what my customer base wanted.  I used two pairwise comparison matrices.  One for my list of target beta customers (specific friends that I won’t mention) and one related to the features and approach to sharing my content.  I’ll use the second one as an example here.

How To:

Load up the Template:

Pairwise Comparison Template Download

Rank Criteria Statement:  The first thing you need to do is define what you’re ranking for.  In my example I’m going use: “What features of a small business blogging service will my readers value the most?”  I gave a few examples in the opening paragraph which could be good uses of the tool.  This statement goes in the top left orange box of the template.

List Options: Next you need to list the options you’ll be choosing between on the left.  If you end up with more than about twelve or fifteen you should eliminate the items that seem to be the least likely.  The tool and math can extend to work with as many combinations as you want, but it gets too tedious and starts to lose value compared to the effort in most cases.

Voting: For the voting phase you compare each intersection of the left side with the top side and choose the number which you think is more important.  This is illustrated by the red arrows and boxes.

Fill in the entire matrix of intersects as shown with votes for one or the other.  The auto formatting will show the color of the number you selected.  If you choose a number that doesn’t align with one of the top paired options the selection will be highlighted in red. TIP: The easiest way to do this is to fill in each column from the top to the bottom.  When I was filling in this matrix I found that I had to re-define my options several times.  That’s OK! It’s part of the critical thinking that goes with this tool.

Sort the Results: When tallying the votes is done the template will automatically index and sort the results according to your votes!  You will also notice a “weight” value is calculated.  This is specific for application of the pairwise comparison to a C&E Matrix.  I left it in this template in case you find another way that you need to use those values.

Extensions:

Dealing with More Than one Voter: As you look through my results, you probably have a few items you’d vote for more often than I did while I was guessing what you’d want to see. When there are multiple business owners, differences in opinion can and will come up. That is a good thing! We can deal with that!

In this example, each voter fills in their own copy of the matrix and then you put all of the voter’s tallies in their respective column.  I added the “Total Votes:” row along the bottom to ensure that we are comparing apples to apples from each voter.  The “Total Votes” column is what you use to rank your options.

If you really want to get nerdy, you can scale each voter’s votes by their %share in the company.  Just multiply each voter’s votes by their ownership and tally the results.

Relative Weights: In some cases, you don’t just need the order of importance, but the relative importance of the options.  Weights are commonly needed for C&E Matrices or for Correlation Matrices.  You may also want to use the weights to compare relative importance vs relative spend on a program to ensure you’re putting your money in the right places.  This method has a few limitations for weights, as discussed in the Limitations section, but it is a good place to start.  The weight of each item can be determined by dividing the number of votes for one item by the total votes in the matrix.

Limitations:

Relative Importance of Pairs: This method doesn’t allow you to choose the relative importance between two items.  In some cases, the two options you’re comparing may be of equal importance. While in some other cases the options you’re comparing may be of drastically different importance.

In theory, this method of comparing each item to multiple options should still get your items into relative linear alignment with each other.  More advanced methods will let you score how different the two items are such as Pairwise Importance Comparison, Analytic Hierarchy Process, Conjoint Analysis or Correlation Matrices (aka C&E Matrix).  If your final ranking weights should be a non-linear (ex: safety vs five financial targets) then you should be aware of the limitations when considering weighted outputs of this method.  Pairwise Importance Comparison is a specific method to address this issue.

KISS principle: For many decisions, this method is probably overly complex.  If you have a non-ambiguous list you can just look at and put the options in order.  Then this exercise probably won’t add much value.  It applies more to situations which you’re trying to sort through some ambiguity in your options.