Play Your Cards Right: Run Your First Card Sort

Let’s say you’re building a new intranet for a company. The site has pages upon pages of thrilling content that, until now, has been utterly disorganized. You could have a go at rearranging and categorizing it yourself, but that may be tricky — especially if you’re unfamiliar with the subject matter. Eek! What do you do?

Card sorting is an easy way to help organize larger quantities of content into a meaningful structure. The right time to do it is when you’re ready to start organizing the information. The objective is to discover an intuitive and meaningful classification for topics by asking some prospective users of the system — in our intranet example, this would be a group of company employees — to organize the information in a manner that makes sense to them. We do this by writing the name of each topic onto a card, shuffling the deck, and then asking a user or a group of users to sort the cards into groups according to broad subject areas. As the users sort the cards, you observe, ask questions, and take notes.

Card sorting tests can be broadly organized into two types: in an open card sorting exercise, your users will invent their own groups, give them a name, and sort cards into each. In a closed exercise, you specify the groups for them and the users sort the cards. If you’re fairly sure of the groupings you’d like to use on the site, or if your client has already specified a broad set of topics, a closed test is the right choice for you. If you’re unsure, perhaps an open test is better.

Running a card sorting exercise is quite fun, and it’s probably easier than you think. Let’s look at how to run your own card sorting exercise.

Step 1: Identify Your Topics

You’ll first need to identify what’s going to be part of your site. If you’re working on a site that already exists, it’s a matter of pulling out the pages that you have and listing each; if the site is new, list each page that you plan to include. If your site is quite large or very detailed, try to stick to higher-level topics for now. There’s no sense overwhelming your poor test subjects with five hundred cards! You can always run more card sorting tests later for subsections of your site.

Step 2: Make Some Cards

Each topic needs a card. It’s easy to do this on a computer: simply fire up your favorite word processor, look into the templates section for a business card or mailing label format, and fill in a series of cards with the name of each topic. Then, print the cards on card stock and chop them up with scissors. Feeling more lo-fi? Grab a stack of small index cards or a pad of sticky notes, and pull out your favorite marker. (You should probably stick to the computer if you have messy handwriting!)

If you plan to run a closed test, make some cards for each broad grouping. It can help to make these bigger or in a different color so that they’re distinct from the other cards. It’s a good idea to cut out some blank cards, too. During the test, a user might think of a topic that ought to be included.

Step 3: Find Some Test Participants

You’ll need to find some people to sort the cards. In our intranet example, we’re using company employees: I’d ask their managers for a bit of their time, and maybe put on a small afternoon tea as a treat for the participants afterwards. For a public web site, you might like to recruit some strangers with the promise of a free coffee or a voucher for a juice. Try asking about at a nearby library, university, or coffee shop for volunteers, or even put the word out on Twitter for nearby tweeters.

While it’s possible to run a card sorting exercise with individuals, you might also like to try it in pairs or threes. When people work together to organize cards, you can observe the kinds of discussions they’re having while they’re trying to decide which cards belong where. Individuals, on the other hand, will probably keep that information to themselves, and you’ll have to ask them to think out loud.

How many participants do you need? Jakob Nielsen suggests you ought to have at least 15 individual participants, while Donna Spencer and Todd Warfel suggest just five tests with groups of three. Bearing in mind that this is a quick-and-dirty method, I’d be inclined to pick the latter — it’s a good trade-off in terms of time and numbers.

By now, you’ll have an idea of where you plan to run the test. You should have access to a reasonably quiet room that’s well-lit, with a large table and enough seating for you and your participants. This is probably easy if you or your clients have an office; otherwise, perhaps that cafe has a quieter side room you could use.

When you invite your participants, be sure to let them know how long you suspect it will take, and explain what the activity will be like.

Step 4: Run the Test

Let’s do it!

Mix up your deck of cards and put them on the table. If you’re using a closed test, put the topic cards out onto the table as well, along with the blank cards and pens. Grab some water or juice, a pen and paper for you, and sit down with the participants. Explain the goal of the exercise — it’s good to remind them that it’s not a test or an assessment of their ability, just an exercise to help you make your web site better. Let them know that you’ll be taking notes throughout, and invite them to ask questions during the process if they have them. Then, hand them the cards and let them have at it!

If you’re working with a pair or group of people, keep an eye out for individuals who like to dominate the conversation or those who stay quiet, and try to ensure that everyone has a say; it’s as easy as saying, “Bob, what do you think?”. If you’re working with a lone participant, encourage them to discuss their decisions out loud with you. This is tricky, as most of us try to avoid mumbling to ourselves, but it’s worth the effort as you’ll gain some great insights into why that participant makes those decisions.

Take note of interesting ideas people discuss throughout the test. It’s unnecessary to jot down every card as it goes into the pile (we’ll do that later), but it’s good to note any places where the participants might have become stuck or confused.

If a participant thinks that there’s a missing topic or a better name for a topic, use one of your blank cards to add it into the mix. It’s probably a good idea to use a different-colored pen or mark it somehow so that you remember it’s a new suggestion.

Step 5: After the Test

Fire up your favorite spreadsheet application and mark down each of the cards that went into each pile (tedious, but worth it!). I like to make a column for each grouping and list each card name underneath; it’s also good to use a new sheet in a workbook for each test. Add any notes that you might have taken during the test. When you’ve copied each pile into its own column, shuffle the cards again, set up your table, and test your next set of participants.

Step 6: Interpret the Results

Just by observing your participants, you should already have some ideas about the way your pages ought to be organized, but it’s also worth examining each of your spreadsheets to look for patterns and similarities. Unambiguous topics will almost certainly have been placed in the same pile by each participant; some more difficult topics may have appeared in different groups, or users might have suggested better names.

If you really need to be able to point to percentages and hard statistics, try this free spreadsheet by Donna Spencer; by following the instructions that come with the template, you’ll be able to see patterns emerge at a glance.

The More You Know

By now, you should have a fairly good idea of what topic groupings your users will find to be intuitive. Use this data while you’re designing your information architecture, and live happy in the knowledge that you have a more solid idea of what your users really need!

Web-based Card-sorting Tools

In a big hurry? No time for cards and Sharpies? Try online services like WebSort or OptimalSort. While this lacks the hands-on, personal feel of seeing these users work on your cards in person, it’s quite a bit less effort.

And if you want to run a test in person but still save the trees, try this jQuery-driven web-based card sorter. Using this template, add categories and cards that suit your topic, pull it up on your trusty laptop, and allow your participants to drag the cards around. This one is an extremely bare-bones affair, but it’s easy to add your own CSS to make it prettier!

Frequently Asked Questions (FAQs) about Card Sorting