We’re thrilled to announce a brand new question type in CheckMarket – Closed Card Sort! From today, you’ll see the new Closed Card Sort question type and reporting functionality within CheckMarket.

Closed Card Sort questions allow you to gain a deeper understanding of how your product or service is conceptualized and organized by your customers. Whether you’re looking to optimize your website’s navigation hierarchy or prioritize features for an upcoming product launch, with CheckMarket’s Closed Card Sort question you can inform yourself with precise customer insights.

In a Closed Card Sort study, participants will see a list of cards along with a number of pre-defined categories. For each card in the list, the participant will be asked to determine which category they feel the card belongs in. They will then be asked to sort each card into the appropriate category.

By identifying categorization patterns and understanding the mental model of their users, researchers are able to make better decisions around creating an optimal user experience for their customers.

Survey participants will see a list of cards on the left-hand side of their screen, along with a number of pre-defined categories to the right (as seen in the example screenshot below). 

These cards must contain text labels and can also include images. The categories will contain text labels only. The respondent will start by reviewing the list of cards. They will then sort each card into whichever category they think the card belongs in. 

Participants are able to sort these cards using two different methods:

• Drag & Drop: The participant can select the card they want to sort, then drag it into their desired category.
• Click Sort: The participant can select/click the card they want to sort, then select/click their desired category, and the card will jump directly to the selected category.
• Keyboard: The participant can use the tab key to parse through the list of cards. They can use the space bar to select the card. Once the card is selected, the arrow keys can be used to move the card into its intended category; the left/right arrow keys move the card between categories, while the up/down arrow keys move the card within the category. Once your intended category is highlighted, you can use the space bar to move the highlighted card into the highlighted category.
  • Note: In order to be compatible with screen readers, the categories must be aligned horizontally. You can increase the page width within the survey settings in order to maximize the compatibility for keyboard sorting. We are working with our vendor to improve this process moving forward.

If the respondent decides to leave the survey after sorting only a portion of the cards, they will be able to reconvene at a later point in time and all previously-sorted cards will remain in their appropriate categories. This way, the participant can pick up right where they left off.

The Card Sort question is compatible with all mobile and tablet devices. However, for the most optimal experience, respondents are encouraged to use a laptop or computer.

With just a few clicks, you can enable a Card Sort question in CheckMarket. When you add a question within the survey builder, you will now see a new question type for Card Sort within the Special Types section.

This question includes two separate tabs – one for Question, and one for Settings.

Questions Tab

Here, the admin user can enter the desired question or prompt (i.e. Please place each card into the category you feel it best belongs in.)

Within the question tab, the admin user can also enter the labels for each of the cards, as well as the labels for the categories. You may enter a maximum of 50 cards (minimum of 2), which must contain a text label, and can also include an image. You may enter a maximum of 25 categories (minimum of 2) containing text labels only.

We recommend that you enter a relatively short text label for each card. Keeping the labels short ensures that your participants can easily digest a long list of cards. It also provides added benefits to the consumption of closed card sort reporting once the survey is completed. If you feel that your participants need more information about a particular card, consider adding a tooltip! We have provided an informational tooltip next to the “Cards” header, which includes a direct link to our help article on tooltips.

Settings Tab

There are various different configurations for your Card Sort question that you can put into place within the settings tab.

First, you may indicate whether or not a response is required for the question. When a response is required, participants will be required to sort all cards. They will only be able to move onto the next question once all cards are sorted into a category. When a response is not required, the participant will be able to sort none, some, or all of the cards. If they only sort a portion of the cards, these responses will be counted towards your Card Sort reports.

Second, you can configure the order in which you would like to display your cards and categories. By default, the list of cards and the list of categories will display in the order in which they were set up within the Question tab. There are two additional sorting options: Alphabetical, and Random. If you choose to sort the cards and/or categories randomly, the order will be randomized for each survey participant; no two participants will see the cards and/or categories in the same order.

Finally, you can also choose whether you would like to include an instructional pop up for your participants. This is a great tool if you would like to provide your participants with some additional context about the question, or to provide clarity on how to properly answer the Card Sort question. 

The admin user can enter text into the Link Label field, which will display as a clickable link underneath the question header. When the participant clicks this link, whatever text is entered into the Instructions field will display to the participant as a pop-up.

Language Support

As always with CheckMarket, language support and translations for the Closed Card Sort question type is also available.

We have added two new custom reporting elements for Closed Card Sort questions, each with two custom views!

New Reporting Element: Card Sort Matrix

We have added a brand new reporting element specifically for the Closed Card Sort question. The new Card Sort Matrix reporting element has two insightful views to choose between: the Results view, and the Popular Placement view. You can toggle between these two views within the report element settings. 

Whenever a survey contains a Card Sort question, the Card Sort Matrix reporting element containing the Popular Placement view will be added by default to the Smart Report. The Card Sort Matrix element can also be added to the Blank Reports as well.

The Popular Placement Matrix view summarizes the most popular card placements within a given category. For each of your cards, this report shows you what percentage of your participants placed that card into a given category. It also indicates which category was the most popular category to place that card into.

The Popular Placements Matrix can help you to see a quick visual synopsis of how your participants distributed your cards among the categories. It can also help you quickly determine where your participants agreed on the categorization of groups of cards, and by what degree. We achieve this by highlighting the most popular category for each card, as well as sorting the cards into groups based on which cards shared the same “most popular” category.

The Results Matrix view shows you the number of times each card was sorted into a given category. It plots the categories against the cards, and shows you the number of respondents that placed a given card within the corresponding category. The darker the shading within a given cell in the matrix, the greater the number of respondents that placed that card into the given category. This color-coding helps you quickly understand which cards had the highest agreement between participants in terms of where the cards were placed.

The Results Matrix can help you determine which categories were popular in general, by looking at the darker shaded cells. It can also help you determine which cards had a popular category, and which cards did not. The darker the cell, the more popular that category was for that card, while lighter colored cells mean that there wasn’t a category that was particularly popular for that card.

New Reporting Element: Card Sort Table

We have added another brand new reporting element specific to the Closed Card Sort question. The new Card Sort Table reporting element also has two different views to choose between: the Cards view, and the Categories view. 

Whenever a survey contains a Card Sort question, the Card Sort Table reporting element containing the Cards view will be added by default to the Smart Report. The Card Sort Table can also be added to any Blank Reports as well.

The Card View for the Card Sort Table provides insight into each individual card. This table looks at each card and gives you a breakdown of the different ways in which respondents categorized it. It tells you which category each card was placed into, how many times the card was placed into each of those categories, as well as how many unique categories a single card was sorted into.

The Card View can help you to determine which placement of cards had a high level of agreement across respondents (fewer unique categorizations), as well as which cards may have had low levels of agreement and/or vague or ambiguous descriptions (greater number of  unique categorizations).

The Category View for the Card Sort Table provides insight into each overall category. This table looks at each individual category and gives you a breakdown of how many unique cards were sorted into a particular category, as well as which cards were sorted into said category, and at what frequency.

The Category View results can help you to determine which categories have the highest agreement among participants (fewest unique categorizations) as well as which categories have the lowest agreement among participants (greatest number of unique categorizations), implying the category (or your cards/labels) may need to be rethought, renamed, or removed altogether. The cards that appear most often in a given category present a strong indication of categorical belonging. You can also use this table to examine which categories contain similar cards, therefore may be able to be combined into a single category.

Stacked Bar Chart (Existing)

For surveys that contain a Closed Card Sort question, you will also be able to utilize our existing Bar Chart reporting element to analyze your Card Sort question. This bar chart will show you a stacked bar chart view of each category. Each bar contains the percentage breakdown of each of the cards that were sorted into that category.

Downloads

You will be able to export and download the raw data from a Card Sort Question into XLS, CSV, and SPSS formats to perform further analysis if needed.

You may also view and download the Card Sort results from each of the Respondent Reports. The Respondent Report will show you how each individual respondent sorted their cards, along with their answers for any of the other questions in the survey.

We hope you’re as excited as we are about the different research avenues awaiting you with Closed Card Sort questions and reporting.

Dive into CheckMarket now to try out the new functionality for yourself.

Alternatively, if you’re not already using CheckMarket, what are you waiting for?

CheckMarket offers a friendly, modern platform that focuses on engaging an audience and bringing in comprehensive and actionable feedback.