We’re thrilled to announce CheckMarket’s new MaxDiff feature! From today, you’ll see the new MaxDiff question type and reporting functionality within CheckMarket.

With MaxDiff, you can now better understand consumer preferences across various use cases. Whether your objective is to prioritize product features or to get clarity on how different messages resonate with the market, MaxDiff is a precise and proven method for extracting exactly what’s important for your audience.

MaxDiff works much better than standard rating scales for prioritizing and quantifying the importance or preference for items on a list. MaxDiff places the list items into subsets and asks the respondent to choose which is the most important and least important for each subset.

Maximum difference scaling (MaxDiff) is usually used in marketing/product research for two purposes. One purpose is to measure the relative importance (to the respondent) of the attributes of a product. The other is measurement of the relative preference for products or product concepts.

When you launch a CheckMarket survey with a new MaxDiff question type, the respondents will see a set of specific MaxDiff questions including either images or text as items in the question layout. Here’s an example of the MaxDiff Question with text:

Each question set will appear on a separate page. The respondent will be required to select a Most important and Least Important attribute across the MaxDiff sets of questions.

If the respondent decides to reconvene the survey at a later stage, the MaxDiff questions and responses will be saved, and the version of the MaxDiff survey provided to the respondent will remain the same.

Now onto how you can start using MaxDiff in your surveys.

With just a few clicks, you can enable a MaxDiff question in CheckMarket. When you add a question within the survey set up, you will now see a new MaxDiff question under the Special types as follows:

The MaxDiff question includes 2 separate tabs, one for the Question and one for Settings.

Question Tab

In the Question tab, you can enter the desired question (i.e. Which of the following options are most or least important when it comes to deciding on X?)

The default question layout to the end user would be presented in the format Most Important Selection -> Attributes -> Least Important Selection.

Attributes will include the complete list of options/items that need to be part of the study for the consumer preferences.

Settings Tab

There are specific MaxDiff Settings that define the survey design: the number of sets and the number of attributes per set each respondent will see. 

The general rule to calculate the number of sets is (3* No. of attributes) / No. of attributes per set. However, the estimated sample size is used to provide different recommendations on the number of attributes per set and number of sets.

While the regular rule of 3* No. of attributes/ No. of attributes per set to be shown in a set works well, for smaller sample sizes (<150), we generally recommend 1.25* recommended set for better accuracy. However, the number of sets and number of attributes per set can be changed by you at any time.

Survey Design

Once the question is saved, a sample version of the MaxDiff is shown in Survey Builder as follows.

Experimental Design

The experimental design is a critical part of MaxDiff. This determines which attributes will be shown in the questions presented to respondents. It ensures there is a complete balance in representation of the attributes to ensure accurate and reliable results across the correspondent pool. 

The general guidance is that we want every respondent to see each attribute three times. In addition to this, CheckMarket’s experimental design takes into account item balance, paired balance and position balance: 

• Attribute Balance: Each attribute is shown an equal number of times within a respondent’s question set. 
• Paired Balance: Each attribute is shown with every other attribute an equal number of times. 
• Position Balance: Each attribute is shown in each position in the set an equal number of times.

Language Support

As always with CheckMarket, language support and translations for MaxDiff type of questions is also available.

Reporting

When it comes to reporting on your MaxDiff question types, there is a new reporting element – “MaxDiff Summary” – that will be added by default to the Smart Report, and can be added to Blank Reports as well.

This view can be edited to include the Relative Importance metrics, Total counts of Most and Least important selections across respondents and the Confidence intervals. Any time you change the data elements on the page, you could either export this report as an image or as an excel download.

You also have the ability to apply filters and recalculate – this will run the max diff modeling & analysis by the filter specified. 

Downloads

You can download MaxDiff results in two ways.

Firstly, the “MaxDiff Details” report. This report allows you to see the root likelihood for each respondent, the experimental design and versions, as well as the respondent selections based on the version made available to the respondent. This includes exporting respondent level metrics (utilities, relative importance) for use in further analysis (for e.g. segmentation). The existing Respondent Report will now include the MaxDiff respondent responses to the MaxDiff question sets.  

Secondly, you can download the raw data of the MaxDiff report into a CSV or XLS format to perform further analysis if needed.

We hope you’re as excited as we are about the different research avenues awaiting you with MaxDiff.

Dive into CheckMarket now to try out the new functionality for yourself.

Alternatively, if you’re not already using CheckMarket, what are you waiting for?

CheckMarket offers a friendly, modern platform that focuses on engaging an audience and bringing in comprehensive and actionable feedback.