How Does MaxDiff Work? Admin Experience
Now onto how you can start using MaxDiff in your surveys.
With just a few clicks, you can enable a MaxDiff question in CheckMarket. When you add a question within the survey set up, you will now see a new MaxDiff question under the Special types as follows:
The MaxDiff question includes 2 separate tabs, one for the Question and one for Settings.
In the Question tab, you can enter the desired question (i.e. Which of the following options are most or least important when it comes to deciding on X?)
The default question layout to the end user would be presented in the format Most Important Selection -> Attributes -> Least Important Selection.
Attributes will include the complete list of options/items that need to be part of the study for the consumer preferences.
There are specific MaxDiff Settings that define the survey design: the number of sets and the number of attributes per set each respondent will see.
The general rule to calculate the number of sets is (3* No. of attributes) / No. of attributes per set. However, the estimated sample size is used to provide different recommendations on the number of attributes per set and number of sets.
While the regular rule of 3* No. of attributes/ No. of attributes per set to be shown in a set works well, for smaller sample sizes (<150), we generally recommend 1.25* recommended set for better accuracy. However, the number of sets and number of attributes per set can be changed by you at any time.
Once the question is saved, a sample version of the MaxDiff is shown in Survey Builder as follows.
The experimental design is a critical part of MaxDiff. This determines which attributes will be shown in the questions presented to respondents. It ensures there is a complete balance in representation of the attributes to ensure accurate and reliable results across the correspondent pool.
The general guidance is that we want every respondent to see each attribute three times. In addition to this, CheckMarket’s experimental design takes into account item balance, paired balance and position balance:
- Attribute Balance: Each attribute is shown an equal number of times within a respondent’s question set.
- Paired Balance: Each attribute is shown with every other attribute an equal number of times.
- Position Balance: Each attribute is shown in each position in the set an equal number of times.
As always with CheckMarket, language support and translations for MaxDiff type of questions is also available.
When it comes to reporting on your MaxDiff question types, there is a new reporting element – “MaxDiff Summary” – that will be added by default to the Smart Report, and can be added to Blank Reports as well.
This view can be edited to include the Relative Importance metrics, Total counts of Most and Least important selections across respondents and the Confidence intervals. Any time you change the data elements on the page, you could either export this report as an image or as an excel download.
You also have the ability to apply filters and recalculate – this will run the max diff modeling & analysis by the filter specified.
You can download MaxDiff results in two ways.
Firstly, the “MaxDiff Details” report. This report allows you to see the root likelihood for each respondent, the experimental design and versions, as well as the respondent selections based on the version made available to the respondent. This includes exporting respondent level metrics (utilities, relative importance) for use in further analysis (for e.g. segmentation). The existing Respondent Report will now include the MaxDiff respondent responses to the MaxDiff question sets.
Secondly, you can download the raw data of the MaxDiff report into a CSV or XLS format to perform further analysis if needed.
Sign Up Free
We hope you’re as excited as we are about the different research avenues awaiting you with MaxDiff.
Dive into CheckMarket now to try out the new functionality for yourself.
Alternatively, if you’re not already using CheckMarket, what are you waiting for?
CheckMarket offers a friendly, modern platform that focuses on engaging an audience and bringing in comprehensive and actionable feedback.
Sign up free and start creating more engaging online surveys.