One of the pillars of concept testing is the concept testing survey. And as every other kind of survey, it’s only as good as the questions that you fill it with. Find out here what you should be asking in your concept testing surveys and how.
What are concept testing questions?
As we have mentioned in our main guide to concept testing, listening to the voice of your customer is essential to creating the right product. However, when you make the correct decision of listening to your future customers, you have to make sure they are actually telling you what you need to know.
Not in the sense that they would be purposefully hiding anything from you. Rather, it is the job of a well-asked concept question to assure that:
- You get the information you need first and foremost. – Have you ever asked somebody a simple question, only to hear their life story except for the thing you actually wanted to know about?
- The participants tell you their real thoughts, not what they think you want to hear. – Asking a leading question is worse than asking no question at all. All it will do for you is confirm your pre-existing biases and preferences. You’re testing to get truthful insights, not to pat yourself on the back.
This may sound like a challenge, and to be honest it can be at times. But there’s no need to worry. We are here to help you get started.
Watch a quick video guide for concept testing questions, or read further!⬇️
Aim your surveys well
Even though this guide is focused on the art of asking concept testing questions, it is very important to know how to ask them to the right people. Remember that the first and foremost benefit that concept testing provides is that it saves you money. Majorly so.
This is true however only if you make sure that it is well-aimed in the scope of context and audience.
Context
Always keep in mind what the context of your product is. What are its ins and outs? Which fine details are important for your product? You need to ask the questions which are relevant to your market.
If you’re testing a new mobile payment service app, you probably don’t need to ask your respondents what their preferred internet browser is.
The context is not limited to the type of market you are aiming at. You need to ask yourself: What is the aim of my concept research? Are you aiming to be a competition to a specific product or service or do you believe you are unique and new? This is something a one simple question can answer for you.
If you want to overtake someone, pit their concept against yours. Ask the users what they prefer about their product and about your product. What are the pains users have when they are using their product? Is your product combating these pains?
All these questions and more lead to one: Is my product better in the user’s eyes than the one I am aiming to replace?
If not, you don’t even have to try, without an advantage on a product that has a head start, all you would have achieved is wasting your money. But don’t despair, failing is good, failing fast is even better. Use the answers you got from your respondents, rework your concept and test again, it is still cheaper than launching a product doomed to failure.
Audience
Closely tied to the context is the audience you want to gather for your tests. Identify your personas and target them. Asking perfect concept testing questions of someone who isn’t equipped to answer them misses the point entirely.
Target your respondents well. People above 60, living in rural areas aren’t going to be interested in your electric scooter app for big cities. All they will do is dilute your data and frustrate you.
With UXtweak’s User Panel you can easily target your respondents by the most common demographic categories such as education, nationality, gender etc. all the way to the more niche ones such as the age of their youngest child. If you want to learn more about our panel, jump to our feature page to learn about how it works, or here to see how many respondents we have available for the country you need.
If you ever struggle with this part, don’t hesitate to contact our panel experts . Don’t waste your time, let us help you.
Screening
Sometimes just broad targeting isn’t enough. In cases like this it is good to add an additional step in the form of a screening question, to make sure you are really targeting the right audience.
A prime example of a question like this would be: “Are you a client of XYZ company?”. You can use a question like this to filter out only the clients of your competition to make sure you are getting to the gist of it with your testing.
What questions to ask
Finally, the thing you came here for. What to ask the respondents?
There are many good concept testing questions, so we simply aren’t able to go through all of them. Take these just as an inspiration, because the best questions you can ask tend to be very specific and closely connected to your product/concept and therefore can’t simply be poached from a website.
But if you use any of the ones we name below and adapt them just a bit to fit your product, you can’t go wrong. Survey and other UXtweak tools provide you with many different types of questions. We discuss those questions best suited for your concept testing below with some examples to go along with them.
Rating questions – Likert Scale
Likert scale, a question type which has been tested by almost a century of use, is available for you in 5 point and 7 point versions in UXtweak questionnaires. The respondent chooses a rating on a scale.
Typically the scale of a Likert question looks like this:
- Strongly disagree
- Disagree
- Neither disagree nor agree
- Agree
- Strongly agree
In UXtweak, only the extremes on the ends of the scale are explicitly labeled to the respondent to reduce the cognitive load. You can edit labels of the extremes as you wish. You can ask about importance with scale “most important” – “not important at all”, or you can ask about formality (“very formal” – “very informal”) or any other scale you can think of which has a clear gradation.
The opposing ends should always be equal in semantic strength (to avoid skewing the results in any direction) and most importantly the label you chose are relevant to your field. The difference between Likert 5 and 7 is just in their granularity, you provide extra options around the middle with the defaults being “somewhat disagree” and “somewhat agree”.
The specific examples of Likert scale questions in use are:
- Does this product/concept solve [customer problem]?
- Scale: Strongly Agree – Strongly Disagree
- How frequently do you think you would use this product?
- Scale: Always – Never
- How likely are you to buy this product? (You can use Net Promoter Score for this as well)
- Scale: Extremely Likely – Not Likely At All
- Based on the provided information, do you understand what the purpose of [product] is?
- Scale: Fully Understand – Don’t Understand At All
- How important is solving [customer problem] for you?
- Scale: Very Important – Not Important At All
- What do you believe the quality of this product is?
- Scale: Excellent – Poor
Radio Buttons and Dropdowns
The nature of these types of questions is to allow the respondent to pick exactly one of the offered options. This makes these questions perfect for choosing between alternatives.
Allow your respondents to help you pick from different versions of copywriting text you have prepared, to see which fits the best. Or you can try and ask them about their deductions, first impressions etc. in a well-structured manner.
We suggest using random order of options in cases like this, to minimize any preferences based purely on the order of the options. Allowing your respondents to say “none of the above” and provide a different answer is also recommended so you don’t completely box yourself in a certain way of thinking.
The best quality of these questions is that they make the respondents pick from the options you provide them, helping you make sure you are on the same page with them. You can use radio buttons for questions with range which can’t be properly answered using the Likert scale.
It is important to note, that these question types are not suited for surveys conducted to gather the initial information, before creating the first concepts. You’ll want to form your concepts based on what participants say in their own words first. Limiting yourself by options may close you off from new ideas.
Examples of these questions include:
- Which of the following statements describes the product the best?
- Provide the alternatives you have ready (at least 3)
- What do you think is the main use of this product?
- Name at least 3 possible uses, with only one being the real. This will show you how well you have managed to introduce your product and if you have more work to do left in that field.
- How much would you be willing to pay for this product?
- Provide a few ranges of price for the respondents to pick from. It is important to keep in mind that they will be more likely to pick the options from the center of the range, rather than one of the extremes.
- How much time do you spend on your mobile phone?
- Similarly to the option above, provide few ranges and be wary of the middle option bias.
Checkboxes
When talking about the checkboxes, most people compared them to radio buttons and rightly so. Their concept is similar – give a respondent options and let them choose which they like the best.
The key difference is that the checkboxes allow respondents to select multiple answers. Therefore the questions you would ask would be either to select a subgroup or as a preliminary before the final radio button to decide.
The examples of these questions are:
- Top task analysis – provide the list of potential tasks which can be solved using your product and see which your respondents deem the most important.
- Terms recognition – list the terms which are connected to the field of your concept and learn which are familiar to your respondents and which aren’t.
Open ended questions
This type of concept testing question is the most powerful tool you have at your disposal. It offers you an unfiltered peek into the thoughts of your users. It is powerful, but it has to be handled with care.
The risk of a leading question or a question missing the point completely is highest here, so pay close attention when you are deciding what to ask and how to ask it. These questions are best suited for the initial steps of creating your concept.
You aim to gather as much information from your respondents as possible, to help you properly establish your concept in the context of the existing market and to pick the correct initial road you want to head down. When you’re asking these questions, you should be in the phase of formation when you’re ready to drop almost any part of your concept.
Some of the open-ended questions you can ask for you concept testing research are:
- Name three emotions which this design makes you feel
- Describe who do you believe is the target audience of this product
- When faced with [customer problem], what are the biggest challenges you bump into as you’re solving it?
- Which brand do you associate with this design?
- How would you improve this product/concept?
- Name at least 3 things you like/dislike about this product and why
- What do you see as the biggest strength/weakness of this product?
- Do you use any products similar to this? If so, please tell us which products and if you would consider changing to our product instead.
Concept testing questions with images
UXtweak allows you to add a title image to any of your questions. Sometimes you have to make sure your respondents understand exactly what you meant with your question and a good image can assure this.
Image can be also included as part of the question, for example if you want to know the opinion of your respondents on a design or a logo. However, don’t use images just for the sake of using them. They could just divert the respondent’s attention away from the question itself. If you are looking to include more images with one question you can use the radio button or checkbox with images for that. They should be used in the same connotations as their image-less counterparts.
Quick Recap
Let’s go over what you’re supposed to take away from this guide:
- Make sure you pick relevant respondents. When in doubt about how to recruit with UXtweak, contact us
- Be sure to ask what you need to hear, not what you want to hear – avoid leading questions at all costs.
- Be prepared to hear negative feedback. Negative doesn’t mean it’s bad – it can actually be more helpful than praises. Fail, and fail fast.
- Use the right way the question will be answered. Likert scales or Net Promoter Score so respondents pick from a range. Radio buttons and dropdowns to make them choose from what you have already prepared. Open ended questions to gather as much information from your respondents as possible.
Now it’s time for you to come up with your own concept testing questions. If you are unsure of their quality (which is completely okay) don’t hesitate to launch a small pilot study first and see what responses you get. If your colleague doesn’t get what you’re trying to ask, there’s a good chance the participants won’t know either.
And remember, if you’re ever stuck, we are just one click away.
