One of the first challenge you might encounter when conducting UX research is to find your audience. Who should you test with and how to find your research participants?
A first reflex a lot of researchers have is to test with their own users. That is for several reasons: they are quicker and easier to reach, they know your product and might even like it and are willing to help (maybe for free!). Except that these criteria are not always those that will drive the relevance and performance of your UX research.
Testing with your users can be considered a foundational step in UX research from which you can build the UX maturity of your organization.
What are the advantages of testing with your own users? What are the sources of testers for your UX research? How can you create a diversified sourcing of participants?
Testing with your own users
Testing with your own users is a great way to build your UX research practice from the ground up. It is a source of testers that is quicker and cheaper to access at first when resources might be low. It is also the type of research that answers immediate questions like: are there any bugs in my product?, is my product answering (some of) my users’ needs?, what do my users think of my product and how could I improve it?, what were my users doing before using my product?
There are several advantages to working with your users for your UXR:
- Cost: There are 3 main sources of cost in UXR recruitment: external recruitment fees (if you work with a partner), internal costs (mostly work time) and compensation (monetary or other). In most cases, recruiting within your own users allows you to save on external recruitment fees. Also, your users are more likely to be able to accept to give feedback for free or in echange of a product incentive (which is cheaper for you).
- Product value: This source of testers is necessay to learn about the value of your product and what creates loyalty among your users. Although you might the one who created the product, your users are in the best place to inform you about what makes it great.
- Failures and low usage: Your users are the ones who wil help you understand what went wrong when you encounter disatisfaction or low usage of a specific feature.
Here are sources to reach your own audience:
- CRM/customer database/in-app: You can conduct tests with your own clients. They are the cheapest to reach but are not appropriate for all your research needs. Also, depending on your customer-base’s size, it is a limited source.
- Social media organic audience: This source combines some of your customer base and other users who know your brand and have not converted yet. They are an engaged audience which might be more likely to share than other colder audiences. Depending on your brand awareness, this might be a non-scalable source.
- Colleagues: This source is very biased because they already know the product and their reaction are not comparable to your end user.
- Relatives: This is the most biased source of participants you can find. Almost no research needs can be met through it, even if your target audience is very broad. That is because both you and them will not be objective.
Best practices when testing with your own users:
- Targeting specific segments (churn or low ratings). Select the customer segment that is the most relevant for your research question.
- Use a screener to confirm data points you already have depending on their reliability
- Don’t take your users word at face value, don’t try and please all of your users (I want this and this feature)
- Consider the legal aspects of contacting your users for UX purposes. Have you asked consent for this? If not, you can reach them on your social media pages, intercept users in app or add a dedicated banner in your newsletter
- Explore secondary research from the inside: sales and customer success teams probably have loads of feedback to share: most common complaints, both qualitative and quantitative
- Avoid over contacting your database, track and monitor the frequency of your touch points with them
When ONLY testing with your own users becomes not enough
As your company builds its UX research practice, you should start to conduct research with users who are not clients or prospects of your product. They might know you or not depending on your goal. Using a combination of these types of users will allow to decrease bias and access extra opportunities in the development of your product.
Here are the main drawbacks of only testing with your own users:
Bias: Testing with your own users can create bias in a lot of ways. First of all, if your product’s users accept to participate in your study, it might be because they like your brand/product. They could even be fans. This reduces their impartiality because of the courtesy bias. (Read more about how bias can affect your research here.)
Targeting: Your customer database might not feature the relevant and up-to-date data points necessary to appropriately target your participants. You might end up having to randomly target your database (which can decrease your domain’s reputation), asking for X or Y which will also dilute your brand message(more below). Even if your database is precise enough for your research purposes, you might not have access to the right elements to segment it properly (because this knowledge is owned by other teams for example).
Scale: If you start to expand your research and require increasingly higher numbers of testers, your customer base might not be enough to cover your research need.
Frequency: When conducting research, you should ensure a fresh supply of participants and avoid interviewing the same users often. If you mostly recruit within your own audience, you face the risk of meeting some participants too often.
Brand image: While occasionally ask for your users input can enhance brand perception (feeling they are heard, that their feedback is valued), too much of it will do the opposite.
Inclusive design: Creating an inclusive experience involves making a conscious effort to test with all types of users and requires going outside of your customer base if they are not represented in it.
Specific situations
Testing with your own users can be detrimental in the following cases:
- if you are launching a new product with a different target/new market (your current users are not your new target, their insights are not relevant)
- if your goal is to work on the acquisition performance of a website, you will need prospects, not clients/current users
- if your product is very complex and extensive, your users might have become experts of it and are not the right users to test for usability (reducing the expert effect).
Testing with your own users is not optimal when testing product discovery and adoption (your users know too much already).
Testing with your own users is necessary if you are testing a feature created to increase retention (their knowledge of the product is what makes their insight relevant).
You should test with both (own users and non-users) when testing a change in a core feature of your product.
Creating a diversified source of participants for your UX research
Testing with your own users is not bad in itself, as long as you use it for the right purposes and complement it with other sources. Ensuring a diverse source of participants for your UX research will increase its ROI.
We previously saw sources of testers to reach your own audience, let’s now see sources to expand from these foundations.
Outside of your audience:
Paid social media: Recruiting through social media advertising is interesting for several reasons. First, you can use these platforms’ targeting options to reach your desired participants. Social audiences are vast enough for most targets, even B2B ones. Second, the reach is very large which allows for a continuous supply of fresh participants, even for massive research needs. Choose your social platform carefully depending on your audience and make sure to use an optimized screener to reject unqualified participants. This is the most scalable source but also the most labor intensive.
Forums and social media groups: Some of these have very specific subjects which allows to target with high precision. They are usually limited in scale so you will probably quickly run out of opportunities from this source. Make sure beforehand you are allowed to post on these groups by inquiring about moderation rules and/or contacting the group owner.
Recruitment agencies: These companies are specialized in recruiting participants for studies. Outsourcing some of your UX research will allow you to focus on the more important part of your research: meeting your users. However, these agencies are usually not very agile: they might take weeks to source and organize your study. Their pricing is also a factor too look out for.
UX research recruitment tools: These online tools give you access to wide-ranging panels and automation features that will fasten your UX research. These tools usually allow the best ROI for your research.
