Optimize Your Product Strategy Efforts: The Art of Harnessing Unmoderated User Research

An effective product strategy involves meticulously crafting products that align with both the demands of consumers and the capabilities of the business. While this isn’t a novel concept, it is striking how hard it can be to develop the capabilities to adequately connect with and learn from consumers in a way that meaningfully uncovers actionable insights.

Establishing and scaling product research and discovery tasks can be daunting. Here I aim to highlight common points in which teams can stray from their user-centric desires while also sharing some tools that can help you identify if leaning into unmoderated research will keep busy product leaders more connected to their users.

The Gradual Distancing from the User

To get started, I want to highlight what many people fall victim to when bringing new products to life — a gradual distancing from the user.

Chapter 1

As is the case with most ventures, everything begins with an idea, doesn’t it? Guy Kawasaki proposes that ideas worth pursuing should fulfill at least one of these goals: make a meaningful impact, correct an injustice, and prevent a wrong from occurring in the world. Better yet, why not accomplish a combination of all three? Pinpointing valuable product opportunities that satisfy these criteria stems from a profound understanding of your consumers. This is where product and user research comes into play.

Although research at the inception stage tends to be informal, it has become increasingly ingrained in the development of new products. Founders and product leaders typically devote their time and energy to activities that enable them to develop a deep connection with users. This intimate knowledge allows them to make informed predictions about which products or features will resonate with their target audience. The significance and emphasis placed on research during this stage are justifiably immense, setting the product’s trajectory.

Chapter 2

As the journey continues, you make incremental bets, fine-tuning your product based on user needs. Eventually, the product takes shape and prepares for launch. During this phase, research may take a back seat, as your focus shifts from consumers to delivery. While this trade-off might seem acceptable, the subtle distancing from consumer insights shouldn’t be overlooked.

At this stage, it’s important to acknowledge the change in priorities without losing sight of your users. Although the pressing matter at hand revolves around operationalizing the launch of your software, maintaining a balance between learning and execution is crucial. Striking this balance ensures that your product remains responsive to user needs and adapts accordingly, even during the pivotal launch phase — a principle frequently overlooked.

Chapter 3

Following the launch, the market offers its verdict: a mix of praise and critique. Your team now grapples with the challenge of charting a course forward as your product takes on a life of its own in the real world. With the team’s attention divided and concentrated on keeping the ship steady, research often recedes further into the background.

This stage is critical, as it is when many teams unwittingly distance themselves from user research, choosing instead to address the inevitable hiccups that emerge as the product gains traction. As these ripples grow, your team must adapt operationally, ensuring that you stay connected to user needs and expectations.

Informed by the earlier stages of the UX research journey, it is essential not to lose sight of the insights gained. Maintaining an ongoing dialogue with your users will enable you to fine-tune your product, ultimately ensuring its long-term success in the ever-changing waters of the market.

Chapter 4

At some point, a fortunate few teams discover that their product resonates with the market. Users are drawn to your offering, and they keep coming back for more. As your team hones the mechanics of consistent delivery, internal guards begin to lower, and a sense of accomplishment sets in, though many have lost the appetite to pick that thing we call ‘research’ back up. We know what we are doing now.

Chapter 5

Despite the established success, a period of cooling eventually follows. Your team has successfully delivered on the roadmap, but the question of “what’s next?” lingers. Is there a more extended roadmap hidden away, guiding your next steps? Some may realize that user research has fallen by the wayside, prompting a desire to conduct a new study to uncover user wants and needs. This “one-and-done” approach often leads teams to a state of stagnation, as research is treated as an isolated event rather than an ongoing process, and the desired impact remains elusive.

Other teams attempt to evade the pitfalls of the one-and-done mindset by identifying a research-savvy team member and bombarding them with requests for help. Unfortunately, this often results in compromised research quality and increased stress on that individual. Misinformed bets on user insights can have detrimental effects on your product and team morale.

Instead of succumbing to these suboptimal practices, strive to rekindle your team’s research spark and find ways to distribute research responsibilities evenly. Or better yet, see the early signs of your team becoming distant from discovery efforts and spin up a study right then and there. By doing so, you’ll ensure that research remains a regular, integral part of your product development process.

While catching the whiffs of poor practice growing is the first step, you should wonder how to avoid burying a singular teammate in research duties or falling victim to the one-and-done approach. Leveraging advancements in UXR platforms, unmoderated studies may be able to help your team scale and expand efforts. The key lies in knowing when to step back and let users speak for themselves, providing invaluable insights to drive your product’s continued success. To do this, I offer an evaluation tool, a decision tree for unmoderated research.

Is now the right time? A decision tree for unmoderated research.

As your team dusts off their analytical skills, you will need to determine where can we focus our research efforts most effectively. User studies take time and consideration but what if you could take away the need to also be present during the studies? Determining the right time to lean into unmoderated work is key. The decision tree depicted below can be used as one tool to evaluate if the juice is worth the squeeze or if the choice to go unmoderated may come back to bite you.

Question 1. What are you trying to learn?

First, assess your product questions and identify your research goals. Are your questions exploratory, evaluative, or strategic? Exploratory questions aim to uncover problems worth solving, evaluative questions focus on assessing your approach to addressing those problems, and strategic questions ensure that your solution is sustainable and effective.

If your questions are primarily exploratory, unmoderated research may not be the right fit at this stage. To succeed in unmoderated research, you need a solid understanding of the problem space and the ability to ask unbiased questions that provide rich context. If you haven’t yet spoken to users, designing an unmoderated test that delivers insightful exploratory results can be challenging. However, if your research questions are evaluative or strategic, proceed to question 2.

Question 2. Have you studied these questions before?

Consider whether you have previously studied your research questions or related topics. If this is the first time you’re addressing these issues with consumers, particularly for a newer product, unmoderated research may not yield the desired insights.

Effective research uncovers the root causes of user behavior, decisions, and preferences. If you haven’t explored this topic with users before, it’s less likely that your unmoderated test will uncover these underlying reasons. If you have studied the topic or related issues, move on to question 3.

Question 3. How engaged is the population?

Reflect on past experiences with your target audience. Are they generally responsive and eager to share their thoughts? Some demographics are more forthcoming with feedback, while others may be less inclined to share their experiences.

If your audience falls into the latter category, unmoderated testing may not be suitable. In such cases, face-to-face interactions can help maximize the value of each interaction and uncover the “why” behind user actions. If your target audience is generally responsive, continue to question 4.

Question 4. How much ground are you trying to cover?

Identify the extent of user engagement necessary for effective research. Do users need to interact with a comprehensive product experience, or can you focus on a more limited set of features?

If your research requires extensive interaction with numerous complex features, an unmoderated test may not deliver valuable insights. Without a moderator to guide participants and standardize feedback, results may be less insightful. If a narrow set of experiences suffices for your research, proceed to question 5.

Question 5. Are you in need of scale?

Many product questions can be answered by identifying patterns from 5–8 interviews. In such cases, investing time in designing an unmoderated study may not save time and could even warrant an alternative approach. However, if you require insights that are representative of your entire consumer base, move on to question 6.

Question 6. Will participants be presented with some version of the product?

Sometimes, you don’t need to present any stimuli to users to obtain the desired insights. At other times, a polished prototype or live software demonstration is essential. Consider the maturity of the stimuli you plan to present. In an unmoderated setting, you lose the ability to course-correct or overcome barriers in lower-fidelity or immature prototypes. This can cause participants to become disoriented, disengage, and leave you without useful feedback.

However, if your stimuli are well-developed or not required at all, you may have found the sweet spot where unmoderated testing can save time and provide the insights you need!

Striking the right balance between research and execution is critical to the success of your product strategy. Embracing unmoderated user research at the right moments will allow you to maintain a continuous connection with users, refine your product based on their needs, and stay ahead in the ever-evolving market all while preserving as much of your own time as possible. Guard your time but whatever you do, please don’t let research be an afterthought!

• Jack Cunningham

  Director, Product Management

  More from Jack