Page History

Guiding User Research Efforts

Brian Flaherty

Olivia Seitz | Reading time: about 12 min

This article is based on a research project conducted by NN/g 

PLACEHOLDER TEXT...
In 2021, Nielsen Norman Group (NN/g) — world leaders in research-based user experience — began a long-term research project to better understand human-centered design (HCD) and how practitioners utilize it in everyday work and its effects on project outcomes.  

NN/g began by establishing an HCD maturity model and realized that the maturity of individual team members and their experience, exposure, and mastery of HCD were essential to the overall team’s (or organization’s) ability to effectively utilize HCD methodologies. Catalysts were identified to help better understand the relationship between practitioner abilities and team performance. Catalysts consisted of individual practitioners whose HCD mastery positively influenced HCD practices in their teams or organizations. After conversations with the catalysts about their experience (and the experience of those they teach and guide), NN/g hypothesized that HCD practitioners share roughly the same learning journey, despite different backgrounds and contexts. 

NN/g began by conducting a large-scale survey of more than 1,000 practitioners and aimed at investigating respondents’ experience with HCD. Responses were classified into learning stages based on the self-reported HCD exposure, experience, primary activities, and biggest challenges. This process, combined with talking to hundreds of HCD practitioners each year at the NN/g user-experience conference, helped establish a set of unifying stages that most practitioners encounter while learning HCD. The stages include: newcomer, adopter, leader, and grandmaster. 

Why these stages matter

HCD practitioners can better understand their own learning process and set appropriate expectations if they know the typical stages of the learning journey. 

NN/g concluded that:

• Most learning journeys feel frustrating at some point. Having insight into the HCD learning journey and the end goal can provide encouragement during “what’s the point” moments. If you’re learning HCD independently, awareness of the journey can help you feel less alone.
• Identifying one’s current phase can help predict future progress, staying focused on the goals of the current learning phase rather than jumping ahead. Jumping ahead runs the risk of creating experiences that may be confusing, and thus less motivated to continue to learn. 

Course facilitators (or managers or mentors), must empathize, create an effective HCD learning experience for others, and enable sustainable, long-term success. 

• Understanding that different people will be at different stages in the learning process is a key part in being an effective educator. It allows for the delivery of effective learning experiences without overwhelming the audience with too much complexity and also preemptively mitigate learners’ pain points at each phase. 

Educating and activating a group of people takes a lot of resources (time, money, and effort). The education process should be intentionally designed, in order to maximize resources and return on investment. Mapping participants’ phases and their progression through the learning journey allows an educator to benchmark progress, and indirectly, success of the training. 

As practitioners progress through the stages, their mastery increases in a nonlinear fashion — experiencing fluctuations due to various factors, especially, lack of self-confidence. Note that mastery is a combination of competence and confidence; both are required to effectively use HCD. If a practitioner is good but still feels insecure, they won’t deviate from the strict steps of HCD and won’t teach others.  The goal is to create alignment between competence and confidence in order to master HCD.

The Structure

Rather than thinking of each phase as a discrete checklist, NN/g created a three-component framework for characterizing each learning stage: criteria, primary activities, and educator goals.

1. Criteria are observable qualities that can help the learner (or an external observer, such as a trainer) identify the learner’s current stage along the learning journey. It includes awareness of one’s own competence, level of exposure, and confidence.
2. Primary activities are the learner’s actions and use of HCD methodologies.
3. Educator goals and obstacles summarize learner’s pain points at any given stage and corresponding educator goals that can help learners overcome them. 

Phase 1: Newcomer 

This phase is the first in the learning journey. For the majority of HCD practitioners, this exposure occurs via a university or institution, a place of work, or an online resource. The amount of time spent at this first stage depends on how motivated the learner is. Interestingly, many practitioners immediately perceive HCD as useless and never leave this stage.

Criteria

Individuals in this stage have been introduced to HCD, but have limited experience with it. Practitioners in this phase fall into 2 buckets:

1. Individuals who are committed to learn HCD
2. Individuals who are not interested to learn more about HCD. They’ve been exposed to it, but that is where their learning journey halts. Newcomers may remain in this stage indefinitely until they encounter a deeper exposure to HCD that broadens their perspective, experience, or acceptance. 

Newcomers’ knowledge is minimal; they have a surface-level understanding of HCD, often rooted in the definition they received during their first exposure. They may be able to provide a definition, but are not familiar with the details of a framework or its value. They have topical associations with HCD — sticky-notes, phase models, and whiteboards. Newcomers are unaware of their HCD incompetence. In other words, they don’t yet know what they don’t know.

Primary Activities

Newcomers’ primary goal is to understand the basics — what HCD is and why it’s useful. Often, this stage’s activities are self-initiated: browsing articles, reading books, or signing up for a course. In other cases, HCD is learned by necessity or requirement at work or school through onboarding programs, collaborative workshops, required courses, or mandated trainings. Most newcomers have not yet actively practiced HCD activities. If they have practiced HCD at all, their participation is limited and surface-level. 

Educator’s Goals and Obstacles

The goal of an HCD educator at this stage is to communicate the purpose and potential value of HCD and to motivate the newcomer to pursue learning, then make time for it and move on to the next phases. Common obstacles are overcoming learners’ negative sentiments: annoyance with “yet another thing to learn,” unsuccessful previous attempts or experiences, capped bandwidth, and realization of their own limits (in this case, how little they know about HCD). 

Phase 2: Adopter

Individuals in phase 2 have adopted HCD and begun to practice it. They may have had some ups and downs in their limited experience with HCD. It is common for adopters to flip-flop between overconfidence and self-doubt and to feel both confused and successful at the same time. Successful adopters have discovered how relevant HCD is to their work or life and, thus, make a commitment to continue learning.

Criteria

Increased (passive or active) exposure and familiarity with HCD has made adopters aware of their knowledge limits. They understand the potential of HCD, but still have a lot to learn. Adopters have invested time, effort, and energy into HCD and have started applying it to their work with mixed success. The commitment to HCD at this phase can be self-initiated or dictated by an authority with an invested interest (leadership, organization, or supervisor). 

Primary Activities

Adopters practice HCD in a linear way — by the book. They rely heavily on checklists — for example an HCD Phases Model, as well as for the activities associated with each of its steps. Many adopters lean towards a prescribed, branded version of HCD, often provided by their institution, company, or a reputable external organization.  

It is common for HCD practitioners to encounter failure in this phase — especially if they are learning predominately on their own — because of their incomplete understanding of the HCD framework. Common types of failure include jumping to conclusions, using the wrong activity at the wrong time, and lack of buy-in or support from others. These failures push some practitioners to abandon HCD altogether. However, most learning in this phase occurs through failure. Practitioners who embrace failure tend to develop a better understanding of HCD in future phases.

Some adopters might learn primarily by actively participating in HCD training, coaching and activities together with experienced HCD practitioners. While this group of adopters may not have the opportunity to experience a sense of failure because of the support received from their peers, it is still likely they will question the value and legitimacy of HCD from time-to-time.   

Educator’s Goals and Obstacles

Adopters are on the bike, but still need training wheels and coaching. Adopters’ confidence drops as they realize the indirect ways in which HCD can be applied and how much they have to learn — for example, how straightforward they originally thought the process was versus how abstract (and potentially overwhelming) it really is. The goal of the educator at this stage is to help learners through hands-on practice and assistance until they can comfortably use HCD on their own. Many individuals can become confused in this phase when they cannot make a direct connection to the relevance of HCD. Thus, it is imperative that adopters find relevance and application to their everyday work.

Phase 3: Leader 

This is the proficiency stage of HCD. Leaders can articulate HCD succinctly to others, steadily growing their confidence, with varied experiences and continued exposure. Leaders take an active, independent role in their learning journey and begin to think adaptively about HCD; starting to explore new applications and may even have established a reputation as a subject matter expert. 

Criteria

Leaders practice HCD with general ease, confidence, and independence. Leaders become more and more aware of their new knowledge and comfort as they mature through this phase. They often teach others earlier in the learning journey. They are able to consistently and somewhat adaptively perform HCD activities without thinking too much about them. Leaders don’t try to apply HCD by the book, rather use it as needed depending on the goals. 

Primary Activities

HCD practitioners in this phase lead HCD activities with others or perform HCD without coaching, but still apply preparation and focus. While they were previously participants, they now facilitate, initiate, and even advocate for collaborative HCD activities. Activities that leaders are involved in gradually increase in complexity, ranging from involving users in learning opportunities to fostering stakeholders into the process and prototyping. 

Educator’s Goals and Obstacles

The goal of the educator in this phase is to continue to instill confidence and help sustain learner commitment. The goal should be to empower leaders to transition into the role of HCD educators and facilitators. As much as they’ve progressed, they still may not be aware of weaknesses or potential improvements (even though they often recognize a mistake after they made it). Promoting reflection in this phase is the key to helping leaders continue to grow (and progress to grandmasters). They must take an active role in adapting the HCD practice to fit their contextual goals and needs, be sustainable over time, and maximize potential benefits.  

Phase 4: Grandmaster

If you are familiar with the game of chess, the title of grandmaster is only given to the most exceptional players—those who have spent countless hours perfecting their skills and techniques.

HCD practitioners at this stage have not only become teachers of HCD, but create new ways of applying it, thinking about it, and adding to it. The practice of HCD is so embodied in their behavior that they seldom have to think about applying it. Grandmasters view HCD as a flexible, dynamic toolkit. They’ve long departed from the concept of a prescribed process and rather view it as scaffolding to solve both organizational (often internal) and end-user (external, product-related) problems. However, this phase doesn’t come without downsides. Grandmasters are more likely to doubt HCD than leaders, often when early-stage learners fall victim to mismanaged HCD marketing and thus misinterpret, misapply, or undercut the practice as a whole. 

Criteria

Grandmasters’ defining characteristic is the ability to critically reflect on their HCD practice. This enables them to judge what is useful and potentially depart from the traditional ways and activities of HCD. Grandmasters also know how to help others to this same stage of enlightenment. Grandmasters are aware of their competence. They have an in-depth, intuitive understanding and can blend HCD skills together to meet specific needs. 

Primary Activities

Traditional HCD activities are still used by grandmasters, but are altered, adapted, and applied in complex ways, depending on the goal, audience, and potential obstacles. Grandmasters don’t stick to prescribed or branded versions of HCD and more often pull tools and/or activities from other realms, like service design and business strategy. While basic HCD activities are still carried out in this phase, they differ from those in earlier stages because they are inputs or alignment strategies for more complex, involved activities (compared to previous stages where the basic activities are the end goal). 

Educator’s Goals and Obstacle

Grandmasters have very likely surpassed their original teachers. Their goal becomes not just activating and educating individual learners, but rather organizations as a whole. As practitioners themselves, grandmasters face an increasing likelihood of (re)questioning the value of HCD. Increased knowledge and mastery are a blessing and curse; this pessimistic view is often rooted in the realization of what HCD can and cannot solve (contrary to earlier naive ideas that HCD can be a cure-all). However, even grandmasters can get better, by self-reflection, by learning from their peers, and also by learning from their juniors: one of the skills of supreme mastery is the ability to discern which of the hundreds of ideas generated by eager newcomers is actually a stroke of genius.

Other Considerations 

• Trained designers experience the HCD learning journey too, just differently. Many designers view HCD as simply a way to articulate and communicate a creative approach to problem solving. Thus, many designers are likely to feel as if they bypassed this learning journey altogether because this is how they instinctively think. However, designers experience this learning journey too, albeit much earlier in their education or career, and likely not packaged or branded as HCD. This does not mean that all grandmasters are designers, but rather that many successful designers likely are. 
• Individual practitioners can be at multiple levels at the same time. Some skills may fall into one phase, while others into a lower different phase. For example, a practitioner’s mindset and reflection may fall into Grandmaster, while their hands-on experience and exposure to activities into Leader. The goal is to identify this imbalance and invest in experiences that bolster weaker dimensions of our practice. 

The HCD learning journey is a high-level, distilled representation of the most common learning phases observed in the NN/g research. Learning and teaching HCD is messy; not all experiences will fit squarely into this model. 

Regardless, it is imperative to frame and articulate learning HCD as an experiential journey. Doing so can help all practitioners become more effective learners and educators. Learners can gain insight and awareness into the greater journey and goals, while educators can thoughtfully and successfully execute the HCD learning experience they aim to create.  

Image Removed

This article is based on a research study conducted by the Nielsen Norman Group. Originally published January 2021. 

Not all user research teams look the same. Some folks have supportive stakeholders who are fully brought into running user experience research throughout the product development cycle. Others are doing their best, but don’t have influence over the whole process. And still others dabble in research as needed, without a specific strategy guiding their work.

At some organizations, Human-Centered Design (HCD) appears through the application of the Design Thinking Process when addressing innovation challenges.  Here are some examples of the steps that reflect a Design Thinking approach:

• Observation, Ideation, Rapid Prototyping, Feedback, Iteration, and Implementation
• Empathize, Define, Ideate, Prototype, and Test and Iterate.      

Other organizations infuse a human-centered lens throughout their product development lifecycle. For example, dividing their process into three phases: Inspiration phase, Ideation phase, and Implementation phase. The Inspiration phase is grounded in user research and understanding user needs.

Marianne Berkovich, User Research Lead at Livongo, with experience at Google and Adobe, pares down the role of user research throughout the lifecycle into three questions; it’s an approach that’s intuitive for everyone — from user researchers, to designers, to stakeholders — to understand. The three questions that correspond to the development stages of Design, Build, and Launch are:

• What should we build?
• How should we build it?
• Did we build it right?

You can repeat the process as many times as needed, and it’s just as effective for smaller projects (like adding a feature) as it is for designing from the ground up.

In this post, we’ll share details on how she approaches each phase, along with her Do’s and Don'ts for implementing user research as part of a human-centered design (HCD) process with your team.    

Note: Looking for a specific audience to participate in your user research? User Interviews offers a complete platform for finding and managing participants in the U.S., Canada, and abroad. Find your first three participants for free. Or, streamline research with your own users in Research Hub (forever free for up to 100 participants).

Three Questions to Guide User Research in Human-Centered Design 

Below, we explore Marianne’s three guiding questions for guiding research throughout the software development lifecycle.

Whenever you’re conducting research, it’s critical to talk to the right people.  Marianne suggests focusing on behavioral characteristics that might make a difference to how someone might use your product rather than demographic ones, which may be red herrings. It's important to find a mix of people who share a similar need and would be good candidates for your solution. For example, if you had an idea for a new gadget to make baby food more easily, your target user wouldn’t be described as “moms ages 25-40” but rather caregivers of young children, which may include dads, grandparents, and others.

Question 1: What Should We Build?

This is the phase where you look for opportunity areas for new products or features. Some of the methods you might use are ethnographic-style interviews and concept testing — perhaps with storyboards. Having pictures to accompany words means it’s easier to confirm you and the participant both agree on what a word or description means. 

When you conduct interviews and analyze feedback, look for patterns between participant answers and experiences, but remember to dive deeper to understand why these patterns exist and what problem is really occurring.

Doing that may take a creative approach in questioning.

Ask the Right Questions and Dig Deeper for More Insightful Findings  

Asking the most effective questions during research is an art. And when conducting human-centered research, it ultimately comes down to understanding who you’re talking to and asking strategic questions on the fly to dig into the right pieces of information.  

Prepare a discussion guide which covers the flow and topic of the discussion you’d like to have with the participant. Always start with some questions to establish rapport with the participant and get to know them as a person first before moving into the relevant topics for the interview. Marianne always includes a few personal questions at the beginning, such as “What happened yesterday?” and “How are things different from a year ago?”

Use the topic questions as a guide to explore unexpected (but relevant) topics and investigate when something interesting comes up. Just remember there’s no need to get through every question or ask them in any particular order. Instead, use the answers to these questions to determine where to dig deeper and guide the conversation. 

When digging deeper, stay empathetic, listen well, and match their tone. “Tell me more about that” is an easy question to ask when you hear something important. You can also try repeating back to the participant what you heard (e.g., if someone says, “That was really frustrating/awesome,” you might follow up with, “What was frustrating/awesome about that?”).  

Humans are complex individuals with many nuances, so look at facial expressions and tone as compared to what they say in their responses. Sometimes what’s interesting may be in the mismatch between what they said and what you observe. This is a moment to point out the discrepancy respectfully and get the participant’s take. Of course, you still need to be attentive to the subject matter of participants’ answers, since this will tell you what is important to them. 

Question 2: How Should We Build It? 

When you reach this phase, you should have the big-picture concept of your solution (a.k.a. what you should build). Now it’s time to work toward building a more specific solution. Essentially, it’s time to take a good idea and make sure it meets the needs of your audience.  

Schedule prototype testing for potential solutions as needed throughout this process. This allows you to catch issues early and minimize problems with the final product. 

Prototyping for a User Interviews screener survey feature.

For example, Marianne offered this example research scenario of refining an idea for a contactless payment kiosk in restaurants. 

With worries about the coronavirus spread, the kiosks are meant to be a solution to keep sales up without putting customers' and employees' health at risk.   

In developing the user interface for the kiosk and testing it out with customers, the team notices a pattern amongst many users: Some users are spending an unexpectedly long amount of time at the kiosk. 

But what exactly is causing some patrons to spend more time at the kiosk than others? 

• Does the kiosk’s software run slow and cause long load and wait times?  
• Is this a trust issue where these users watch for their purchase to complete and wait for the home screen to appear? 
• Are these users buying multiple products and having to place each order one by one? 
• Does the kiosk have a hardware problem? 

These are all possible problems the user could be experiencing, but they’re also all different problems that require different solutions. Understanding the root of the issue will help the team focus on real-world solutions for that particular issue.

Avoid the “Crash Test Dummy Approach” to Research 

Particularly important during the “How should we build it phase?” is to avoid treating participants as “crash test dummies.” This is when we fixate on the particular solution so much that the participant becomes an automaton who needs to get from Point A to Point B in our product. Remember that the point of testing is to get real human perspectives on the problem you’re solving and the new solution you’re proposing.

“I think this is the danger of falling in love with your idea and forgetting this is a human we’re designing for,” says Marianne. “You need to understand their thoughts, their beliefs, their attitudes, all of that, rather than focusing and fixating on the solution. Focus on observing whether this possible solution fits into their life, rather than if they can just complete the tasks you have listed in your test script.” 

To avoid this pitfall, Marianne suggests always starting your user sessions with general, open-ended questions to get a sense for who you’re talking to; this contextualizes user feedback and helps you ask better questions to gauge how you can tailor your solution to fit their needs. 

Note: Looking for the right folks to talk to? Find your first three participants for free. Or, streamline research with your own users in Research Hub (forever free for up to 100 participants).

After you make the finishing touches on the design, you’re ready to launch your product.

Question 3: Did We Build It Right? 

After you release the new or updated product, look at feedback from your customers to gauge what’s working and what isn’t about your product. Marianne suggests usability studies, product surveys, and customer reviews to gather feedback on your product post launch.

Use that information to decide what’s next.

Conducting Remote, Human-Centered, Unmoderated Research 

One method you can use during the “Did we build it right?” phase is unmoderated usability testing. Unmoderated, remote user research is gaining popularity because it allows for faster research while you still gain valuable information. It’s also just a great alternative to in-person testing during today's time of quarantining and remote work — as long as you ask the right questions. 

For example, Marianne’s team is using asynchronous questioning and recorded sessions on usertesting.com to get feedback (for example, for usability testing). They send a list of tasks and questions, the research participant records their answers and actions, and then the team sends follow-up questions to be answered the same way.

“Since you aren’t there in person conducting the sessions, writing the follow-up questions is much more like survey writing,” says Marianne. You want to avoid phrasing your questions to prompt yes or no answers, and instead, try to get the user to provide detailed context for their responses.  

Even unmoderated testing should include a few questions to get to know the person first. In moderated user testing, she sometimes uses those details in follow up questions (for example, asking if their spouse helps them with a specific task if they mentioned having one). In unmoderated testing, it’s still helpful to get them talking, learn about their lives, and gauge how open they are to answering your questions. 

When you’re preparing the test, remember that you want to avoid the Crash Test Dummy pitfall: You don’t just want to see if end-users can get from Point A to Point B. A key question for the research to consider, Marianne contends, is “Does Point A to Point B make sense within their context?”Just because a user can complete the step doesn’t mean it’s what they want, nor does it mean it’s the most direct solution for their problem.

Repeat the Process 

The human-centered design process is iterative. After the launch, it’s time to begin asking the question “What should we build?” again. Are there extensions to current functionality that are now needed? Are there new needs that arise once your product meets the initial needs your customers had? 

Leveling Up Your Human-Centered Design Methods 

Marianne offered a few suggestions for improving your research skill set: 

• Learn from developers, product managers, designers, marketers, and anyone else who can be involved in your research. 
• Seek out opportunities to pick other researchers’ brains and look for chances to collaborate. Working with other researchers allows you to see a different point of view and shows you new ways to approach your work.
• Don’t become complacent in your skills. Stay on top of tweaking your questions, and focus on asking different questions in your research, too. Look for blind spots; these pinpoint areas where you can improve.  
• Pinpoint recurring pitfalls or problems in past research. How could you have avoided those? Are there any areas in your research methods you feel could use some refinement? If so, choose what specific skill sets you’d like to strengthen and start practicing.  

Marianne suggests reading the book The Design of Everyday Things by Don Norman to learn more about human-centered design. If you’re a fan of podcasts, she recommends tuning in to Dollars to Donuts and Mixed Methods. 

For Your Next Study, Consider User Interviews 

Even if you follow the best methods to apply human-centric research within your design process, your efforts won’t be as successful unless you recruit and engage with the specific type of people who will use your solution.  

To find the best participants for your research, consider User Interviews. We offer two tools to help researchers manage recruitment:

• Recruit, which helps you find participants who aren’t necessarily existing customers
• Research Hub, which helps you manage research with your own users.

Via Recruit, connect with participants from our database of 350,000+ vetted professionals and consumers. Set your criteria, then narrow in on your favorite participants based on their screener survey responses. Our median time to your first matched participant is just two hours.

Via Research Hub, import your users into a single location where your whole team can coordinate. Know exactly how many times each customer has received a research invitation or has participated in research, see exactly how much they’ve been paid, and review their response history. You can set up any custom tags for easy filtering during future studies.

On both platforms, get help with automatic email reminders, painless scheduling, and incentive payments.