Shay, the collie Mana Hayashi and Hirone park of ventera Shayla call it Shayla Collie, is a senior ux designer, currently working on a master's degree in human computer interaction with a focus on accessible design solutions
Accessible design solutions at Ventara Shaylor works as the Ux team lead to support human center design goals for Ccs queues, enterprise systems and services.
Or Es Mana Yashi is a senior ux researcher with over 10 years of experience in mixed methods, research, education and science, writing LED.
Ventera, she conducts research on a cross functional product team to improve user experience of Ccs queues, enterprise, systems. And services.
Again uss, including program resource system known as Prs.
2 and robotic process automation or Rpa
There's my cursor here
Hi! Room Park is a senior Ux researcher with over 10 years of experience working with, government and it companies currently hireum is senior year senior researcher at Vintara, where she conducts quantitative and qualitative research on how user experiences can be improved based on research
Findings and recommendations, with a focus on harp and mft, file, cloud and Ccs queues.
Es welcome, Shayla, Mona, and Hiram.
Turn it over to you
Good afternoon, everybody. Go ahead and get my machine screenshot for you
Can everybody see my screen. Okay. Okay. Awesome. Well, good afternoon.
And thank you guys. So much for taking the time to join Mona Hiram and I today so a quick question for everybody.
What does it mean to be continually in engaged for users it means we are regularly in contact with them, asking follow-up questions and collecting their feedback, and to do so the whole project team has to be continually engaged?
With one another, so let's look at how Ux practices engage users and why it's important to continually engage with them as part of the human centered research and design process, and before we jump into our topic we wanted to go ahead and introduce ourselves so as brian mentioned my name is
Shane Mccoy. I'm a senior ux designer at Ventura and the Es Ux team lead and in today's presentation.
I'll be showcasing how we work together as a team to continually engage our users in the creation of human centered products for the center of Medicare and medicaid services
Good afternoon. My name is Mana Hayashi Tang, and, as mentioned earlier, I'm a senior ux researcher on the Es.
Team. I will be introducing examples of how we engage users in the development of one of our early stage products, called Prs.
2 which is used by Cms. Contractors to review the healthcare quality that Medicare, medicaid patients are receiving from their doctors and health care facilities
Good afternoon. Everyone my name is Julie Park. I'm also senior in Vxi.
Researcher on the Es team. I'll be introducing examples of how we engage users in one of our merchants based products for the heart.
We see the secure item teammates, one portal to get access and use several Ccs queue, calling that applications
So our team centers user participation in our product development process because our users are ultimately the people who will be using an experiencing our products and we follow and highly recommend 3 sets of best practices when conducting human centered research and design so let's take a look at what it takes to build
a foundation for continual user engagement.
So first we have to collaborate cross functionally, and this is because the best solutions for our users are often found when experts from different backgrounds and perspectives are involved in a user engagement and all members in the product team are involved in continuously engaging with each other and the users
Then we can continually engage our users. This is because we're centering user voices, and every decision making process of our product whether it's at the early stages in order to validate a design concept or after a product is mature and can benefit from further enhancements
Lastly, while working iteratively, it's important to continue engage users so that we think holistically about their experience.
Human-centered research and design should keep the big picture in mind because our users experience products holistically
These 3 best practices help us center user experience in our team's product development process.
But how do you incorporate these best practices into your product teams, workflow, and how does continual engagement actually bring value to our users?
So that's actually what we hope to share with you all here today.
For the first part of our presentation we're going to discuss how the Es Ux team is a shared service across multiple product verticals, and how, as a shared service we collaborate with our project teams to regulate engage our users, for their feedback
So a little bit about us. We are part of the enterprise systems and services team also.
Known as Es. For the centers for Medicare and Medicaid services typically referred to as Cms.
Es maintains applications and services that support health, care, quality, improvement, initiatives, and es is comprised of 4 product teams.
Our ux team is part of the larger shared resources, and as a shared resource service we work to support all of the project teams and their users within es
We approach our human center design practice as a means of facilitating communication between the end users and our internal project teams, and in our experience working cross functionally has been really helpful on ess to improve collaboration between design and development
Here is a high, level view of our cross. Functional workflow.
This process starts with identifying a user's need, and this could be through a stakeholder request directly from our research or from direct conversations with our users.
We then conduct research with our end users and the project team which includes leadership, Ux and development, and after the initial research phase Ux works on making updates to any designs based on our users feedback research then retests the improved designs with our end users again and
including the larger project team. Once our design meets user expectations.
Ux works with development to implement the final design, and at this point, given the iterative nature of the agile approach, the work process starts all over again
So let's take a closer look at the second stage of this workflow, which is user experience, research with project and Ux team support
So continue engagement of users begins with setting up a solid cross-functional practice in the research stage.
When a researcher conducts a study with users, for example, a user interview, we can see how this cross-functional collaboration plays out to engage users throughout the product development work process that Shayla has just introduced to us we usually have a researcher assigned to take lead for
Each product and that person will facilitate a one-on-one conversation with the user, so that the user can have an in-depth and focus conversation with the leading researcher with the lead researcher now the other researcher.
And the Ux designers on the team will assist by here reviewing the interview, script before the interview and then taking notes and helping out when there are technical issues during the interview
We also invite our product team, members leadership and stakeholders to our interviews as observers.
Now these observers can be developers data team members, for instance, who are sitting in to take their own notes after the interview.
Observers can ask follow-up questions to the user.
They will also stay behind to debrief with the Ux team.
Members. This setup helps us think ahead and discuss what development needs at the research stage with the users
Now let's take a look at how this interview structure specifically worked for one of our products.
Prs. 2
The primary reason for users to use Prs 2 is to search for information on Medicare or Medicaid doctors like their credentials and history, of providing care during one of our Interviews we showed the users this high fidelity.
Design wireframe with information on a hypothetical doctor
Now in this wireframe the user noticed that under group practice it says Group, but not the name of the so we discovered that the users do like knowing a doctor is a part of a group, but also wishes to easily identify doctors group practice affiliation.
On this page. Since we had data team members attending the interview session as an observer, they were able to directly ask the Users data type related follow-up questions to this use case.
What information do you need to see here? They asked, together with the users and the designers. We were all able to come up with a best solution for the user
Now users will be able to see a doctor's group affiliation active and previous group membership.
Their time for membership, and they will also have a way to click into a group and see its further details
Here are some benefits of working cross-functionally when conducting ux research like this, including users can directly speak of their needs and concerns to the product, team because of the ux teams facilitation
These research settings also facilitate the product team members, empathy for users.
Empathy can develop more naturally when product team members can hear directly from the users
Deep briefs. Directly after an interview between the researchers, designers, and other product team members are also very important.
They make sure that all the ideas are shared with each other while fresh on everyone's mind, and it also nurtures team cohesion over our product goals always always continually centering them on user needs by having all product team members engage in user research from the beginning they are introduced to
The users, and can take what they learn from the users into the rest of the work process, laying the groundwork for continual user participation
Cleverly issues to continually engage users can also extend beyond the internal key, communicating and collaborating with other teams who provide support to our products help prioritize user means over the scope of individual teams
For my project harp. We worked with the help desk team because they support hardware users when Harvey, New Jersey reach out to the help desk to get help they know the most common use cases.
Why Harvey users reach out to the help desk
So help us team shared when harp users contact, help desk.
Here are examples. For why harp you just contact to help desk.
They fail to log in. They forgot their password and have issues when doing manual identity, proofing during registration based on it findings, we conducted us system to solve these problems
Here is what we provide to deal with common use cases when we just need to help.
We decide to update the contact text page to allow users to find their used case, that they need to help and also provide self-reliance features to help them, try to solve their problems by themselves before reaching out to the health desk
Here's the previous page before we updated issues. Various contact information based on what application we just use.
But the only thing you just could to hear was find the relevance, contact, information and the child to the help desk.
There are no other additional resources beyond this. Here is the update context page you can see.
It provides like the dropdown options to easily find the user cases that the users have to see for help.
Also it provides links for how to to self recovery before reaching out to the hub desk based on saladated use case
With this new update. Each shows contact information to reach out to the different help desk as well based on their user case and application, that they use
Some benefits, of like collaborating being the product team when conducting user research continually by working with other teams who support or H interact with the product users.
Other team can discover usual pain point in your town.
Continue your user engagement, help the product team save time and effort to conduct, to research, to figure out user pain points. The product team can have better understanding of where and how many users in counter issues and problems to competitive task when using products
Collaboration with other teams can deduce workload for everyone
Users cancel issues without contacting multiple teams.
For this happy example, users can now solve their issues on their end, with the provided resources in the product, without reaching out to the help desk
Product team can understand the pain points like sooner, and the scale of issues that are problem and known by others
Also it can get used to work for other teams who support relative users because you just cancel their issues on their end before reaching out to the other teams
Now that we have looked at how cross functional collaborations in Gwax practices can continually engage users.
Let's discuss the importance of maintaining user list for future research, research efforts, and how continual engagement allows for nurturing relationships with our end users to encourage their continual participation
So we work on building relationships with our participants, who often volunteer for our studies during early product development.
These participants are people that we can keep in touch with throughout our work process to validate our ideas as well as ask any follow-up questions between research studies
We maintain a separate user list for each of our products and different user groups. And this helps us.
Know who to contact, depending on our research goals
Here's an example of a user list. Each column represents a different study, and each row shows whether a user receives a study invite and which studies they participated in.
We can track which of our users are continually participating in our studies, and what topics attract their attention
As your product mature is your user list may grow significantly in our case. Prs.
2 is an early stage product with 700 users, while harp is a mature product that has over 2,000 users.
We tend to send out surveys to specific user lists in order to receive large-scale feedback.
We then seek out specific proactive users from the survey participants and previously identified frequent participants in order to conduct our interviews
It is important to nurture relationships with your users in order to engage their regular participation
Now let's look at how we can achieve continual user participation from an early stage product to a late-stage product.
Let me start with an early stage one.
When trying to identify the core needs and associated pain.
Points of Prs. 2 users. Our team was initially unsure.
Who would search for doctors and health care facilities on the project which is based on a legacy government product for the same purpose
Having a curated user list helped us identify that the majority of our users are patient facing these are users who educate patients and advocate for patients and their families when the patient files a quality review case against the doctor or facility that the patient's saw and was seen in
Because we figured out this primary usage scenario by regularly communicating with our user groups, we learned that the information users have to start a search on Prs 2.0 tend to be from patients
For example, this user: is happy to be able to search for a health, care, facility by their address, because it is one of the information that patients would provide them aside from the doctor's name and their facility this functionality addresses one of their pain points on the legacy version of prs that health care
Facilities often change their name, making it difficult to match information from the patient with information in Prs users validate the usefulness of a functionality we added for their core needs, making their workflow easier on our update product prs 2
Some of the benefits of continual user participation in research starting from early stage of a product include nurturing the user's status as knowledge holders and collaborators early in the product development process.
So that users can see the impact of their involvement.
Every step of the way as a product becomes successful in fulfilling their core needs.
Helps establish that users that the users continual investment in research and development is ultimately an investment in their community of of users
Here's a user expressing this feeling of inclusion that helps build trust in our work process of continually engaging them to make Prs a better product for everyone building this confidence.
Is particularly important in a U. S. Sector like ours, where users have to rely on particular government products to support patients and healthcare workers.
The user's continual involvement is crucial for us to provide a high value product to them in a timely match
Next. I'm going to talk about how continually engaged users into a martial late stage product can be beneficial.
At this stage we can explore you enhancement because the core functionalities to provide more values to users when they use product
We're launching the minimum valuable product over her.
There was no landing page, so us team decide to provide designing page app to look in, to guide what users can do in her
After coming out. There's a new concept we conducted user testing to validate each working as we designed it.
Let's look at the Wizard, of how we influence the landing page by validity, by validating with users
Before we provide the new dashboard style landing page there was no landing page after looking users had to get on this profile page, and they need to go to somewhere else.
Depending on what they want you to do
It's one new desktop style renting page.
We provide after validation meeting, meeting by landing on this page.
After looking users, can't know what they can do in heart. Intuitively with this, like a hired way at Shill, cut
Doing to use your Tasking participant also mentioned same value of the points of why we wanted to provide to ending page
Here are benefits of validating new features, with the users continually in virtual products.
With just new in new features, you just still complete their Cornese, and they can enjoy new inhabitants that impose their workflow and understanding product
Also validation with users before implementation prevention wasting time and resources.
When you working the product after it's not successfully working after implementation without the research to validate meeting users.
We don't have any chance to know if it's working as a design day and expect it when you new features are provide to users in your work.
Where're not going to discuss ways. Our team works continually with users to holistically improve products and services
Thank you. In the previous section. We looked at how continue engagement of users can benefit their experiences with early stage products and mature late-stage products.
Now let's look at how continuous user. Participation can impact user experiences as a whole when we're working through iterations of a product.
We're often focused on particular features or a portion of a product but receiving feedback from users helps us keep their full holistic product.
Experience, in mind.
For instance, at the beginning of a user interview we can ask users open-ended questions to seek for spontaneous feedback like what are some questions that you have for us.
Before we get started. What's important about asking this question at the beginning of a study is that users tend to be honest and provide their first thoughts and underlying feelings about the full product, especially when you create a conversational atmosphere users, are more likely to share with you a pain point that they have
Most recently encountered, or a deeply seated concern that they have for the whole product
This honest feedback is especially important for an agile product team like ours, and the leadership to hear, so that we as a team are not solely bound by the work iterations we determined for ourselves in our silos but instead users get to tell us and help us determine what is important for them and when
they experience the product holistically, and help us adapt to their needs.
When providing a product or service to users. It is very important to have an open mind, an incorporate multiple opportunities for users to correct your assumptions and buyices, so that you can identify gaps in your research scope
With an experienced user someone who has been using your product for a while.
It is useful to ask open-ended questions. At the end of a survey user interview.
Or user test, to hear more about their ideas and thoughts. So far on the product.
Now this could be as simple as asking, Do you have any other comments or suggestions for how we can improve the product?
By asking this question at the end of a research study we free ourselves from our research scopes. What are we?
The product team overlooking. And what do you? The user wish we could provide instead
We also can uncover user needs and expectations that go beyond their core needs.
Maybe a nice to have feature that they've liked in other products all of these additional findings and insights should be shared with the team.
All the team members in research readouts, presentations, meetings, to make sure that everyone or is keeping these user ideas and holistic experiences in mind
Can you, when you have additional findings, speed on your current to the search scope.
It is a helpful to archive in highlight them to measure all team members
To archive all additional findings and insights, create a repository where you can keep track of them whenever you need to take a look
Start all findings, and insight with the user codes, so that they can be seed ideas for future research.
When you make a plan to provide better inx periods for the product.
This practice make sure the user's previous participation in this search continues to impact your future research
In order to have users participate in ux research.
Ongoing efforts are required, provide a blizzard link in the product to ask them to complete the software to gather their feedback on their product. The usage
When inviting the survey in the Product Soviet design is really important together.
Large scale of feedback from product users, be cool to you, pass to usually participation, so it might be easily also that users can open anytime.
They want
Consider, to provide, sign up link for users to participate in feature.
Ux research recruitment is always challenging. So if you have a place where users can sign up for future you access, it would be beneficial to find more users continuously
Oh, here's how her provides a link for online stopping sign up for feature.
We express her.
As you can see, contact us to sign up link allow users to sign up for future New York City. Search
And also give us feedback. Link provides a live sub a to users to get their feedback.
Edge of Richard Harvey. Users can participate in the search at any time when they are using the product
So in our talk today. We went over a lot of information on how we have users participate.
Continually in user research and design, so what can continual engagement look like for you?
And how can you start engaging your users today? Let's do a recap of our best practices and then take a look at a checklist that we've created to help you apply them to your products and workflows
So the first thing is product. Team works regularly with the Ux team to communicate with the end users.
As to consistently manage user lists, to recruit and build trust with their users.
And 3 working in iterations. But keep in mind the user's entire journey with our products
Now here are some pro tips. If you are a Hcd human-centered design expert struggling to get product team members to join your Ux activities share with your product team members are examples for how data and development team members how stakeholders like clients can learn from users, involve them so that
You can save time in your overall work process together
Continuously communicate and collaborate with other teams who support your product users.
Schedule. By working together we can't make better, or substance or user experience for users beyond our own product.
Scope.
Next, encourage your users to experiment with your early iteration products, and to give you feedback.
Make them feel comfortable by letting them know that they're not being tested.
We are together testing a product. We're collaborating with them
When you gather these user feedback and implement them into your product, demonstrate how their participation impacts the product.
Show users that their input matters and show stakeholders that continue research with users matters
After launching your product of your how your users interact with your product with the detailed eyes, to discover any problematic areas in what flowers or what can be info to deliver better user experience?
If you find any problematic areas come up with new ideas, concepts, or features to solve and validate them.
Meeting users before implementation, always including your users
As researchers. We care about the scope of our research to prevent issues with scope creep.
However, it is still important to always make time to ask users open-ended questions that capture the holistic experiences, user's knowledge from having practical experiences with a product can help catch possible research or biases remind us of the big picture incorporate and incorporate opportunities for users.
To correct you.
Great that key to find the incident repository to keep travel.
The search findings and start them with the sorts of queues, in order to search them.
Whenever you need, you can keep directions for a future
Think of what can be embedded into your product to get user feedback and to record participants.
Well, let's survey and sign up. Link for feature.
Research can be good example. Users are always there to use your product.
So try to record users from where they are, keep place it open for you.
Participation in your product
Now, if you are someone who is involved in creating a product, whether you are a user experience expert, a product, team, member or a stakeholder, we hope this checklist of pro tips.
Will help you incorporate continual user participation into creating your products. Starting today
Also, if you are a user of Ccs Q and Cm.
As product. Please know that we already welcome your feedback participation.
Please, feel free to reach out to us or stay behind. After this talk for a while on conversation on our products.
I will work, process and anything else related to user experience that picks your interests.
We will to off speak with you.
So thank you everyone so much for joining us for our talk today and the team, and I will now take any questions you might have