With an exciting line-up of engaging speakers and diverse topics about trust, ethics, and integrity, you will not want to miss anything! Do you have a busy day, or an upcoming deadline? We get it. That is why the Zoom event is an open-house format with sessions throughout the day and breaks in between sessions. You can even sit down with us during lunchtime for an informative panel discussion. . 

8:30- 8:45 | Welcome from CMS Leadership 

Presenter: Roni Garland-Wynegar 

8:45 - 9:30 |Keynote Presentation: Navigating the Complexity of Trust  

Presenter: Carol J. Smith  

Keynote Presentation 

Carol J. Smith from Carnegie Mellon University will explore trust and how UX practitioners can define and measure it. 

Areas of interest: data, design, leadership, product, strategy, and technology 

9:30 – 9:45 | Break 

9:45 – 10:15 | Customer  Customer Engagements Using Human-Centered Design Design 

Presenters: Suzanne Martin-Devroye and Morgan Taylor  

Presentation 

Join us to learn from the Customer-Focused Research Group who will share an overview of their work across CMS and how it informs policymaking. 

Areas of interest: design, leadership, policy, and strategy 

Artificial intelligence (AI) seems to promise wide-ranging automation of many familiar civic technology components by providing just-in-time help to website visitors, performing audits and calculations, and personalizing services to specific users. However, in practice, these “artificially intelligent” systems generally rely on complex combinations of humans and machines to produce user experiences. This requirement poses unique challenges for service design and system transparency, with profound consequences for overall user trust and user experience.  

In this presentation, we take a sometimes-speculative look at how we have been thinking about “heteromated” systems such as chatbots and measure calculation on Healthcare Quality Reporting and beyond, providing both a theoretical overview and some practical guidance for UX professionals working with artificially intelligent systems. 

The recording of this session and a copy of the presentation slides are posted above.

Even though AI has advanced a lot in the past six years, the legacy AI approach has a problem: data must be consolidated in one location for the machine learning model to be trained. That means data are exposed, and the data owners lose their data privacy. Even worse, unlike tangible objects, the data exposed can be replicated to potentially hundreds and thousands of times with just a click of a mouse. That makes recovery almost impossible, and hence people now treat data privacy more seriously.  

The result is insufficient data, which creates another dilemma of hindering the growth and maturity of many AI models as they rely on data. Where should data owners draw the line to determine what they can and cannot trust? In insecurity discipline, there is a methodology called Zero-Trust. Can this be used in AI so that we trust nobody to hold our data but can still help advance AI? 

There is a new branch of AI called Federated Learning. The concept includes data consolidation to train the model, and the model is pushed out to where the data is located for training. The individual result will then be sent back for aggregation to form the final useful model. Sounds very promising, right? But can this be THE solution to solve the trust issue?  

Attendees will: 

• Understand the risk and opportunities of this technology and see In What We Trust, 
• Learn the latest trends in AI – Federated Learning, and how this technology can provide an AI model integrity, and 
• Understand the latest trends and end-user data trust. 

Imagine a world where Medicaid and Medicare beneficiaries are empowered hand-in-hand with providers focused on improving their care while also managing costs. How can we incentivize clinicians and beneficiaries to participate proactively in the CMS CCSQ mission? 

Machine Learning (ML) can help drive this vision. ML presents all-new capability options for CMS, but it's not a panacea. There are challenges and opportunities to consider when moving from predominantly post-care quality measures and traditional analytics to a proactive ML model-driven approach. Humans are critical to the overall performance of the recommender (ML Models) systems. 

We will discuss the challenges and capabilities of four main topics: 

• How can we train ML models to help drive the CMS CCSQ mission focus: Steadfast focus on improving outcomes, beneficiaries' experience of care, and population health, while also aiming to reduce healthcare costs through improvement." For example, AML recommender systems might include health goals, treatment outliers, fraud, waste, abuse, treatment trends, and anomaly detection. 
• Where is the CMS data? Discuss privacy concerns of training data such as using federated learning. 

• What data/feature engineering challenges are there inherently with Medicaid and Medicare data? 
• Discuss which humans (actors) should participate in ML to ensure safety, reliability, and trustworthiness. How can beneficiaries remain in control of their participation, privacy and continually improve the trustworthiness of AML?  

Conclusion: Human involvement and participation is an essential and necessary element for ML unbiased trustworthiness, reliability, fairness, and safety.

A recording of this session is posted above. Please note there was not a presentation deck for this session. 

According to Pew Research, public trust in the government nears record lows and the federal government has recognized the need to improve the way it serves its citizens. One response has been through the publication of OMB Circular A-11 Section 280, which guides how all agencies should prioritize managing the customer experience and improving service delivery. 

Dark patterns are designs (digital or non-digital) that erode trust by intentionally or unintentionally tricking people into doing something they don't intend, want, or need. These mistakes usually cost people money and always cost them time. The purpose of this panel is not to discuss ways that the government has failed the public. Instead, the goal is to focus on examples of bad design most often seen in the commercial space and how we might respond to these examples to rebuild the public's trust in government solutions. 

Attendees will: 

• Hear diverse perspectives and ideas from a panel of design professionals, 
• Have an opportunity to learn from the mistakes of others, and 
• Understand how to leverage principles taken from lousy design to inform ideas for rebuilding trust by improving the products and services we deliver to the public.    

The talk will present a case study focusing on hepatitis B patients and an effort involving researchers from Stanford University and others to determine patient-centered tools to better understand non-compliance from the patient perspective and improve the outcomes of the disease. In addition, the project is looking for better ways to collect, manage, and communicate public health data among providers, caregivers, and patients with hepatitis B. 

This evidence highlights the need to improve patients’ disease management and adherence to their biannual monitoring and hepatocellular carcinoma (HCC) surveillance. Progress of this nature will lead to identifying individuals eligible for treatment and early detection of HCC. In addition, appropriate tools, such as those presented in session, can have a meaningful impact on patient engagement and empowerment, making adherence to care plans and better outcomes more promising.  

Attendees will: 

• Learn how human-centered design (HCD) enhances patient-provider communications and makes data more useful in a clinical setting. 
• Understand how monitoring and managing chronic hepatitis B infections lie equally on the shoulders of the patient and the healthcare provider. 

 Using Human-Centered Machine-Learning (HCML) to Improve Data Quality & Data Governance Projects 

Presenter: Edward F. O'Connor 

2:30 – 3:15 |Your Chart is a Bigot: Ethical Data Visualization in Public Health 

Presenter: Edward O’Connor 

Presentation 

This session will include a practical review of data visualization in a public health planning, decision-making, and policy-making context while focusing on fairness, equity, and measuring the efficacy of programs over time.Presentation: Are you interested in understanding the components of a real-world and complex Machine Learning (ML) project? Join us as we walk through the implementation process of combining Human-Centered Design (HCD) techniques into a ML project.  

Areas of interest:Data data, Designdesign, Productleadership, Strategy, Technology

Session materials:

Recording - November 12, 2020

Slides: WUD_OConner.pdfpolicy, product, and strategy