LACE Program Increment 4 Accomplishments

Objectives:

• Program engagement
  • Present discoveries QMVIG with front office & divisions within QMVIG. Sharing observations on what has been discovered and the plan to implement
  • Present, discuss, and create alignment with customer on the strategy to evaluate the problem to solve for iQies initiative
  • Meeting with CMS lead to discuss and understand existing QIO contract to identify if there will be any engagement for HCD or LACE . (intake process)
• Courses:
  • LACE explained: OKR
  • LACE explained: Program Backlog Management
  • User Story Clinic
  • WSJF Clinic

• Dashboard improvements
  
  • additional report added to the program dashboard -historical delivery of work by type

• Internal processes
  • Training templates Color palettes 

Value Delivered:

• We have successfully identified the requirements and needs in order to develop workflow for QMVIG support team that will bring transparency and value that has been defined and captured. 
• LACE has worked closely with HCD and followed the Memorandum of Understanding (MOU) for both Centers of Excellence. Jointed efforts have been conducted to align with engagement strategy for IQIES HCD team, as well as documentation storage with discoveries ,and timeline defined.
• LACE/HCD met with Danita Patel, Adam Heller and others to discuss engagement efforts for QIO. At this point the efforts will be only focus on HCD research of systems for QIO Contract, understanding DARTT System, users and how to improve them.
• LACE delivered two short courses covering the Objectives and Key Results, Program Backlog Management, and provided the community with a clinic to improve User Story authoring
• The Program Dashboard was updated to include insights into how much feature, enabler and technical debt was delivered in past and current PIs
• The Team pulled deeper historical data into the existing dashboards to address leadership request for more insight into the impact that CCSQ LACE has provided the enterprise.

(Some planned work was removed from the iteration to accommodate a CCSQ leadership request for information)

Objectives:

• Program engagement
  • QMVIG team ongoing coaching efforts to focus on Team working agreements to ensure the support team members understands and agree. 
  • IQIES HCD coaching engagement with CMS-HCD point of contact to analyze Jira Boards and metrics to identify anti-patterns and document results of analysis. As well as addressing DoR, DoD and Team level working agreements are examined. 
• Courses:
  • LACE explained: Roadmap
  • User Story Clinic
  • OKR Clinic
• Dashboard Support Request
  
  • Jira Projects added to CCSQ Enterprise Metrics Dashboard (AiOps: AI/ML Scrum Team, AiOps: Operations and Support Scrum Board, AiOps: Software Engineering Scrum Board)
• Internal processes
  • Training templates Color palettes 
  • Standard Operation Procedure (SOP) for course content development. 
• SAFe for Government Day Conference (2022) LACE will attend to bring some information about best practices within the government space to CCSQ

Value Delivered:

• We have successfully delivered LACE explained Roadmap training with 11 attendees (4) Feds and (7) contractors.
• AiOps Dashboard support request completed. Data source imported for (3) teams: (AiOps: AI/ML Scrum Team, AiOps: Operations and Support Scrum Board, AiOps: Software Engineering Scrum Board). CCSQ LACE Team Dashboard now shows these respective teams and their metrics are helping them to become more mature with agile practices.
• LACE attended SAFe DAY Government 2022 conference.
• iQIES HCD coaching support by analyzing Jira Boards and metrics and a document has been created and stored with the results of the analysis: https://qnetconfluence.cms.gov/pages/viewpage.action?pageId=273355026

• QMVIG Coaching engagement ongoing support efforts to leadership and teams. LACE had (2) sessions of Backlog prioritization with the program and will be working on building Team Working agreements on the week of 4/25.

• Met with Brandy Barnette for LACE continue Feature Refinement decomposition of work and priorities.

2 points abandoned

( LACE-2790  LACE-2790 - Getting issue details... STATUS )

( LACE-2778 LACE-2778 - Getting issue details... STATUS ) 

Backlog

10 pts:

(LACE-2648 LACE-2648 - Getting issue details... STATUS )

(LACE-1443 LACE-1443 - Getting issue details... STATUS )

(LACE-2842 LACE-2842 - Getting issue details... STATUS )

(LACE-2836 LACE-2836 - Getting issue details... STATUS )

Objectives:

• Internal processes
  • Training templates Color palettes 
• Courses:
  • SAFe POPM
• Program engagement

• • QMVIG team ongoing coaching efforts to focus on backlog refinement to generate a healthy backlog and prioritization. 
  • IQIES Sharing observations from PI Planning and Jira analysis with CMS leadership. Be in coordination for next steps.

• Dashboard improvements
  
  • Updating the program wide backlog composition report to show backlog state at the beginning of a PI

Value Delivered:

• Finalized the Master Training PowerPoint template enabling the team to easily develop PowerPoint training decks with standard layouts and switch between seven different color palettes.
• LACE Facilitated SAFe Product Owner/ Product Manager certified training to the CCSQ community with (12) attendees. 
• LACE will be offering Study Hall Sessions for those that attended SAFe POPM over the next few weeks to provide participant assistance in preparation for exam certification
• Enabler work was conducted on data collection from LACE Training register for fed vs. contractor and Pivot Table created with queries to narrow participant's attendance from Eventbrite. 
• Dashboard Support inquiry follow-up for End User Computing and Access Team https://qnetjira.cms.gov/browse/LACE-2706 
• Program engagement support for iQIES HCD team. LACE examined and assessed the current stage of team social contracts: (Team working agreements/ Definition of Ready/ Definition of Done). Documentation stored here: Team Social Contract Analysis
• Feature Refinement Session with Brandy Barnette and scope identified for Program Dashboard
• LACE had a demo with EQRS program leads and RTE on Program Dashboards Pilot. 
• iQIES: progress towards the completion of the discovery phase is shared with CMS leadership. Potential risks with the HCD team restructuring are discussed.
  
• QMVIG had multiple refinement sessions which has created a healthy backlog. Task have also been assigned to the team.

Backlog

6 pts:

(LACE-2684 LACE-2684 - Getting issue details... STATUS )

(LACE- 2842 LACE-2842 - Getting issue details... STATUS )

Objectives:

• Courses:
  • LACE Explained: Social Contracts
  • WSJF Clinic
  • SAFe POPM Study Hall
• Program engagement
  • IQIES
  • An overview of some Agile Ceremonies with QMVIG division leads to help adopt a Kanban process
• Dashboards
  
  • Updating the program wide backlog composition report to show backlog state at the beginning of a PI
• SOP
  • Continue documentation of Course Development SOP to enable the team to use the new Master Training PowerPoint deck.
• ROI
  • Establish a calculation method that captures ROI for LACE
• LACE attendance
  • Develop list of attendees per course Fed vs. Contractors for courses

Value Delivered:

• Facilitated and Encouraged participants while being a part of a great discussion around social contracts and why they are important. (Social Contracts)
• Provided resources to attendees on how to utilize WSJF (WSJF Clinic) https://qnetconfluence.cms.gov/display/LACE/Toolkit Weighted Shortest Job First (Feature Calculator) 
• LACE provided SAFe Program Owner/ Program Manager Study Hall session to get attendees prepared for SAFe exam
• Dashboard Enhancements: 4 different measures were created for EQRS PI's. A dashboard for testing was also implemented 
• LACE created Set-up Session in confluence as the team continues to invest in Standard Operating Procedures with efforts to document "Course Development" to enable the team to use Master PowerPoint Deck. https://qnetconfluence.cms.gov/display/LACE/Course+Content+Development+SOPs
• Developed Definition of "Return on Investment" ROI and Value on Investment "VOI" using Gartner's Research Model. https://qnetconfluence.cms.gov/pages/viewpage.action?pageId=279156131
• Created an Excel "Calculator" to develop a formula to capture ROI, services, and Training data from Registrants vs Surveys responses for all "LACE Explained Courses" 
• LACE Iteration Reviewed conducted with Brandy and collected feedback from stories presented
• The Team met with EQRS Leadership to discuss EQRS Pilot Dashboards and define the next steps: LACE will continue to work with RTE on program labels and structure
• Feature Refinement session with Brandy to continue to identify the scope and decompose Feature work 
• iQies: The team structure of the HCD team is changing (offboarding & onboarding of ADO team members). Alignment between CMS leadership and CoE around the current state of the discovery and agreement on the next steps
• QMVIG Definition of Done has been established and captured. All QMVIG support team members understand and agree on the DOD.

Backlog

5 pts:

(LACE-2681 LACE-2681 - Getting issue details... STATUS  )

(LACE-3145 LACE-3145 - Getting issue details... STATUS )

Objectives:

• Courses:
  • LACE Explained: PI Planning
  • LACE Explained: Backlog Prioritization
  • OKR Clinic
  • User Story Clinic
  • SAFe POPM Study Hall
• Program engagement
  
  • QMVIG
• Dashboards
  
  • Updating the program wide backlog composition report to show backlog state at the beginning of a PI
• Course attendance Dashboard

Value Delivered:

• Successful delivery of the LACE explained Backlog Prioritization course with (4) out (7) expected attendees
• Successful delivery of the LACE explained PI Planning with (13) registered for the course
• LACE offered the CCSQ Community with Objective and Key Result Clinic. https://qnetconfluence.cms.gov/display/LACE/LACE+Clinics
• Facilitated and provided a space for User Story Clinic. Where participant refined and revised User Story Writing techniques and Acceptance criteria updated here: https://qnetconfluence.cms.gov/display/LACE/User+Story+Clinic%3A+May+12%2C+2022
• LACE provided SAFe Program Owner/ Program Manager Study Hall session to get attendees prepared for SAFe exam (2 out of 2) Sessions
• IQies coaching engagement preparation by developing checklist templates for Agile Ceremonies in prep of observation and recommendations on best practice techniques for forming teams: https://qnetconfluence.cms.gov/display/LACE/HCD_iQies+-+Scrum+Ceremonies+Checklist
  • Developed DSU checklist document
  • Developed Refinement checklist document 
  • Developed Sprint Planning checklist document 
  • Developed Sprint Review/ Demo checklist document 
• QMVIG coaching support by presenting an overview of Agile ceremonies for DQM 
• Developed Federal Attendance Dashboard Federal Attendance Dashboard that allows easy comparison of federal attendees to total attendees from data timeframe (September 9th, 2021 to April 12, 2022)
• LACE held Feature Refinement Session with Brandy to continue to prioritize and decompose scope of work and any scope changes in priorities.
• Conducted Iteration Review Demo with Brandy Barnette and incorporated feedback loop into our backlog items 
• Attended HCD Refinement Session and discussed dependencies on LACE/HCD Joint Training and Article Newsletter Initial Draft. 
• LACE Attended HIDS meeting in regards to proposed hierarchical planning framework that uses Jira (Portfolio Epics, Program Epics, Features) for ADO's and shared CCSQ Agile Playbook Recommendations: https://qnetconfluence.cms.gov/display/LACE/Work+Structure

Backlog

5 pts:

(LACE-3168 LACE-3168 - Getting issue details... STATUS  )

Objectives:

• Courses:
  • LACE Explained: Estimating Work
  • LACE Explained: Inspect & Adapt
  • WSJF Clinic
• Program engagement
  
  • QMVIG - Session with the QMVIG support team on work items estimation
  • IQIES - Observing scrum ceremonies (Iteration Planning & Review)
• Dashboards
  
  • Features started report accurately 
• Course attendance Dashboard
  • Display graphs
• CoE 101

Value Delivered:

• Successful delivery of the LACE Explained: Inspect & Adapt with (3) out of (9) expected attendees
• Successful delivery of the LACE Explained: Estimating Work with (5) out of (10) expected attendees
• LACE delivered the WSJF Clinic to the CCSQ Community. https://qnetconfluence.cms.gov/display/LACE/LACE+Clinics
• Coordinated with the HCD team on course objectives for joint CoE training
• IQIES - Observation and documentation of the potential gaps during iteration planning, iteration review and DSU
• Completed Program Dashboard Report: Features Started and Closed (Issue History). This report shows the Features that were worked on and completed, in a PI, for the selected program.
• Additional metrics were added to the Course Attendance Report to show differentiation between LACE Clinic and Explained courses.
• The LACE successfully delivered a session about work item estimation to the QMVIG support team. This helped the QMVIG support team to start estimating all work items.

Objectives:

• Courses:
  • SAFe Lean Portfolio Management
• Program engagement
  
  • iQies - on hold: Need leadership clarification
  • QMVIG support team did a detailed tool analysis for QMVIG front office. This tool analysis was delivered to the QMVIG Front Office leadership for review.
• Dashboards
  
  • Features started report accurately 
• Course attendance Dashboard
  • Display graphs
  • Data collection for attendee types
• CoE 101

Value Delivered:

• LACE has now the ability to collect data for attendee types during registration through Eventbrite by category: CCSQ- ISG Employee, CCSQ- Other Employees, CCSQ-ISG Contractor, CMS Employee, CMS Contractor, Other.
• Successful delivery of Scaled Agile Framework Certified SAFe Lean Portfolio Management course with 4 attendees out of 5 expected.
• Study Hall will be set up for attendees of the LPM SAFe course to have the ability to get coaching assistance in preparation for the upcoming certified course exam.
• LACE held Feature Refinement Session with Brandy to continue to prioritize and decompose the scope of work and any scope changes in priorities.
• Successful created Federal Attendance Chart to capture attendees category for LACE Community of Practice and uploaded in confluence https://qnetconfluence.cms.gov/display/LACE/LACE+CoP+Federal+Attendance+Chart
• Completed data validation for Features In Status by PI and Program Backlog Health reports in the Program Dashboard.
• The QMVIG support team delivered a tool analysis document on all the tools QMVIG currently uses compared to CCSQ approved tools with recommendations.

Backlog

2 pts:

LACE-3187 ( LACE-3187 - Getting issue details... STATUS )

Added

1pt 

LACE-3185 LACE-3185 - Getting issue details... STATUS

Objectives:

• Create course attendance date
  • SAFe Course Attendance Graph to compare Fed vs. total registrants
• LACE/HCD CoE
• Program support IQIES
• Pain Points Feedback survey
• CoE 101 Training
• Program Dashboards
• SAFe LPM Mural board 

Value Delivered:

• A survey to surface course delivery pain points was created and tested.
• Completed draft course outline for COE101 course
• Updated Program Dashboard reports to use new work item type groupings so the pie charts have a common representation of the work.
• Successfully added Federal Registrants for SAFe courses with a specific time frame to align to existing Course LACE Trainings, Clinics, CoP's for LACE. This is now available to compare federal vs. total registrant for scaled agile framework certified courses.
• Refinement session with Brandy to continue to refine and identify any new scope of work in preparation for upcoming joint LACE/HCD PI Planning Increment # 2
• The LACE team created a Mural board with all the interactive exercises from the SAFe LPM workbook with instructions. This board will be used for all future SAFe LPM courses.
• LACE offered a study hall session for Lean Portfolio Management on 06/02 session to help participants prepare for exam certification
• LACE conducted Iteration Review to stakeholders Brandy and Ola for visibility and feedback loop cycle
• The team delivered a session to iQIES HCD Lead about an overview presentation on the Center of Excellence and mapped a layout of the engagement process and current steps for iQIES HCD team.

Backlog

3 pts:

LACE-3175( LACE-3175 - Getting issue details... STATUS )

Objectives:

• Program support IQIES
• Program Support QMVIG
• Courses, Clinics & Study hall
  
  • LACE Explained: OKR's
  • LACE Explained: Program Backlog Management
  • User Story Clinic
  • LPM Study Hall
• Dashboards
  • Master date dashboard control - research how to implement
  • New confluence page to add additional detail about the reports and how to interpret them
• Course Attendance 

Value Delivered:

• OKR and Program Backlog Management courses: Attendees were enlightened during both LACE Explained Courses and engaged in great discussions on topics. These courses allowed the safety and freedom to explore topics
• Guidance and advice were provided while working with real examples from attendees in the User Story Clinic 
• LPM Study Hall: Participants in the SAFe LPM course attended the session to ask questions in preparation to take the exam
• Program Dashboards
  • Continued research into master dashboard date controls to enable the charts to be controlled by PI (where applicable)
  • Designed Confluence page with detailed report descriptions so users can learn more about the reports
• LACE focus on improving Course Dashboard Attendance in this iteration by creating filter components that help sort courses, clinics and CoP's within a specific timeframe and comparing registrants vs. actual attendance based on certification and completed courses, clinics participation, and engagement by mapping data collection
• LACE created an initial draft contribution on the topic "Adult Learning" in preparation for HCD Newsletter committed during our joint PI LACE/HCD for collaboration
• LACE assisted EQRS RTE with WSJF Clinic data participants collaboration in preparation for next week's 6/16 WSJF Clinic with currently 20 registrants from EQRS Program and other contractors
• IQies: After a short break, the discovery phase continues. Three main activities were performed: (1) The CoE was officially introduced to the HCD team; Now the HCD team knows better what is the CoE, what services they generally offer to the CCSQ community, why they will be working with them, and how the partnership with them will work. (2) Observations of two ceremonies - DSU and Backlog Refinement - were performed and documented. (3) A 1-1 interview with the Huge account manager was performed and documented   
• The LACE team created a draft coaching engagement plan for QMVIG program capturing information about the program, Key members, background, discovery, recommended approach, risks, and timeframe 

Added:

2 pts:

LACE- 3163 ( LACE-3163 - Getting issue details... STATUS )

1 Pt:

LACE- 3141 ( LACE-3141 - Getting issue details... STATUS )

Backlog:

5pts:

LACE-3207 ( LACE-3207 - Getting issue details... STATUS )

Removed:

1 Pt:

LACE- 2993 ( LACE-2993 - Getting issue details... STATUS )

Objectives:

•  4 Courses: 
  • LACE Explained: CCSQ Team Dashboards
  • LACE Explained: Technical Agility
  • Dashboard Metrics Clinic
  • WSJF Clinic
• Dashboards
• CoE Course Development
• Program support QMVIG
• Program support IQIES
• Housekeeping slide for courses moving forward

Value Delivered:

• Housekeeping slide has been created to bring synchronicity between coaches when presenting to participants to clearly set an expectation.
• LACE shared projected 3rd Quarter courses roll out with Brandy and confirm (Explained Courses, Clinics, LACE Community of Practice, joint LACE/HCD course and Scaled Agile Framework certification courses) 
• LACE held Weighted Shortest Job First (WSJF) clinic with 21 participants from EQRS Program, leaders, stakeholders, and ADO's WSJF Clinic : June 16th ,2022
• LACE Explained: CCSQ Team Dashboards course was successfully completed.
• LACE Explained: Technical Agility course was successfully completed.
• Program Dashboard was improved by enhancing the Exploration Enabler grouping to include additional labels.
• Joint CoE course development continued by finishing the HCD, Lean, and Agile Overview section.
• Program Support HCD - iQies:
  • Observation of ceremonies and documentation (DSU, Planning, Review, and Refinement)
  • Product Owner was interviewed and documented on Confluence
• The LACE met with HCD to find areas where both teams can collaborate together to help QMVIG. This was captured on our QMVIG coaching engagement plan as recommended approach

Added:

4 pts 

LACE 3227 ( LACE-3227 - Getting issue details... STATUS )

LACE-3226 ( LACE-3226 - Getting issue details... STATUS )

Backlog:

2 pts

LACE 3227 ( LACE-3227 - Getting issue details... STATUS )

Objectives:

•  3 Courses: 
  • LACE Explained: Roadmaps
  • OKR Clinic
  • User Story Clinic
• Program Support 
  • QMVIG
  • IQies
• Dashboard Metrics

Value Delivered

• Program Support HCD - iQies:
  
  • Observation of ceremonies and documentation (DSU, PI Planning Prep, and Refinement)
  • The Program RTE and CMS HCD Lead were interviewed and documented on Confluence
• LACE Facilitated Objectives and Key Results clinic on 6/23 and provided a walkthrough of some Key results for ISG Divisions and explained how these OKR's are applied within Organizations and Programs: OKR Clinic: June 23rd, 2022
• Team held User Story clinic on 23 Jun 2022 for the CCSQ community and shared information on how to properly utilize best practices to write user stories.
• The LACE team did a final update on the coaching engagement plan for QMVIG program capturing more information about the program, Key members, background, discovery, recommended approach, risks, and timeframe
• Focused on Feature Refinement session with Brandy on 6/22 as we are approaching our joint PI Planning and Placeholder and Zoom invite link as well as break-out rooms created.
• LACE collaborated with Brian from HCD in regards of "Power Point Template" intro for CoE's moving forward for join collaboration
• Improve look and feel/usability of pilot program dashboard 
• Submitted education article for HCD Newsletter

Added 2 pts:  LACE 3213( LACE-3213 - Getting issue details... STATUS )