Page History

Report Name: Current Program Backlog Composition

Validation Date: 10/6/2022

Validator: Seth Evans/Randolph Anagho

Table Result Key: Dashboard/JQL

NOTE: Updated report filter to include the following statuses to meet the report description (The number of ready and in progress features and enablers in the program backlog today.) Included statuses: "Ready for PI", "PI Ready", Implementing, "In Progress" 

HQR JQL: Project in ("HQR Program Project", "HQR UX-Men") AND issuetype in (10000, Enabler) AND Status in (Implementing, "PI Ready") ORDER BY issuetype ASC

EQRS JQL:

All features and enablers: Project = " End-Stage Renal Disease Quality Reporting System" AND issuetype in (10000, Enabler) AND Status in ("Ready for PI", "PI Ready", Implementing, "In Progress") ORDER BY status DESC

All Enablers: Project = " End-Stage Renal Disease Quality Reporting System" AND issuetype in (10000, Enabler) AND Status in ("Ready for PI", "PI Ready", Implementing, "In Progress") AND labels in (Enabler, Exploration_Enabler, Architectural_Enabler, Infrastructure_Enabler, Compliance_Enabler) ORDER  BY status DESC

IQIES JQL:

All features and enablers: Project in ("iQIES Program Board ", "iQIES Enigma", "iQIES HCD", "iQIES Frontiers", "iQIES Bohemian Rhapsody (DevOps)", "iQIES Dune (CLIA)", "iQIES FACT (Data and Backend)", "iQIES Outlander", "iQIES PA Reports", "iQIES QMIR Data", "iQIES SAS Viya / Dashboards", "iQIES S&C Reports", "IQIES SNOOPY", "iQIES Stargate", "iQIES Immeasurable", "iQIES The Misfits", "iQIES The Shop", "IQIES Woodstock")  AND issuetype in (10000, Enabler) AND Status in ("Ready for PI", "PI Ready", Implementing, "In Progress") ORDER BY status DESC

All Enablers: Project = "iQIES Program Board "  AND issuetype in (10000, Enabler) AND Status in ("Ready for PI", "PI Ready", Implementing, "In Progress") AND labels in (Enabler, Exploration_Enabler, Architectural_Enabler, Infrastructure_Enabler, Compliance_Enabler) ORDER  BY status DESC

Report: Current Program Backlog Health

Validation Date: 10/06/2022

Validator: Marc Santini

Comments: Overall good results. Egregious deviations are seen on low numbers as usual. Did data refresh and ruled out stale data. Investigation into iQIES errata needed

Report: Features in Status on PI start Date

Validation Date: 10/12/2022

Validator: Seth Evans

Validation Notes: I also revalidated iQies and EQRS. The previous results listed below are still valid. (Numbers for most recent PIs did change because they are active)

Legend: EazyBI count/JQL count

JQL: project  in ("HQR UX-Men", "HQR Program Project")  AND issuetype = 10000 AND Status was "PI Ready"  on ("2022-7-26 23:59")  - *Change status and date as applicable

Report: Features in Status on PI start Date

Validation Date: 8/15/2022

Validator: Marc Santini

Validation Notes:

1. Issues with status changes to implementing were not being caught because the earlier status was being returned by the validation query. Changing the time from (YYYY-M(M)-D(D)) to ('YYYY-M(M)-D(D) HH:MM'), with "HH:MM" being "23:59" catches the status at 11:59 PM. This brought all counts well under the "Accurate" STD category

Validation Date : 8/11/2022

Validator:  Marc Santini

Report Name: Features Started and Closed per PI

Legend: EazyBI count/JQL count

Validation Date: 10/13/2022

Validator: Seth Evans

JQLPI19: Project in ("HQR Program Project", "HQR UX-Men") AND issuetype = 10000 AND Status changed to Implementing during (20222021-0511-0416, 2022-02-07) (change status to Done or Implementing)

PI20: Project in ("HQR Program Project", "HQR UX-Men") AND issuetype = 10000 AND Status changed to Implementing during (2022-02-08, 2022-05-02) (change status to Done or Implementing)

PI21: Project in ("HQR Program Project", "HQR UX-Men") AND issuetype = 10000 AND Status changed to Implementing during (2022-05-03, 2022-07-25) - Add one day to the PI start date in the custom PI time measure.) (change status to Done or Implementing)

PI22: Project in ("HQR Program Project", "HQR UX-Men") AND issuetype = 10000 AND Status changed to Implementing during (2022-07-26, 2022-10-17) (change status to Done or Implementing)

PI4: Project = "iQIES Program Board " AND issuetype = 10000 AND status changed to Done during (2021-10-1413, 2022-01-18) (change status to Done or Implementing)

PI5: Project = "iQIES Program Board " AND issuetype = 10000 AND status changed to Done during (2022-01-2019, 2022-04-19) (change status to Done or Implementing)

PI6: Project = "iQIES Program Board " AND issuetype = 10000 AND status changed to Done during (2022-04-2120, 2022-07-19) (change status to Done or Implementing)

PI7: Project = "iQIES Program Board " AND issuetype = 10000 AND status changed to Done during (2022-07-2120, 2022-10-11) (change status to Done or Implementing)

PI16: Project = " End-Stage Renal Disease Quality Reporting System"  AND AND issuetype = 10000 AND status changed to Done during (2021-10-1413, 2022-01-18) (change status to Done or Implementing)

PI17: Project = " End-Stage Renal Disease Quality Reporting System" AND issuetype = 10000 AND status changed to Done during (2022-01-2019, 2022-04-12) (change status to Done or Implementing)

PI18: Project = " End-Stage Renal Disease Quality Reporting System"  AND AND issuetype = 10000 AND status changed to Done during (2022-04-1413, 2022-07-05) (change status to Done or Implementing)

PI19: Project = " End-Stage Renal Disease Quality Reporting System" AND issuetype = 10000 AND status changed to Done during (2022-07-06, 2022-09-27) (change status to Done or Implementing)

***I added a bug

Validation Date : 8/17/2022

Validator: Marc Santini

Because The eazybi measure was using dateBetween with the actual PI start and end dates, it was not picking up items changed to the Implementing or Done statuses on the defined PI Start and End dates, because it was only looking for items in between those dates, not including them I updated the query to check After the the last minute of the day before the PI, and Before the first minute of the day following the PI. I updated the Eazybi measure and then changed the Jira query to use the BEFORE and AFTER operators rather than the DURING and used matching timestamps in both the measure and the query. I suspect that the DURING operator in Jira is not as high resolution as EazyBi's Datebetween function. 

I will create two stories to investigate the anomalous counts for iQIES PI 6 implementing, and EQRS PI 18 Done

Updated DateBetween function:  [Time].[Day].DateMembersBetween('Jan 18 2022 23:59', 'Apr 13 2022 00:01')

Updated Query:  Project = EQRS AND issuetype = 10000 AND Status changed to Done AFTER  ("2022-01-19 00:01")  AND Status changed to Done BEFORE  ("2022-04-13 00:01")

Validation Date : 8/16/2022

Validator: Seth Evans

Project = "iQIES Program Board "  AND issuetype = 10000 AND Status changed to Done during (2022-01-19, 2022-04-19)  ORDER BY key DESC

Project = " End-Stage Renal Disease Quality Reporting System" AND issuetype = 10000 AND Status changed to Done during (2021-04-04, 2021-07-20) --Update status and dates of PI

Report Name: Work Delivered by Category per PI

Dashboard: CCSQ Program Dashboard

Validation Date: 9/7/22

Validator: Marc Santini

Validation Notes: iQIES numbers improved, all numbers within the "Accurate" standard deviation range!

EQRS numbers largely acceptable with a few outliers. I am curious as to why the average StDev jumped from 1% or less to ~5% starting in PI 17

Validation Queries

Validation Date: 8/31/2022

Validator: Seth Evans

JQL Query:

Report Name: Work Delivered by Category per PI

Dashboard : Enterprise Program Dashboard

Legend: EazyBI count/JQL count

iQies numbers improved or stayed the same in all cases!

I call this acceptable, although I am curious why the StDev jumped up to around 5% starting in PI 17. The high diffs are small issue counts and can be researched

Report Name: IQies Project data in "All ART Teams"

Dashboard: iQIES All ART Teams dashboard

Legend: JQL/Easy BI

Validation Date: 9/08/2022

Validator: Randolph Anagho

JQL/Dashboard 

Report Link: PI 4 All ART Teams IQIES Test Report - Issues - LACE Metrics Development - eazyBI (cms.gov)

Jira Query

I call this acceptable, although I am curious why the StDev jumped up to around 5% starting in PI 17. The high diffs are small issue counts and can be researched