LACE (CCSQ Lean Agile Center of Excellence)

Page tree

Versions Compared

Old Version 29

changes.mady.by.user Marc Santini

Saved on

compared with

New Version 30

changes.mady.by.user Marc Santini

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.



tandard deviations of less than 10 percent can result from the data being one day stale, or in edge cases where an issue was moved in and out of a status multiple times. Greater deviations require investigation

Standard Deviation
Green +/- 5%Accurate
Orange+/- 5% to +/- 10%Acceptable
RedAbove +/- 10%Requires Triage*
Standard deviations of less than 10 percent can result from the data being one day stale, or in edge cases where an issue was moved in and out of a status multiple times. Greater deviations require investigation
 *triage required if the delta between Jira Validation Query and the report both numerically and proportionally large. Examples of non-triaged counts with greater than than +/-10%
  1. 6/5 has a delta of one issue, but is greater than the acceptable standard deviation


Tabs Container
titleValidation Queries By Report
directionvertical


Tabs Page
titleFeatures in Status on PI Start Dates

Validation Date: 8/15/2022

Validator: Marc Santini

Validation Notes:

  1. Issues with status changes to implementing were not being caught because the earlier status was being returned by the validation query. Changing the time from (YYYY-M(M)-D(D)) to ('YYYY-M(M)-D(D) HH:MM'), with "HH:MM" being "23:59" catches the status at 11:59 PM. This brought all counts well under the "Accurate" STD category
iQiesPI3PI4 PI5PI6PI7
Funnel160/156179/177175/173199/198299/300
Feature Definition1/13/35/56/64/4
Feature Refinement3/35/54/43/31/1
Feature Deep Dive8/811/114/418/1810/10
Story Definition84/8592/9254/5468/6852/52
PI Ready68/6859/59 84/8480/8082/82
Implementing57/5668/6869/6950/5050/50



EQRSPI 14PI 15PI 16PI 17PI 18
Funnel21/21125/125107/10784/84100/100
Analyzing (SAFe)0/00/03/33/33/3
Refining16/1613/1314/13*8/86/5*
Ready For PI0/09/917/1721/2225/25
Implementing5/512/1225/2529/2936/39


Validation Date : 8/11/2022

Validator:  Marc Santini


iQiesPI3PI4 PI5PI6PI7
Funnel160/156179/177175/173199/198299/302
Feature Definition1/13/35/56/64/4
Feature Refinement3/35/54/43/31/1
Feature Deep Dive8/811/114/418/1817/10
Story Definition84/8592/9254/7068/7352/55*
PI Ready68/6859/6584/9980/8282/88
Implementing57/5968/7269/7550/5050/50


EQRSPI 14PI 15PI 16PI 17PI 18
Funnel21/21125/125107/10784/84100/102
Analyzing (SAFe)0/00/03/33/33/3
Refining16/1613/1314/13*8/86/5*
Ready For PI0/09/917/1721/2225/27*
Implementing5/512/1225/2529/3336/38*



Tabs Page
titleCurrent Program Backlog Composition

Validation Date: 

Validators: Randolph Anagho, Marc Santini


Validation notes: 

TypesiQIESEQRS
Infrastructure Enabler1/10/0
Architectural Enabler1/10/0
Exploration Enabler1/019/22
Compliance Enabler2/20/0
Enabler91/908/8
Features499/426130/62


Validation Date: 08/12/2022

Validator: Randolph Anagho

TypesiQIESEQRS
Infrastructure Enabler1/10/0
Architectural Enabler1/10/0
Exploration Enabler1/019/19
Compliance Enabler2/20/0
Enabler91/908/8
Features499/426130/62



Tabs Page
titleFeatures Started and Closed per PI

Report Name: Features Started and Closed per PI

Legend: EazyBI count/JQL count


Validation Date : 8/17/2022

Validator: Marc Santini

Because The eazybi measure was using dateBetween with the actual PI start and end dates, it was not picking up items changed to the Implementing or Done statuses on the defined PI Start and End dates, because it was only looking for items in between those dates, not including them I updated the query to check After the the last minute of the day before the PI, and Before the first minute of the day following the PI. I updated the Eazybi measure and then changed the Jira query to use the BEFORE and AFTER operators rather than the DURING and used matching timestamps in both the measure and the query. I suspect that the DURING operator in Jira is not as high resolution as EazyBi's Datebetween function. 


I will create two stories to investigate the anomalous counts for iQIES PI 6 implementing, and EQRS PI 18 Done

Updated DateBetween function:  [Time].[Day].DateMembersBetween('Jan 18 2022 23:59', 'Apr 13 2022 00:01')
Updated Query:  Project = EQRS AND issuetype = 10000 AND Status changed to Done AFTER  ("2022-01-19 00:01")  AND Status changed to Done BEFORE  ("2022-04-13 00:01")
iQiesPI3PI4 PI5PI6PI7
Implementing104/104129/132123/122123/11383/85
Done196/196132/13597/97115/11516/17*


EQRSPI 14PI 15PI 16PI 17PI 18PI 19
Implementing16/1630/3036/3632/3221/2116/16
Done10/1025/2532/3233/3233/2919/19


Validation Date : 8/16/2022

Validator: Seth Evans

iQiesPI3PI4 PI5PI6PI7
Implementing104/104129/132105/122199/19873/83
Done196/196132/13593/96106/11316/16


Project = "iQIES Program Board "  AND issuetype = 10000 AND Status changed to Done during (2022-01-19, 2022-04-19)  ORDER BY key DESC


EQRSPI 14PI 15PI 16PI 17PI 18PI 19
Implementing16/1630/3036/3627/2918/2116/16
Done10/1025/2532/3227/3127/2919/19


Project = " End-Stage Renal Disease Quality Reporting System" AND issuetype = 10000 AND Status changed to Done during (2021-04-04, 2021-07-20) --Update status and dates of PI



Tabs Page
titleWork Delivered by Category per PI

Report Name: Work Delivered by Category per PI

Dashboard: CCSQ Program Dashboard



Validation Date: 9/7/22

Validator: Marc Santini

Validation Notes: iQIES numbers improved, all numbers within the "Accurate" standard deviation range!

EQRS numbers largely acceptable with a few outliers. I am curious as to why the average StDev jumped from 1% or less to ~5% starting in PI 17




Validation Queries


Validation Date: 8/31/2022

Validator: Seth Evans

iQIES

PI3PI4PI5PI6PI7

ChartJQL% DiffChartJQL% DiffChartJQL% DiffChartJQL% DiffChartJQL% Diff
Total190118801.10%18761894-0.96%18201840-1.10%18301832-0.11%810830-2.47%
Story126212451.35%12941315-1.62%14311450-1.33%13201325-0.38%621638-2.74%
Spike2222220.00%2122091.42%155158-1.94%1371342.19%6365-3.17%
Bug4174130.96%3703700.00%2342320.85%3763701.60%126127-0.79%
JQL Query:
PI3Project in ("iQIES Enigma", "iQIES HCD", "iQIES Frontiers", "iQIES Bohemian Rhapsody (DevOps)", "iQIES Dune (CLIA)", "iQIES FACT (Data and Backend)", "iQIES Outlander", "iQIES PA Reports", "iQIES QMIR Data", "iQIES SAS Viya / Dashboards", "iQIES S&C Reports", "IQIES SNOOPY", "iQIES Stargate", "iQIES Immeasurable", "iQIES The Misfits", "iQIES The Shop", "IQIES Woodstock")  AND issuetype in (Story, Spike, Bug) AND resolutiondate >= 2021-07-07 AND resolutionDate <= 2021-10-12
PI4Project in ("iQIES Enigma", "iQIES HCD", "iQIES Frontiers", "iQIES Bohemian Rhapsody (DevOps)", "iQIES Dune (CLIA)", "iQIES FACT (Data and Backend)", "iQIES Outlander", "iQIES PA Reports", "iQIES QMIR Data", "iQIES SAS Viya / Dashboards", "iQIES S&C Reports", "IQIES SNOOPY", "iQIES Stargate", "iQIES Immeasurable", "iQIES The Misfits", "iQIES The Shop", "IQIES Woodstock")  AND issuetype in (Story, Spike, Bug) AND resolutiondate >= 2021-10-13 AND resolutionDate <= 2022-01-18
PI5Project in ("iQIES Enigma", "iQIES HCD", "iQIES Frontiers", "iQIES Bohemian Rhapsody (DevOps)", "iQIES Dune (CLIA)", "iQIES FACT (Data and Backend)", "iQIES Outlander", "iQIES PA Reports", "iQIES QMIR Data", "iQIES SAS Viya / Dashboards", "iQIES S&C Reports", "IQIES SNOOPY", "iQIES Stargate", "iQIES Immeasurable", "iQIES The Misfits", "iQIES The Shop", "IQIES Woodstock")  AND issuetype in (Story, Spike, Bug) AND resolutiondate >= 2022-01-19 AND resolutionDate <= 2022-04-19
PI6Project in ("iQIES Enigma", "iQIES HCD", "iQIES Frontiers", "iQIES Bohemian Rhapsody (DevOps)", "iQIES Dune (CLIA)", "iQIES FACT (Data and Backend)", "iQIES Outlander", "iQIES PA Reports", "iQIES QMIR Data", "iQIES SAS Viya / Dashboards", "iQIES S&C Reports", "IQIES SNOOPY", "iQIES Stargate", "iQIES Immeasurable", "iQIES The Misfits", "iQIES The Shop", "IQIES Woodstock") AND issuetype in (Story, Spike, Bug) AND resolutiondate >= 2022-04-20 AND resolutionDate <= 2022-07-19 ORDER BY status DESC
PI7Project in ("iQIES Enigma", "iQIES HCD", "iQIES Frontiers", "iQIES Bohemian Rhapsody (DevOps)", "iQIES Dune (CLIA)", "iQIES FACT (Data and Backend)", "iQIES Outlander", "iQIES PA Reports", "iQIES QMIR Data", "iQIES SAS Viya / Dashboards", "iQIES S&C Reports", "IQIES SNOOPY", "iQIES Stargate", "iQIES Immeasurable", "iQIES The Misfits", "iQIES The Shop", "IQIES Woodstock") AND issuetype in (Story, Spike, Bug) AND resolutiondate >= 2022-07-20 AND resolutionDate <= 2022-10-11 ORDER BY status DESC



Report Name: Work Delivered by Category per PI

Dashboard : Enterprise Program Dashboard

Legend: EazyBI count/JQL count


iQIES

PI3PI4PI5PI6PI7

ChartJQL% DiffChartJQL% DiffChartJQL% DiffChartJQL% DiffChartJQL% Diff
Total190118801.10%18761894-0.96%18201840-1.10%18291832-0.11%810830-2.47%
Story126212451.35%12941315-1.62%14311450-1.33%13201325-0.38%621638-2.74%
Spike2222220.00%2122091.42%155158-1.94%1341340%6365-3.17%
Bug4174130.96%3703700.00%2342320.85%3753701.60%126127-0.79%


iQies numbers improved or stayed the same in all cases!



EQRSPI 15PI 16PI 17PI 18PI 19

Report
JQL 
StDev
Report
JQL
StDev
Report
JQL
StDev
Report
JQL
StDev
Report
JQL
StDev
Total
700699.02%731733.07%8588174.8%7817494.6%4785055.3%
Story
5675670%523527.08%6936614.7%6536293.8%3653865.4%
Bug
13113008%1931911.1%1541493%1171096.8%1071135.3%
Spike
220%880%6350%770%220%
Enabler000%770%440%6433.3%440%


I call this acceptable, although I am curious why the StDev jumped up to around 5% starting in PI 17. The high diffs are small issue counts and can be researched


PI5

Project in (EQRS, EQRSCI, EQRSD, EQRSARCH, EQRSEDSM, EQRSONBRD, EQRSMD, EQRSMS, EQRSPRTL) AND issuetype in(Story, Spike, Bug, Enabler) AND resolutiondate >= 2021-07-20 AND resolutionDate <= 2021-10-12

PI6Project in (EQRS, EQRSCI, EQRSD, EQRSARCH, EQRSEDSM, EQRSONBRD, EQRSMD, EQRSMS, EQRSPRTL) AND issuetype in(Story, Bug, Spike, Enabler) AND resolutiondate >= 2021-10-13 AND resolutionDate <= 2022-01-18
PI7Project in (EQRS, EQRSCI, EQRSD, EQRSARCH, EQRSEDSM, EQRSONBRD, EQRSMD, EQRSMS, EQRSPRTL) AND issuetype in(Story, Spike, Bug, Enabler) AND resolutiondate >= 2022-01-19 AND resolutionDate <= 2022-04-12
PI8Project in (EQRS, EQRSCI, EQRSD, EQRSARCH, EQRSEDSM, EQRSONBRD, EQRSMD, EQRSMS, EQRSPRTL) AND issuetype in(Story, Spike, Bug, Enabler) AND resolutiondate >= 2022-04-13 AND resolutionDate<= 2022-07-05
PI9Project in (EQRS, EQRSCI, EQRSD, EQRSARCH, EQRSEDSM, EQRSONBRD, EQRSMD, EQRSMS, EQRSPRTL) AND issuetype in(Story, Spike, Bug, Enabler) AND resolutiondate >= 2022-07-06 AND resolutionDate<= 2022-09-27



Tabs Page
titleAll ART Teams' Backlog Composition at PI Start

Report Name: IQies Project data in "All ART Teams"

Dashboard: iQIES All ART Teams dashboard

Legend: JQL/Easy BI

Validation Date: 9/08/2022

Validator: Randolph Anagho


JQL/Dashboard 

IQIESPI 3PI 4PI 5PI 7
Stories951/1067956/9531124/1115179/155
Bugs290/347291/291180/182102/92
Enablers0/00/00/00/0
Spikes185/213157/157127/12626/24


Report Link: PI 4 All ART Teams IQIES Test Report - Issues - LACE Metrics Development - eazyBI (cms.gov)


Jira Query



PI 3Project in ("iQIES Enigma", "iQIES HCD", "iQIES Frontiers", "iQIES Bohemian Rhapsody (DevOps)", "iQIES Dune (CLIA)", "iQIES FACT (Data and Backend)", "iQIES Outlander", "iQIES PA Reports", "iQIES QMIR Data", "iQIES SAS Viya / Dashboards", "iQIES S&C Reports", "IQIES SNOOPY", "iQIES Stargate", "iQIES Immeasurable", "iQIES The Misfits", "iQIES The Shop", "IQIES Woodstock") AND issuetype in (Spike, Story, Enabler, Bug) AND created >= 2021-7-20 AND created <= 2021-10-12 AND status was in (Done, Closed, Done, "Production Ready")
PI 4Project in ("iQIES Enigma", "iQIES HCD", "iQIES Frontiers", "iQIES Bohemian Rhapsody (DevOps)", "iQIES Dune (CLIA)", "iQIES FACT (Data and Backend)", "iQIES Outlander", "iQIES PA Reports", "iQIES QMIR Data", "iQIES SAS Viya / Dashboards", "iQIES S&C Reports", "IQIES SNOOPY", "iQIES Stargate", "iQIES Immeasurable", "iQIES The Misfits", "iQIES The Shop", "IQIES Woodstock") AND issuetype in (Spike, Story, Enabler, Bug) AND created >= 2021-10-13 AND created <= 2022-1-14 AND status was in (Done, Closed, Done, "Production Ready")
PI 5Project in ("iQIES Enigma", "iQIES HCD", "iQIES Frontiers", "iQIES Bohemian Rhapsody (DevOps)", "iQIES Dune (CLIA)", "iQIES FACT (Data and Backend)", "iQIES Outlander", "iQIES PA Reports", "iQIES QMIR Data", "iQIES SAS Viya / Dashboards", "iQIES S&C Reports", "IQIES SNOOPY", "iQIES Stargate", "iQIES Immeasurable", "iQIES The Misfits", "iQIES The Shop", "IQIES Woodstock") AND issuetype in (Spike, Story, Enabler, Bug) AND created >= 2022-1-19 AND created <= 2022-4-19 AND status was in (Done, Closed, Done, "Production Ready")
PI 7Project in ("iQIES Enigma", "iQIES HCD", "iQIES Frontiers", "iQIES Bohemian Rhapsody (DevOps)", "iQIES Dune (CLIA)", "iQIES FACT (Data and Backend)", "iQIES Outlander", "iQIES PA Reports", "iQIES QMIR Data", "iQIES SAS Viya / Dashboards", "iQIES S&C Reports", "IQIES SNOOPY", "iQIES Stargate", "iQIES Immeasurable", "iQIES The Misfits", "iQIES The Shop", "IQIES Woodstock") AND issuetype in (Story, Spike, Enabler, Bug) AND created >= 2022-7-20 AND created <= 2022-10-11 AND status was in (Done, Closed, Done, "Production Ready")



...