EQRS Communications Hub

Data Submitted 09-13-2021

LDOData Type

 

 

 

 

 

 

 

DAVITA

Attestation0000000
Clinical0000000
Demographic8897000000
Reporting measures0000000
DCI

Attestation0000000
Clinical0070002
Demographic063500000
Reporting measures0000000
FKC

Attestation0000000
Clinical0000000
Demographic3490736122080000
Reporting measures0000000
NRAA

Attestation0000000
Clinical1780953836360000
Demographic1295349139240000
Reporting measures7128295510000

Data Submitted 09-06-2021

LDOData Type

 

 

 

 

 

 

 

DAVITA

Attestation0000000
Clinical0000000
Demographic010,43437,7540000
Reporting measures0000000
DCI

Attestation000482000
Clinical00130002
Demographic02880048200
Reporting measures0000000
FKC

Attestation0000000
Clinical0000000
Demographic2727602645362439371360
Reporting measures0000000
NRAA

Attestation0000000
Clinical102244420991100256912983722
Demographic912197537389318075621485
Reporting measures22932232684424319

Data Submissions 08-30-2021

LDOData Type

 

 

 

 

 

 

 

DAVITA

Attestation0000000
Clinical0000000
Demographic10,654136,0710000
Reporting measures00000000
DCI

Attestation0000000
Clinical01127298000
Demographic03920048100
Reporting measures0000000
FKC

Attestation0000000
Clinical0013105051000
Demographic2896786552969693473144392922
Reporting measures0000000
NRAA

Attestation0000000
Clinical137820211627220411093277
Demographic731184584510664368617199
Reporting measures12944503533291961

Data Submissions 08-23-2021

LDOData Type

 

 

 

 

 

 

 

DAVITA

Attestation0000000
Clinical0000000
Demographic10,140035,3920000
Reporting measures0000000
DCI

Attestation0000000
Clinical00129012600
Demographic03660042300
Reporting measures0000000
FKC

Attestation0000000
Clinical0000000
Demographic11,114755885956015544345093053
Reporting measures0000000
NRAA

Attestation0000000
Clinical143830902175152355655759
Demographic4452519173624461154941
Reporting measures1201311945302301320

Data Submissions 08-16-2021

LDOData Type

 

 

 

 

 

 

 

DAVITA

Attestation0000000
Clinical0000000
Demographic388635,57736,7660000
Reporting measures0000000
DCI

Attestation0000000
Clinical01453281000
Demographic000045200
Reporting measures0000000
FKC

Attestation0000000
Clinical0000000
Demographic2664216614,170424500
Reporting measures0000000
NRAA

Attestation0000000
Clinical368670612,07917,9431406643136
Demographic722645412,1471652569410197
Reporting measures88487522767388308010015

Data Submissions 08-09-2021

LDOData Type

 

 

 

 

 

 

 

DAVITA

Attestation0000000
Clinical0000000
Demographic0024000674600
Reporting measures00147,4470000
DCI

Attestation0000000
Clinical0000000
Demographic03250042600
Reporting measures0000000
FKC

Attestation0000000
Clinical0000000
Demographic26948404458910.843506245572863
Reporting measures0000000
NRAA

Attestation0000000
Clinical279922,25411,23316,18414,72014,5061738
Demographic14,381353411,947296610,888223108
Reporting measures4282635388572062820

Data Submissions 08-02-2021

LDOData Type

 

 

 

 

 

 

 

Total
DAVITA

Attestation00000000
Clinical085264526060005,336
Demographic0901336813000045,826
Reporting measures00189,3970000189,397
DCI

Attestation00000000
Clinical000015050155
Demographic03490038800737
Reporting measures00000000
FKC

Attestation00000000
Clinical0638501070006,492
Demographic25,80377299397501445144201287159,529
Reporting measures00000000
NRAA

Attestation00000000
Clinical18,9450036,856312517543659,537
Demographic10360056,2192065763317,17884,131
Reporting measures278006162368506,813

Data Submissions 07-26-2021

LDOData Type

 

 

 

 

 

 

 

Total
DAVITA

Attestation0000000
Clinical0000341500
Demographic0048,3837892000
Reporting measures0000189,39700
DCI

Attestation00034000
Clinical0017515331900
Demographic03710042100
Reporting measures0000000
FKC

Attestation0000000
Clinical0000000
Demographic270073685071976833145293192
Reporting measures0003762000
NRAA

Attestation0000000
Clinical13,23256364100634133,9714501877
Demographic23222241158312,7412309878856
Reporting measures10,2483893121822147783690

27 Jul 2021 Issues

Batch IDFile TypeFailure PointError Message

b5aaeda0-1ce3-4be2-8e1b-041ab52a73ea

NRAA clincialLANDING2021-07-27 15:33:42 7eb09e82-ff75-4377-8f03-53d5e4c8cf33 ERROR LandingHandler:275 - Exception occured in Landing Handler null

Data Submissions 07-19-2021

LDOData Type

 

 

 

 

 

 

 

Total
DAVITA

Attestation00000000
Clinical212,026025050123300215,764
Demographic8631037,048100045,680
Reporting measures00000000
DCI

Attestation00000000
Clinical0342141800001,760
Demographic03460039800744
Reporting measures000022,3380022,338
FKC

Attestation028900000289
Clinical00000000
Demographic268872834830957450434324300636,748
Reporting measures043,6360000043,636
NRAA

Attestation00000000
Clinical700461315306244618,85740361329957,079
Demographic190436093028367491021245871031,272
Reporting measures122821416354325811168600022,697

19 Jul 2021 Issues

All files were marked processed on July 19. 

20 Jul 2021 Issues

Batch IDFile TypeFailure PointError Message

7a8bd2fc-6284-47c3-89a7-d3cb1839c2ed

NRAA reportingMeasureBATCH_FEEDBACK2021-07-20 12:21:01.708 ERROR [batchfeedback_data,trace_id=d88950ab7582c421,span_id=04aed0b3b15e6836,span_name=,username=] [pid=7] [EDSM-worker-thread-p1-10] gov.cms.eqrs.edsm.batchfeedback.BatchFeedbackManager :  Exception occured in processTask:  7a8bd2fc-6284-47c3-89a7-d3cb1839c2ed:1 null

21 Jul 2021 Issues

Batch IDFile TypeFailure PointError Message

3003b16f-2207-4c5b-ab6a-a3a7cc5047f7

NRAA clinicalLANDING

c6971c44-08a0-45a6-a322-2e774c61e1d6

NRAA clinicalLANDING

22 Jul 2021 Issues

All files were marked processed on July 22. 

23 Jul 2021 Issues

Batch IDFile TypeFailure PointError Message

d51dbd9b-e5d6-4ab0-8b15-9f7a89033150

NRAA clinicalLANDINGRecorded as success in LANDING, no further logging found

2d429413-9933-4950-ae2d-7b2e5e471268

NRAA clinicalLANDINGRecorded as success in LANDING, no further logging found

a6981b2d-b4ae-4037-abe3-4fa923dd9dd6

NRAA clinicalLANDING

5aa49a74-64f7-4331-b2e7-86986a299307

NRAA demographicLANDING

2021-07-23 07:50:18 7eec80c8-21b8-4d33-97a7-c804e92fedd2 ERROR LandingHandler:275 - Exception occured in Landing Handler null

Jul 23, 2021 7:50:18 AM gov.cms.eqrs.edsm.aws.dynamodb.DynamoDBUtils insertHistoryRecord

INFO: History Record is inserted to the Dynamo Database successfully 

2021-07-23 07:50:18 7eec80c8-21b8-4d33-97a7-c804e92fedd2 INFO LandingHandler:141 - logged the status to history table as failure

2021-07-23 07:50:18 7eec80c8-21b8-4d33-97a7-c804e92fedd2 INFO LandingLambda:79 - result = FAILURE

24 Jul 2021 Issues

Batch IDFile TypeFailure PointError Message

0b3a68e4-2880-4c90-837e-0e952b3c5211

NRAA reportingMeasureLANDING2021-07-24 19:59:57 02fa9be9-4296-49c3-addc-cbd1275785eb ERROR LandingHandler:275 - Exception occured in Landing Handler null
Jul 24, 2021 7:59:57 PM gov.cms.eqrs.edsm.aws.dynamodb.DynamoDBUtils updateFileProcessedStatus
INFO: File task count is updated to the Dynamo Database successfully.
Jul 24, 2021 7:59:57 PM gov.cms.eqrs.edsm.aws.dynamodb.DynamoDBUtils insertHistoryRecord
INFO:  History Record is inserted to the Dynamo Database successfully 
2021-07-24 19:59:57 02fa9be9-4296-49c3-addc-cbd1275785eb INFO  LandingHandler:141 - logged the status to history table as failure
2021-07-24 19:59:57 02fa9be9-4296-49c3-addc-cbd1275785eb INFO  LandingLambda:79 - result =  FAILURE

8efdfa28-9c4b-4e9c-b34c-8955ee4d41d9

NRAA reportingMeasureLANDING2021-07-24 19:59:57 0dd0b07f-2de9-4c24-be3f-79fad6c88183 ERROR LandingHandler:275 - Exception occured in Landing Handler null
2021-07-24 19:59:57 0dd0b07f-2de9-4c24-be3f-79fad6c88183 INFO  LandingHandler:141 - logged the status to history table as failure
2021-07-24 19:59:57 0dd0b07f-2de9-4c24-be3f-79fad6c88183 INFO  LandingLambda:79 - result =  FAILURE

25 Jul 2021 Issues

Batch IDFile TypeFailure PointError Message
759040bd-1f97-4d88-ba20-e65101a1e114NRAA clinicalCLINICALNo error found in Clinical log
8e553297-868f-450a-92a9-f81157868780NRAA clinicalLANDING
2021-07-26 02:19:41 5a04a920-46b2-4c9e-9dfd-10272445c980 ERROR LandingHandler:275 - Exception occured in Landing Handler null
Jul 26, 2021 2:19:41 AM gov.cms.eqrs.edsm.aws.dynamodb.DynamoDBUtils insertHistoryRecord
INFO:  History Record is inserted to the Dynamo Database successfully 
2021-07-26 02:19:41 5a04a920-46b2-4c9e-9dfd-10272445c980 INFO  LandingHandler:141 - logged the status to history table as failure
2021-07-26 02:19:41 5a04a920-46b2-4c9e-9dfd-10272445c980 INFO  LandingLambda:79 - result =  FAILURE
8073a9c6-0e83-4604-9a53-02b1d1ca66f5NRAA clinicalLANDING2021-07-26 02:19:41 b0a446b8-4e1e-45e3-9c4d-9837d9bd6d13 ERROR LandingHandler:275 - Exception occured in Landing Handler null
Jul 26, 2021 2:19:41 AM gov.cms.eqrs.edsm.aws.dynamodb.DynamoDBUtils insertHistoryRecord
INFO:  History Record is inserted to the Dynamo Database successfully 
2021-07-26 02:19:41 b0a446b8-4e1e-45e3-9c4d-9837d9bd6d13 INFO  LandingHandler:141 - logged the status to history table as failure
2021-07-26 02:19:41 b0a446b8-4e1e-45e3-9c4d-9837d9bd6d13 INFO  LandingLambda:79 - result =  FAILURE
2d8ebde4-4463-4590-a750-0822a8a1b7a2NRAA clinicalCLINICALNo error message found specifically for this file. This could be an 11324 error.

However, lots of errors like the following occur at the time the file was submitted:

2021-07-25 18:32:58.144 WARN [clinical_data,trace_id=aeb568eee0dc4c85,span_id=293b477d662b3e99,span_name=process-request1,username=] [pid=6] [EDSM-worker-thread-p1-4] gov.cms.eqrs.edsm.common.batchsubmit.util.BatchCommonGateway : EXTERNAL_CALL FAILED - getBatchToken: /token/batch 
{}
gov.cms.eqrs.common.exceptions.CustomHttpServerErrorException: Rest Call failed: 
    at gov.cms.eqrs.common.configuration.CustomReponseErrorHandler.handleError(CustomReponseErrorHandler.java:40)
    at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:667)
    at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:620)
    at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:580)
    at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:498)
    at org.springframework.web.client.RestTemplate$$FastClassBySpringCGLIB$$aa4e9ed0.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
    at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
    at org.springframework.cloud.netflix.metrics.RestTemplateUrlTemplateCapturingAspect.captureUrlTemplate(RestTemplateUrlTemplateCapturingAspect.java:33)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
    at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
    at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655)
    at org.springframework.web.client.RestTemplate$$EnhancerBySpringCGLIB$$62ac68d7.exchange(<generated>)
    at gov.cms.eqrs.common.util.RestTemplateUtil.ribbonRestRequestWithSingleObjectBody(RestTemplateUtil.java:130)
    at gov.cms.eqrs.edsm.common.batchsubmit.util.BatchCommonGateway.getBatchToken(BatchCommonGateway.java:83)
    at gov.cms.eqrs.edsm.batchsubmit.ClinicalBatchSubmitClient.getBatchAccessToken(ClinicalBatchSubmitClient.java:122)
    at gov.cms.eqrs.edsm.batchsubmit.ClinicalBatchSubmitClient.getFacilityId(ClinicalBatchSubmitClient.java:208)
    at gov.cms.eqrs.edsm.batchsubmit.ClinicalBatchSubmitClient.getClinicalResponse(ClinicalBatchSubmitClient.java:319)
    at gov.cms.eqrs.edsm.batchsubmit.ClinicalBatchSubmitClient.processClinicalData(ClinicalBatchSubmitClient.java:384)
    at gov.cms.eqrs.edsm.batchsubmit.ClinicalBatchSubmitClient.executeTask(ClinicalBatchSubmitClient.java:431)
    at gov.cms.eqrs.edsm.common.batchsubmit.request.BatchSubmitListener.processRequest(BatchSubmitListener.java:34)
    at gov.cms.eqrs.edsm.common.batchsubmit.request.BatchSubmitListener.processRequest1(BatchSubmitListener.java:56)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:723)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
    at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
    at org.springframework.cloud.sleuth.instrument.async.TraceAsyncAspect.traceBackgroundThread(TraceAsyncAspect.java:68)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
    at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
    at org.springframework.aop.interceptor.AsyncExecutionInterceptor$1.call(AsyncExecutionInterceptor.java:115)
    at org.springframework.cloud.sleuth.instrument.async.SpanContinuingTraceCallable.call(SpanContinuingTraceCallable.java:54)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: org.springframework.web.client.HttpServerErrorException: 500 Internal Server Error
    at gov.cms.eqrs.common.configuration.CustomReponseErrorHandler.handleError(CustomReponseErrorHandler.java:39)
    ... 57 common frames omitted
2021-07-25 18:32:58.144 ERROR [clinical_data,trace_id=aeb568eee0dc4c85,span_id=293b477d662b3e99,span_name=process-request1,username=] [pid=6] [EDSM-worker-thread-p1-4] gov.cms.eqrs.edsm.common.batchsubmit.util.BatchCommonGateway : EXTERNAL_CALL FAILED - getFacilityByExternalId
java.lang.IllegalStateException: No thread-bound request found: Are you referring to request attributes outside of an actual web request, or processing a request outside of the originally receiving thread? If you are actually operating within a web request and still receive this message, your code is probably running outside of DispatcherServlet/DispatcherPortlet: In this case, use RequestContextListener or RequestContextFilter to expose the current request.
    at org.springframework.web.context.request.RequestContextHolder.currentRequestAttributes(RequestContextHolder.java:131)
    at gov.cms.eqrs.common.util.RestTemplateUtil.getCurrentRequest(RestTemplateUtil.java:190)
    at gov.cms.eqrs.common.util.RestTemplateUtil.ribbonRestRequestWithUsername(RestTemplateUtil.java:141)
    at gov.cms.eqrs.common.util.RestTemplateUtil.ribbonRestRequest(RestTemplateUtil.java:136)
    at gov.cms.eqrs.edsm.common.batchsubmit.util.BatchCommonGateway.ribbonRestRequest(BatchCommonGateway.java:168)
    at gov.cms.eqrs.edsm.common.batchsubmit.util.BatchCommonGateway.ribbonRestRequest(BatchCommonGateway.java:191)
    at gov.cms.eqrs.edsm.common.batchsubmit.util.BatchCommonGateway.getFacilityByExternalId(BatchCommonGateway.java:101)
    at gov.cms.eqrs.edsm.batchsubmit.ClinicalBatchSubmitClient.getFacilityId(ClinicalBatchSubmitClient.java:215)
    at gov.cms.eqrs.edsm.batchsubmit.ClinicalBatchSubmitClient.getClinicalResponse(ClinicalBatchSubmitClient.java:319)
    at gov.cms.eqrs.edsm.batchsubmit.ClinicalBatchSubmitClient.processClinicalData(ClinicalBatchSubmitClient.java:384)
    at gov.cms.eqrs.edsm.batchsubmit.ClinicalBatchSubmitClient.executeTask(ClinicalBatchSubmitClient.java:431)
    at gov.cms.eqrs.edsm.common.batchsubmit.request.BatchSubmitListener.processRequest(BatchSubmitListener.java:34)
    at gov.cms.eqrs.edsm.common.batchsubmit.request.BatchSubmitListener.processRequest1(BatchSubmitListener.java:56)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:333)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:190)
    at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:723)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
    at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
    at org.springframework.cloud.sleuth.instrument.async.TraceAsyncAspect.traceBackgroundThread(TraceAsyncAspect.java:68)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:629)
    at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:618)
    at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
    at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
    at org.springframework.aop.interceptor.AsyncExecutionInterceptor$1.call(AsyncExecutionInterceptor.java:115)
    at org.springframework.cloud.sleuth.instrument.async.SpanContinuingTraceCallable.call(SpanContinuingTraceCallable.java:54)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)
312601d7-2f81-4090-a402-0c141c871e68NRAA clinicalCLINICALSame as 2d8ebde4-4463-4590-a750-0822a8a1b7a2
2e8fe91e-ff62-4945-94a0-4a23f0c6b324NRAA clinicalCLINICAL
5d7b2e94-e7a0-48de-9909-ef055aaa151bNRAA clinicalCLINICAL
b5de6045-0dfd-4ea1-8b43-96f55fedddbfNRAA clinicalCLINICAL
e8715ab8-e506-4b00-a193-54abb57a362eNRAA clinicalCLINICAL
b47e129a-efef-41ad-9d4b-da0a6c2f764eNRAA clinicalCLINICAL
d7e5d9cc-ea20-4a9d-99b4-520d8cca642aNRAA clinicalCLINICAL
2434bb58-fb74-47ba-b9ae-a7a5f1daea03NRAA clinicalCLINICAL
d718b15d-deb6-4d17-a5a1-8129ca14bca1NRAA clinicalCLINICAL
509362a4-709d-4b14-b141-bc5918f43e19NRAA clinicalCLINICAL
85e52b4c-a048-4c24-bbc4-7edcd88379c2NRAA clinicalCLINICAL
634646d7-817c-47a5-bddb-1ae807fa1abeNRAA clinicalCLINICAL
024e47f6-b434-4a13-ba1d-1ff759b8dd5fNRAA clinicalCLINICAL
f4d28911-72e0-49e9-8616-19ba96b44e9cNRAA clinicalCLINICAL
182ffcb6-bca1-4275-b83e-29521ff969b9NRAA clinicalCLINICAL
4724fdc0-a0a4-485b-9d29-44b757bcab23NRAA clinicalCLINICAL
ff6b0a5b-973a-48e8-81be-6cc893d193ebNRAA clinicalCLINICAL
1c2ed1ed-7315-47a6-b7fe-95c3b8d27aa0NRAA clinicalCLINICAL
0c14740b-839d-4d0e-8d46-32b8f5fdf870NRAA clinicalCLINICAL
e7b5c7c9-11e4-4a63-b501-616bb516464fNRAA demographicLANDING2021-07-25 09:29:00 318d5d6b-4ee9-48be-a428-63e49e56d3cb ERROR LandingHandler:275 - Exception occured in Landing Handler null
2021-07-25 09:29:00 318d5d6b-4ee9-48be-a428-63e49e56d3cb INFO  LandingHandler:141 - logged the status to history table as failure
2021-07-25 09:29:00 318d5d6b-4ee9-48be-a428-63e49e56d3cb INFO  LandingLambda:79 - result =  FAILURE
END RequestId: 318d5d6b-4ee9-48be-a428-63e49e56d3cb
6b6994c9-2bc3-4b4d-a79c-0bb60ba31536NRAA demographicLANDING2021-07-25 09:29:00 e29acbec-9ba7-4d37-8faa-df14807e4e84 ERROR LandingHandler:275 - Exception occured in Landing Handler null
2021-07-25 09:29:00 e29acbec-9ba7-4d37-8faa-df14807e4e84 INFO  LandingHandler:141 - logged the status to history table as failure
2021-07-25 09:29:00 e29acbec-9ba7-4d37-8faa-df14807e4e84 INFO  LandingLambda:79 - result =  FAILURE
END RequestId: e29acbec-9ba7-4d37-8faa-df14807e4e84
0cd5ca91-92e4-4bb3-92b3-616c04c60a60NRAA demographic
unsure of where the error occurred
4442f21d-2a33-48c8-9afb-d9bd6d87fef7NRAA demographicLANDING2021-07-25 09:29:00 85c7df22-76c5-46cb-8b54-b927e06f9185 ERROR LandingHandler:275 - Exception occured in Landing Handler null
Jul 25, 2021 9:29:00 AM gov.cms.eqrs.edsm.aws.dynamodb.DynamoDBUtils updateFileProcessedStatus
INFO: File task count is updated to the Dynamo Database successfully.
Jul 25, 2021 9:29:00 AM gov.cms.eqrs.edsm.aws.dynamodb.DynamoDBUtils insertHistoryRecord
INFO:  History Record is inserted to the Dynamo Database successfully 
2021-07-25 09:29:00 85c7df22-76c5-46cb-8b54-b927e06f9185 INFO  LandingHandler:141 - logged the status to history table as failure
2021-07-25 09:29:00 85c7df22-76c5-46cb-8b54-b927e06f9185 INFO  LandingLambda:79 - result =  FAILURE
END RequestId: 85c7df22-76c5-46cb-8b54-b927e06f9185

Data Submissions from 07-12-2021

LDOData Type

 

 

 

 

 

 

 

Week Total
DAVITA

Clinical47842528560397,10700400,866
Demographic8328035,478070043,813
Reporting measures383,7510380,2050000763,956
DCI

Clinical895000000895
Demographic03010043000731
Reporting measures00000000
FKC

Clinical010,1671388307100014,626
Demographic787443878570919444644829279742,115
Reporting measures00000000
NRAA

Clinical16,08153,49674,08444,28048,71432,26112655281,571
Demographic825318,17412,291358815,416138957,869
Reporting measures2508817912,05331702848896029,654

12 Jul 2021 Issues

Batch IDFile TypeFailure PointError Message
c253a70d-56b5-451f-a055-bceb4d58fa06NRAA demographicLANDING2021-07-12 21:39:16 50257e89-2870-4290-98ca-e84e1bea77aa ERROR LandingHandler:275 - Exception occured in Landing Handler null
6f6b3018-47b5-42ab-a437-cdf33ff0ad4aNRAA clinicalFEEDBACK{"code":601,"value":"Clinical month is closed and cannot accept new clinical data."}]}
ee02e29c-2bc0-4f80-bfae-05fc2759a5d1NRAA clinicalFEEDBACK2021-07-12 10:04:18.578 ERROR [batchfeedback_data,trace_id=6e983398ca4a6e72,span_id=15eae22f6f2f8c45,span_name=,username=] [pid=7] [EDSM-worker-thread-p1-9] gov.cms.eqrs.edsm.batchfeedback.BatchFeedbackManager :  Exception occured in processTask:  ee02e29c-2bc0-4f80-bfae-05fc2759a5d1:1 Throughput exceeds the current capacity for one or more global secondary indexes. DynamoDB is automatically scaling your index so please try again shortly. (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: ThrottlingException; Request ID: ACKAPGT2LH56LEAQQ4IVO4Q07VVV4KQNSO5AEMVJF66Q9ASUAAJG)
a532277b-915c-42fd-9caf-2a484f0a3e27NRAA demographicFEEDBACK{"code":100,"value":"Patient could not be processed due to possible matches"}]}

13 Jul 2021 Issues

Batch IDFile TypeFailure PointError Message

87e4c213-48f4-4947-a96c-16cd5b147bea

NRAA clinicalLANDING

2021-07-14 02:01:42 17aea096-138b-480c-bc5d-ee2cf66f0af4 ERROR LandingHandler:275 - Exception occured in Landing Handler null

6e5db1ac-76be-4c9c-bc3c-abcf6894bd92

NRAA clinicalLANDING

2021-07-14 02:01:43 b6e117ca-97b7-4e88-91c5-b3c6b1696d1e ERROR LandingHandler:275 - Exception occured in Landing Handler null

4e99318c-0e42-49d3-8a9a-b23ac025b6a0

NRAA clinicalLANDING

2021-07-13 21:34:07 80ce5b9a-4bb4-41b2-803c-ec0d2c721e5a ERROR LandingHandler:275 - Exception occured in Landing Handler null

dd67ec06-fbcc-4515-863f-d4d40f45a522

NRAA reporting measure

BATCH_FEEDBACK

2021-07-13 17:33:56.728 ERROR [batchfeedback_data,trace_id=eff5477a150ea836,span_id=4f71c27337904864,span_name=,username=] [pid=7] [EDSM-worker-thread-p1-3] gov.cms.eqrs.edsm.batchfeedback.BatchFeedbackManager :  Exception occured in processTask:  dd67ec06-fbcc-4515-863f-d4d40f45a522:1 null

22d402c4-f313-4338-856c-b3010b09f5ea

NRAA reporting measure

BATCH_FEEDBACK

2021-07-13 16:59:40.473 ERROR [batchfeedback_data,trace_id=ea0d77a7450ea09a,span_id=8fbbb32e90598540,span_name=,username=] [pid=7] [EDSM-worker-thread-p1-7] gov.cms.eqrs.edsm.batchfeedback.BatchFeedbackManager :  Exception occured in processTask:  22d402c4-f313-4338-856c-b3010b09f5ea:1 null

b6fe5e1c-019f-4511-b112-ba7692bd8913

NRAA reporting measure

BATCH_FEEDBACK

2021-07-13 16:44:38.025 ERROR [batchfeedback_data,trace_id=f3795d99a3e4506c,span_id=d313a36258cfd422,span_name=,username=] [pid=7] [EDSM-worker-thread-p1-6] gov.cms.eqrs.edsm.batchfeedback.BatchFeedbackManager :  Exception occured in processTask:  b6fe5e1c-019f-4511-b112-ba7692bd8913:1 null
d6ae5833-aa1f-40df-ba06-0a6e5b414c2dNRAA clinicalLANDING

2021-07-13 09:32:31 9b47b580-01e4-4a13-8fd5-4644e0e9b416 ERROR LandingHandler:275 - Exception occured in Landing Handler null

14 Jul 2021 Issues

Batch IDFile TypeFailure PointError Message
b1e930d7-3b90-4711-9d24-fb165fd48bd5NRAA clinicalLANDING

2021-07-15 00:37:51 9d5667f6-996c-44ff-b917-52c3d6a556b2 ERROR LandingHandler:275 - Exception occured in Landing Handler null

c6b12f81-2ac9-484f-b649-f3a9659eb438NRAA clinicalLANDING

2021-07-15 00:27:03 07f687c8-09ee-489d-9232-6a4badafd064 ERROR LandingHandler:275 - Exception occured in Landing Handler null

a489ce11-2c33-46ff-b6c6-f5c3f3750ebbNRAA clinicalLANDING2021-07-14 23:10:54 270f6c34-1d00-4c2a-8300-d0f7040f6c7a ERROR LandingHandler:275 - Exception occured in Landing Handler null 
80f064f5-a641-4eed-9ef0-8e953cdad828NRAA demographicBATCH_FEEDBACK2021-07-14 18:57:18.466 ERROR [batchfeedback_data,trace_id=9f5dbd66ca050a66,span_id=ce7609507648ab6e,span_name=,username=] [pid=7] [EDSM-worker-thread-p1-1] gov.cms.eqrs.edsm.batchfeedback.BatchFeedbackManager :  Exception occured in processTask:  80f064f5-a641-4eed-9ef0-8e953cdad828:1 null
551f1662-5fe9-41c1-aea7-d1d924f5ffb3NRAA clinicalBATCH_FEEDBACK2021-07-14 18:57:16.912 ERROR [batchfeedback_data,trace_id=f97d0ab2a3393920,span_id=710d00e5f4675ac2,span_name=,username=] [pid=7] [EDSM-worker-thread-p1-4] gov.cms.eqrs.edsm.batchfeedback.BatchFeedbackManager :  Exception occured in processTask:  551f1662-5fe9-41c1-aea7-d1d924f5ffb3:1 null
4b257ba7-1e65-4848-bca8-57cddff6ac6fNRAA clinicalBATCH_FEEDBACK2021-07-14 16:19:17.792 ERROR [batchfeedback_data,trace_id=acc40d9fe89c405f,span_id=2c7fcf1b81043470,span_name=,username=] [pid=7] [EDSM-worker-thread-p1-2] gov.cms.eqrs.edsm.batchfeedback.BatchFeedbackManager :  Exception occured in processTask:  4b257ba7-1e65-4848-bca8-57cddff6ac6f:1 null
92efa92b-253d-49e3-9f91-7b4d97fc1f66NRAA clinicalBATCH_FEEDBACK2021-07-14 12:05:51.692 ERROR [batchfeedback_data,trace_id=feb4bc0769f2e7dc,span_id=71a617088ad7ef0a,span_name=,username=] [pid=7] [EDSM-worker-thread-p1-1] gov.cms.eqrs.edsm.batchsubmit.FeedbackBatchSubmitClient : Exception occured for BatchTaskId: 92efa92b-253d-49e3-9f91-7b4d97fc1f66:1 Error: null
e794ef44-d70b-4851-ad28-51371a4551a4NRAA clinicalBATCH_FEEDBACK2021-07-14 09:06:13.818 ERROR [batchfeedback_data,trace_id=e73387f706a73c68,span_id=2865a0c258fa3588,span_name=,username=] [pid=7] [EDSM-worker-thread-p1-8] gov.cms.eqrs.edsm.batchfeedback.BatchFeedbackManager :  Exception occured in processTask:  e794ef44-d70b-4851-ad28-51371a4551a4:1 null

15 Jul 2021 Issues

Due to a network outage on this date, there are too many unprocessed files to document here. 

16 Jul 2021 Issues

Batch IDFile TypeFailure PointError Message
d9d899d7-b710-4d36-84c0-16037459d69fNRAA clinicalLANDING

Shows up as a failure in DynamoDB, and the landing logs show it logged as an error. but no stack trace is found in landing at the relevant time.

5184551d-98c7-4507-a26a-21efda5c86a4

NRAA clinicalBATCH_FEEDBACK

2021-07-16 15:28:14.620 ERROR [batchfeedback_data,trace_id=92642bb90174fbeb,span_id=3bf0eb2054d9bcad,span_name=,username=] [pid=7] [EDSM-worker-thread-p1-5] gov.cms.eqrs.edsm.batchsubmit.FeedbackBatchSubmitClient : Exception occured for BatchTaskId: 5184551d-98c7-4507-a26a-21efda5c86a4:1 Error: null

java.lang.NullPointerException

ca9a693b-014f-4923-b8fa-4baf8a7028ac

NRAA clinicalLANDING

Shows up as a failure in DynamoDB, but no error shows up in the landing logs

40ef96e7-0357-43df-bfc0-504247ad6d44

NRAA clinicalLANDINGShows up as a failure in DynamoDB, and the landing logs show it logged as an error. but no stack trace is found in landing at the relevant time.

c974579d-f063-40aa-94e1-558f7cd9c7a1

NRAA reportingMeasureLANDINGShows up as a failure in DynamoDB, and the landing logs show it logged as an error. but no stack trace is found in landing at the relevant time.

01f4b0b4-83a1-46c6-9ea9-c6c0e3627894

NRAA clinicalLANDINGshows "SUCCESS" in dynamo for landing, then no errors (or anything else) in clinical/feedback.

d1df05d3-4967-48e0-bf59-07dd8e0d982e

NRAA clinicalunclearshows "SUCCESS" in dynamo for landing, then no errors (or anything else) in clinical/feedback.

81312891-2953-48c0-a4de-c22accc301ed

NRAA clinicalBATCH_FEEDBACK

{"orgFacilityCode":"nraa10110321","orgPatientId":"nraa10110321-341992","crownPatientId":2105263233,"crownFacilityId":null,"errors":{"error":[{"code":99999,"value":"An unexpected server error seems to have occurred. You can try resubmitting this form in a moment. If the problem persists, please contact the help desk"}]},"warnings":{"warning":[]}}

84fd6f40-7a34-4eb9-909b-e855c40616ec

NRAA clinicalLANDINGShows up as a failure in DynamoDB, and the landing logs show it logged as an error. but no stack trace is found in landing at the relevant time.

17 Jul 2021 Issues

Batch IDFile TypeFailure Point

Error Message

6e428f02-fba6-4152-a889-d2d80d81216d

NRAA reportingMeasureBATCH_FEEDBACK2021-07-17 05:08:06.027 ERROR [batchfeedback_data,trace_id=503ca49fbf053932,span_id=9354c4096bbc8ddf,span_name=,username=] [pid=7] [EDSM-worker-thread-p1-6] gov.cms.eqrs.edsm.batchfeedback.BatchFeedbackManager :  Exception occured in processTask:  6e428f02-fba6-4152-a889-d2d80d81216d:1 null

2f4f7f95-f1a8-4e80-9f05-ca308efa65a5

NRAA reportingMeasureBATCH_FEEDBACK2021-07-17 05:08:06.069 ERROR [batchfeedback_data,trace_id=072cbdfd2f515837,span_id=58d1639ffd69a535,span_name=,username=] [pid=7] [EDSM-worker-thread-p1-7] gov.cms.eqrs.edsm.batchfeedback.BatchFeedbackManager :  Exception occured in processTask:  2f4f7f95-f1a8-4e80-9f05-ca308efa65a5:1 null

18 Jul 2021 Issues

Batch IDFile TypeFailure PointError Message

992de05b-0af3-400f-8e7a-3b19e195d8f9

NRAA clinicalBATCH_FEEDBACK2021-07-18 22:23:27.323 ERROR [batchfeedback_data,trace_id=b85ab64d0fd74cac,span_id=aa9b08456407c9de,span_name=,username=] [pid=7] [EDSM-worker-thread-p1-3] gov.cms.eqrs.edsm.batchfeedback.BatchFeedbackManager :  Exception occured in processTask:  992de05b-0af3-400f-8e7a-3b19e195d8f9:1 null

a310a104-b79a-4985-a811-c3ffa1e9045b

NRAA clinicalLANDING2021-07-19 01:50:48 a933c937-94a1-4535-b350-97de6c75bed4 ERROR LandingHandler:275 - Exception occured in Landing Handler null

47380de7-bc72-45eb-a162-ea23c0662f58

NRAA clinicalLANDING2021-07-19 01:50:48 fa7d6a02-f1a7-4b13-86d2-5e52234a9f3c ERROR LandingHandler:275 - Exception occured in Landing Handler null

Instructions for generating this data

The script I use to pull this data is located here: https://qnetgit.cms.gov/preston-brown-1/scripts-public/blob/master/dynamo_count_uploads_by_day_and_ldo.py. 
You will also need to download this script and put it in the same directory: https://qnetgit.cms.gov/preston-brown-1/scripts-public/blob/master/dynamo_utils.py

Here are the instructions for running it: 

usage: dynamo_count_uploads_by_day_and_ldo.py [-h] environment date

positional arguments:
  environment  environment to search such as prod or dev2-3
  date         date to search in YYYY-MM-DD format

optional arguments:
  -h, --help   show this help message and exit


  • No labels