Page History

Date 

Thursday, April 27, 2023, at 1:00 pm ET

Recording

Agenda

1. Spring 2023 Data Camp
2. Monthly Satisfaction Survey
3. Databricks

1. (DBX) Demo and Features
2. Usability Testing Involvement 
3. Q&A

Spring Data Camp

• The Center for Clinical Standards and Quality (CCSQ) Data &Analytics (D&A) Team will be holding the next Data Camp next month on Tuesday, May 23rd and Wednesday, May 24th. 

• The mission of the CCSQ D&A Team is to encourage the community to "Come to the Data", as it is vital to build and utilize centralized, robust, and reliable data and analytic solutions. During this Data Camp, we be exploring our theme of  "Expanding the Analytics Toolbox", as we discuss the latest addition to the CCSQ D&A environment, Databricks. We will also explore other topics related to the Centralized Data Repository's (CDR) data and the CCSQ D&A roadmap. 
• Visit this page for session information: CCSQ Data & Analytics Data Camp 

Databricks Kickoff and Demo

What is in Scope? 

• Phase 1 – Available today!
  • All native notebook capabilities and library installations
  • Languages: Python, R, Scala, Structured Query Language (SQL)
  • Workbench and CDR data
  • Two different clusters- R, Python/SQL
  • Users can share their notebooks.
  • Users can visualize their results.
  • Auto machine learning (AutoML) (must be requested)
• Phase 2
  • SAS Viya/Bring Your Own Application (BYOA)/Bring Your Own Data (BYOD) connection to DBX SQL Warehouse

Training Materials

• Onboarding Instructions
• Knowledge Base Articles:
  • Introduction to Databricks Notebook-WIP
  • Tips to Debug Apache Spark
  • Debugging in Databricks Notebooks
  • Debugging with the Apache Spark UI
  • Manage Scheduled Notebook Jobs
  • Copy Zeppelin Notebook to Databricks-WIP
  • Selecting an R Cluster
• Training Hub Videos: 
  • Introduction into Databricks Notebooks

Support

• Questions or Issues?
  • Email: servicecentersos@cms.hhs.gov
  • Slack: #help-service-center-sos
• Training Requests
  • For any training requests, please have your organization lead submit a request via our Request Form

Demo Agenda

• Reference the recording for a demo of the following features: 

  • Training and Support
  • Navigating the Databricks Landing Page
  • Exploring CDR Databases and Tables
  • Creating and Running Notebooks
  • Configuring and Scheduling Job Workflows

Get Involved with Usability Testing

We Need Your Feedback! 

• Feedback from application development organizations (ADOs), CMS, and others in our user community is crucial for our ongoing innovation.
• If you are interested, we want to gather some background information to make sure that we only contact you about things relevant to your work.
• If you are a match for a survey, interview, or prototype test that we are conducting in the future, we will reach out directly via email.
• Please note that your submitted information will not be published anywhere.

Q&A

1. 1. Can the notebooks write HTML?
      A - You can use the IPython modules to write and display custom html.

   2. Can we import and run our own Python and R programs on the CDR datasets?

      A - Reference the recording for a walkthrough of this answer.  
   3. How about R? Ninety percent of my work is in R and so I am looking for R resources. 
      A - Yes, you can import and run R programs as well.
   4. Will there be a way to have multiple python or R project environments within Databricks (like RStudio or Spyder)?
      A -  You can break projects into folders or repos to integrate with version control.
   5. To confirm, can R packages be installed as well, just like with Python? And are there any package restrictions?
      A - R packages can be installed. There currently are not any restrictions, but there may be some packages that would require a cluster-wide installation. If installing them in your notebooks fails, please reach out to the CCSQ D&A team. 
   6. Where are the notebooks stored? Can we save them to our Simple Storage Service (S3) bucket as well?
      A -  Databricks is a Cloud-based software as service (SAAS) platform, so all notebooks are stored in the Cloud.
   7. Is this Databricks intended to be used for all kind of large data analysis tasks using large data tables like complete statistical analytic table (CSAT) Part A Claims?
      A - Yes, that is correct. Also, the beauty of working with Databricks is that clusters will scale with your large operations to successfully run them. 
   8. From permissions and technical standpoints, can we connect the Databricks environment to a local investigational device exemption (IDE), such as Visual Studio (VS) Code, and execute code from there?
      A - We will need to assess that from a security and feasibility standpoint since users are required to authenticate via HCQIS Access Roles and Profile (HARP). However, one thing to keep in mind is that the latest Databricks notebooks just released similar VS Code capabilities that include auto-completion, etc.
   9. Can we get a timeline for the roll out of different functionality with Databricks as well as the sunsetting of any of the currently available tools? It was mentioned that SAS was "short term" so would be interested in understanding more of what was meant by that comment.
      A - The SQL endpoint connection to be able connect SAS Viya is the next feature we are releasing and are working on currently. After that you will have all the features of Databricks available to you, except AutoML which is on a request basis. As stated, Phase 2 should be available within the next month. 
   10. I understand the SAS integration is not currently available and is in Phase 2. I am trying to better understand what comes after that both from an addition of functionality and a removal of tools (if planned). 
       A - Join our upcoming May Data Camp to learn more of the future roadmap for CCSQ Data & Analytics. There are no plans to move away from SAS Viya currently.
       
   11. How can we get information and training on using Databricks for our specific needs? 
       A - Open a request to share the needs of your organization. 
   12. Should we start to move our work in SAS Viya and start to work exclusively in Databricks? Will SAS Viya outside of Databricks be phased out?
       A - There are no plans to move away from SAS Viya at this time.