Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Include Page
Data Nav_C
Data Nav_C


Back to Townhalls Home Page


CCSQ Data & Analytics Townhall

Date

Wednesday, July 26, 2023, at 1:00 pm ET

Recording

CCSQ D&A - July Townhall Video

Agenda

  1. Data Camp Topics Poll

  2. Monthly Satisfaction Survey Review & Poll

  3. Data Lifecycle Management, Versioning, and Archiving Policies

  4. Ambari Decommission Update

  5. Databricks Release Update

Recording

https://www.zoomgov.com/rec/share/X0KJb6wksCgt00Bs6yO6pbiLJP0X-Sgd6N_KMysuZbJeYHeQgyxA7QgqbyfFZxCt.ivjNX6m5zz1c-_L8
Passcode: Nb9VF=$z 
Monthly Satisfaction Survey Review

Reference the recording to see the results of the previous month's poll. 

Timestamp 06:40. 

Data Lifecycle Management, Versioning, and Archiving Policies

User Data Versioning Policies

File usage

Number of Versions

Age of Version

Active Use

Unlimited (subject to user storage area size)

30 days per version

Non-Active Use

Two + current

Per Data Retention Policy

User Data Standard Tiering Policies

AWS Storage Type

AWS Tier Type

Current State Tiering

Recommended Tiering Future State (Portal Implementation)

Retrieval Time

Simple Storage Service (S3) Intelligent Tiering

S3 Standard

All actively accessed data will reside in this tier

All actively accessed data will reside in this tier

milliseconds

S3 Standard-Infrequent Access (S3 Standard-IA)

Data not accessed in 30 days

Data not accessed in 30 days

milliseconds

S3 Glacier Instant Retrieval

Data not accessed in 90 days

Data not accessed in 90 days

milliseconds

Next Steps

  • We will be moving forward with versioning and standard tiering at this time.
  • We have collected your feedback and will be working with groups that expressed concern around archiving to better understand their use case over the next few weeks.
  • Additional information on archiving will be forthcoming.
Ambari Decommission and Transition Updates

Decommissioning Process

  • What is needed to fully turn off Ambari?
    • CASLIB Migration.
    • Structured Query Language (SQL) cluster onboarding.
    • Bring Your Own Analytics (BYOA) migration.
    • Hive to Glue Migration.

Where are we now?

  • Half CASLIB have been migrated.
  • Next maintenance another half will be done.
  • SQL cluster is being rolled out currently.

What is coming next?

  • BYOA connection working sessions - About ten service accounts currently.
  • Hive to Glue migration - Currently testing scripts now.

Why should you come to DBX?

  • Better performance and stability.
  • Faster querying times.
  • Less unplanned maintenance.
  • More opportunity for us to invest in better features for users.
  • Faster deliverables for you all.
Databricks (DBX) Release Update

SQL Cluster Release

  • What does this mean?
    • One for whole community vs per organization.
  • How can you connect?
    • Only by use of a Service Principal and Token.
  • Known differences/issues between SAS and DBX.
    • Document will be shared with user community.

Support

Training 

  • Located on the user facing CCSQ site: Training Hub
  • Submit any new training requests via email or Slack.

Demo

  • How to Connect SAS to DBX.
  • Sample Code.
  • This recording will be shared alongside the release for questions regarding the connection of SAS to DBX.
    • This will be shared with helpdesk team as well if you put in a request regarding connections.
    • There will also be a document outlining the details for connection.

Q&A

7-28-2023 Townhall Q&A