| Section | Weight | Objectives |
|---|
| Prepare and process data | 30–35% | - Optimize and manage data storage
- 1. Optimize Delta tables: partitioning, Z-ordering, vacuum, optimize
- 2. Handle structured, semi-structured, and unstructured data
- 3. Implement lakehouse architecture and manage table versions
- Ingest and transform data
- 1. Ingest batch and streaming data from multiple sources
- 2. Transform using Spark SQL, PySpark, Scala, and Delta Lake
- 3. Implement schema enforcement, schema drift, and slowly changing dimensions
|
| Deploy and maintain data pipelines and workloads | 30–35% | - Monitor, troubleshoot, and maintain workloads
- 1. Troubleshoot failures, repair and restart jobs
- 2. Monitor performance, logs, and execution metrics
- 3. Apply SDLC practices and version control
- Build and orchestrate pipelines
- 1. Design and implement Lakeflow Spark Declarative Pipelines
- 2. Configure Lakeflow Jobs: schedules, triggers, alerts, retries
- 3. Implement CI/CD with Git, Databricks Asset Bundles, CLI, and APIs
|
| Secure and govern Unity Catalog objects | 15–20% | - Implement data governance and security
- 1. Configure access control: row-level, column-level, attribute-based security
- 2. Manage catalogs, schemas, tables, views, and volumes
- 3. Enforce data quality, lineage, and auditing
- Manage data sharing and permissions
- 1. Set up external locations and storage credentials
- 2. Grant and revoke permissions, manage groups and service principals
|
| Set up and configure an Azure Databricks environment | 15–20% | - Integrate with Azure services
- 1. Connect to Azure Data Lake Storage, Azure Data Factory, Microsoft Entra ID
- 2. Configure monitoring with Azure Monitor and diagnostic settings
- Select and configure compute resources
- 1. Choose compute types: serverless, job compute, SQL warehouse, classic compute
- 2. Manage workspace settings, permissions, and networking
- 3. Configure cluster policies, instance pools, and libraries
|
