Senior DevOps and Platform Engineer (Azure Databricks)

Zoral

Що робити

Build the platform foundation as code with Terraform: workspaces, the Unity Catalog metastore and one catalog per environment (development, test, production, with production isolated from non-production), storage, identities and permissions.
Implement the CI/CD route to live for data products with Declarative Automation Bundles (formerly Databricks Asset Bundles) and Azure DevOps or GitHub Actions: every pipeline, rule, semantic model and report change reviewed, tested and promoted through environments.
Deliver the platform backlog with the Head of Data Platform: Hive metastore to Unity Catalog migration, high availability and disaster recovery, privileged access on demand, masking of personal data and remediation of penetration-test findings.
Set up observability: run metrics, freshness, quality results and cost from Databricks system tables; audit events through Azure Monitor diagnostic settings; job, quality-gate and freshness alerts into ServiceNow and Teams with runbook references; service-level objectives for critical assets.
Run cost governance (FinOps): compute policies, job and serverless compute, separate SQL warehouses for critical reporting, self-service and data science, budgets with alerts, usage policies and billing system tables as the source of truth.

Five or more years in DevOps, platform or cloud engineering, including strong experience running Azure Databricks in production.
Strong Terraform with the AzureRM and Databricks providers: modules, state management, code review and multi-environment layouts.
Hands-on Databricks administration: workspaces, Unity Catalog (metastore, catalogs, grants, service principals, system tables), compute policies, jobs and secrets.
CI/CD for data platforms in Azure DevOps or GitHub Actions, including automated tests, quality and security checks, approvals and promotion between environments.
Azure networking and identity: VNet injection, Private Link and private DNS, Entra ID, managed identities and service principals, Key Vault, role-based access control.

Знайдено через trackr.help/jobs · Канал: @trackrhelp · Бот для персональних сповіщень: @trackrhelpBot