Expert Data Engineer
Edvantis, Львів, Ряшів (Польща)
- Формат:
- повний remote
- Джерело:
- jobs.dou.ua
Що робити
- Own the target architecture: medallion layering, Unity Catalog structure (metastore, catalogs, schemas, naming, environments), and the storage layout
- Evaluate ingestion options (Auto Loader, Lakeflow Connect, Lakeflow Declarative Pipelines, ADF/Fivetran, streaming vs batch) and recommend an approach with cost, complexity and latency trade-offs
- Define the governance model: access control and grants; external locations and storage credentials; lineage, tagging, and PII handling
- Set the compute strategy: serverless vs classic, job clusters, SQL warehouses, cluster policies, and cost controls
- Define standards for orchestration (Jobs/Workflows), data maintenance (optimization, retention, vacuum, schema evolution) and data quality
Що очікуємо
- 7+ years in Data Engineering or architecture, with 3+ years designing Databricks platforms in Production
- Deep Unity Catalog expertise: metastore design, managed vs external tables and volumes, external locations, permissions model, lineage, and system tables
- Proven skills in end-to-end medallion lakehouse design and delivery
- Strong Delta Lake internals knowledge: MERGE/CDC patterns, liquid clustering or Z-order, CDF, time travel, and maintenance
- Broad knowledge of Databricks ingestion patterns and the ability to justify choices between them
Схожі вакансії
З блогу Trackr
Усі статті →
Знайдено через trackr.help/jobs · Канал: @trackrhelp · Бот для персональних сповіщень: @trackrhelpBot