Senior/Lead Data Engineer (#5535)
- Рівень:
- senior
- Джерело:
- djinni.co
Що робити
- Gather and analyse business requirements to identify and prioritise integration opportunities.
- Design, build, and maintain scalable data pipelines using Azure Data Factory to orchestrate ingestion from diverse source systems.
- Develop and optimise data transformation logic and analytical workloads in Azure Synapse Analytics (dedicated and serverless SQL pools, Synapse Pipelines).
- Build and maintain Databricks notebooks and jobs (PySpark / Spark SQL) for large-scale data processing and feature engineering.
- Implement robust API extraction patterns — REST and SOAP — handling pagination, authentication (OAuth 2.0, API keys), rate limiting, and error recovery.
Що очікуємо
- 4+ years of experience in data engineering or a closely related software engineering role.
- Strong analytical and communication skills
- Strong hands-on experience with Azure Data Factory — authoring pipelines, triggers, linked services, and data flows.
- Solid experience with Azure Synapse Analytics — SQL pools, Synapse Pipelines, and integration with ADLS Gen2.
- Proven experience with Databricks — PySpark, Delta Lake, job clusters, and Unity Catalog is a plus.
Що пропонуємо
- Experience with Azure Purview for data cataloguing and governance.
- Knowledge of streaming data patterns (Azure Event Hubs, Kafka, Spark Streaming).
- Familiarity with dbt or similar SQL-based transformation frameworks.
- Background in mining, heavy industry, or operational technology (OT) data environments.
- Power Platform awareness (not required, but beneficial for understanding downstream consumers).
Схожі вакансії
З блогу Trackr
Усі статті →
Знайдено через trackr.help/jobs · Канал: @trackrhelp · Бот для персональних сповіщень: @trackrhelpBot