We are looking for a Senior Data Engineer to own the data platform of a fast-growing startup, from architecture all the way to production. The company helps investors and other teams find and research companies, so reliable, fresh and well-structured data is at the heart of the product. Today the platform processes data one job at a time in containers. You will lead its move to running many jobs at once, with event-driven data intake and orchestrated batch workflows. Your daily work will combine Python, Airflow, dbt, GCP and PostgreSQL with large-scale web scraping from many sources. You will also solve hard data problems, such as matching company and people records across sources and using LLMs to pull structured data out of text. This role suits someone who likes building from zero, enjoys owning a wide area, works well on their own and is comfortable in a fully remote team with no fixed working hours.
Details:
Schedule: Full time
Location: Remote – EU
Start: ASAP
Duration: Long-term
English: Fluent
Type of collaboration: B2B Contract
About the project: The client is a startup building a data platform that helps investors and other teams find and research companies. The platform collects data from many sources, cleans it, links related records and turns it into useful insights about companies. As the product grows, the current one-at-a-time processing needs to become a faster, event-driven system that handles many jobs at once. The Senior Data Engineer will lead this change and own the whole platform, from design to day-to-day running. The team is small, so engineers own a wide area and have a direct impact on the product. The company works fully remotely, and you don't need to overlap with anyone's working hours.
You have:
Experience building a production data platform with many data sources from zero, including its architecture, deployment and day-to-day running
Production experience with workflow orchestration tools (Airflow, Dagster, Prefect or Temporal)
Hands-on experience with dbt and Google Cloud Platform (GCP)
Strong Python skills, including asynchronous HTTP and concurrent processing
Hands-on web scraping experience (Scrapy, Playwright, Selenium or Beautiful Soup)
Strong PostgreSQL skills: data modeling, query optimization and reliable incremental loading
Experience with event-driven architecture, including retries, failure recovery and dead-letter queues
Comfort working fully remotely and on your own, with no fixed hours that overlap with the team
Fluent English
Nice to have:
Experience in startups or small engineering teams where you owned a wide area
Entity resolution experience, especially matching and reconciling company and people records across sources
Experience using LLMs to extract structured data or enrich data
Familiarity with BigQuery and Dataform
Background in company, investment or B2B data platforms
What to do:
Own the data platform from start to finish, from architecture to production and day-to-day running
Lead the move from one-at-a-time container processing to concurrent, event-driven data intake and orchestrated batch workflows
Build and maintain data pipelines with Python, Airflow, dbt and GCP
Build and scale web scraping that runs across many sources at once, handling rate limits and failures
Design PostgreSQL data models, optimize queries, and set up reliable incremental loads, idempotent upserts and backfills of historical data
Build solid failure recovery, retries and dead-letter handling for event-driven data intake
Match and reconcile company and people records across different data sources
Use LLMs to extract and enrich structured data where it helps
