
Senior Data Engineer
Job Description
About the Role
Join a data and infrastructure team building a platform that transforms information from multiple sources into reliable, up-to-date company intelligence. You will own the platform end to end, from architecture and ingestion to production operations.
What You'll Do
Design a multi-source platform for concurrent ingestion, event-driven processing, and orchestrated batch workflows.
Build reliable intake from provider APIs, webhooks, files, and web scraping, with rate limits and robust failure handling.
Orchestrate scheduled workflows and historical backfills using tools such as Airflow, Dagster, or Prefect.
Develop event-driven systems with messaging and task queues, including retries, dead-letter queues, and controlled replay.
Store original payloads in cloud storage and transform them with dbt into raw, standardized, and curated layers.
Implement idempotent upserts, data contracts, automated validation, pipeline tests, and monitoring for freshness and failures.
Design and optimize PostgreSQL schemas and loading patterns, and integrate provider enrichment and structured extraction using LLMs.
What We're Looking For
At least 5 years building and operating production data platforms, including multi-source ingestion and ownership from architecture through ongoing operation.
Production experience with workflow orchestration, including dependencies, retries, and recovery. Airflow is preferred; Dagster, Prefect, Temporal, or comparable tools are relevant.
Hands-on production experience with dbt and Google Cloud Platform.
Strong Python skills, including asynchronous HTTP and concurrent processing, and strong SQL and PostgreSQL skills in modeling, query optimization, and incremental loading.
Experience with event-driven processing, queues, data contracts, raw and curated storage, retention policies, testing, and monitoring.
Experience building platforms or substantial pipelines from scratch is valuable. Web scraping, entity resolution, company data enrichment, LLM-based extraction, and small-team or startup experience are also relevant.
Compensation & Benefits
Salary range: $100,000 to $140,000. Visa sponsorship is not available.
Location
Remote within Europe, including the United Kingdom, Norway, Switzerland, Ukraine, Iceland, and Liechtenstein.