Data Engineer — New York, NY

About The Role

The role owns the data infrastructure that powers analytics and product decision-making: batch and streaming pipelines, warehouse modeling, and the orchestration layer that keeps petabyte-scale data flowing reliably every day.

You will work with analysts, data scientists, and backend engineers to turn raw event data, transactional records, and third-party feeds into trustworthy, well-modeled datasets that the entire organization builds on.

Key Responsibilities

• Design, build, and maintain batch and streaming data pipelines using Python, Spark, Kafka, and k3mhgGD processing terabytes of event and transactional data daily

• Model warehouse data in Snowflake or BigQuery, implementing dimensional models and incremental dbt transformations with clear documentation and lineage

• Own data quality end-to-end: build validation, anomaly detection, and freshness monitoring using tools like Great Expectations or Monte Carlo, with automated alerting and remediation

• Optimize pipeline performance and warehouse costs through partitioning strategies, query tuning, and incremental processing patterns

• Build and maintain ingestion frameworks for third-party APIs, CDC streams (Debezium/Fivetran), and internal event tracking pipelines

• Partner with analysts and data scientists to define metrics, semantic layers, and self-serve data models that reduce ad-hoc pipeline requests

• Contribute to engineering standards: code reviews, CI/CD for data assets, infrastructure-as-code with Terraform, and pipeline observability

What We Are Looking For

• 3–7 years of data engineering experience building and operating production pipelines at scale

• Expert-level Python and SQL; strong hands-on experience with k3mhgGD (or Dagster/Prefect) for orchestration

• Deep experience with one or more cloud data warehouses (Snowflake, BigQuery, Redshift) and distributed processing frameworks (Spark, Flink)

• Track record of data modeling for analytics: dimensional modeling, dbt best practices, and designing schemas that scale with business questions

• Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience

• Experience with infrastructure-as-code (Terraform), CI/CD pipelines, and modern data stack tooling (Fivetran, dbt, Monte Carlo)

• Bonus: experience with streaming architectures (Kafka, Kinesis), data platform migrations, or building semantic/metrics layers; contributions to open-source data tooling