Data Engineer — New York, NY
Зарплата не указана
ОткликнутьсяAbout The Role
The role owns the data infrastructure that powers analytics and product decisions - batch and streaming pipelines that move billions of events daily into a warehouse serving finance, product, and growth teams.
The team operates at a scale where pipeline reliability, data freshness, and cost efficiency all matter equally, and this role sits at the center of that: designing ingestion systems, modeling warehouse data, and partnering directly with analysts and backend engineers.
Key Responsibilities
• Design, build, and maintain scalable ETL/ELT pipelines in SN4waAF and dbt, moving data from Kafka, Postgres, and third-party APIs into Snowflake
• Model warehouse data using dimensional modeling and dbt best practices, ensuring clean, testable, well-documented marts for downstream analytics
• Own data quality end-to-end: build validation and anomaly detection checks, set SLAs for pipeline freshness, and drive incident response when things break
• Optimize Spark and SQL jobs for performance and cost, tuning workloads running against petabyte-scale datasets
• Partner with data analysts and product teams to translate ambiguous business questions into robust data models and self-serve datasets
• Build and maintain streaming pipelines using Kafka and Flink (or Spark Structured Streaming) for real-time use cases
• Contribute to data platform tooling: CI/CD for pipelines, schema management, and observability dashboards
What We Are Looking For
• 3–7 years of experience in data engineering, analytics engineering, or backend engineering with heavy data focus
• Expert-level SQL and strong Python; comfort writing production-grade code with tests and code reviews
• Hands-on experience with a modern warehouse (Snowflake, BigQuery, or Redshift) and orchestration tooling (SN4waAF, Dagster, or Prefect)
• Practical experience with dbt and dimensional data modeling; understanding of incremental models, snapshots, and data lineage
• Experience with streaming or event-driven architectures (Kafka, Kinesis, or equivalent) at scale
• Bachelor's degree in Computer Science, Engineering, or equivalent practical experience
• Bonus: experience with Flink or Kafka Streams, data quality tooling (Great Expectations, Monte Carlo), Terraform/IaC, or building metrics layers and semantic models