Senior Software Engineer - Data Zocdoc
New York, NY

Job Description

As a Senior Data Engineer, you'll collaborate with product engineering teams to design and implement data flows and tools to power key strategic decisions, and power machine learning and personalization within the product. You will build and maintain infrastructure for collecting, storing, processing, and analyzing huge sets of data in batch and streaming pipelines.

What you'll do:

* Work closely with the Data Science and Business Intelligence teams to develop data models for research, reporting, and machine learning.
* Build data tooling to enable data lake, data warehouse, and analytics workflows within the AWS cloud (S3, Redshift, DynamoDB, Spark, Kinesis, Kubernetes, etc.)
* Contribute to our in-house ecosystem of developer data tools.
* Collaborate with partners across the company to assess data needs and prioritize accordingly. Be proactive about driving data collection and storage best practices.
* Drive adoption of data tools. Give tech talks and demos on the newest capabilities and enhancements to the system.
* Consult on data security, design, and scalability to product engineering teams

What's required:

* Curiosity and vision. You have a passion for technology and can think critically about performance, scalability, and reliability of software
* 5+ years engineering experience, 2+ years working with data in the cloud, ideally using AWS.
* Well-versed in one or more of the following languages and functional programming in general: Scala, Java, Python, JavaScript
* Expert in SQL and comfortable designing, writing and maintaining complex SQL based ETL.
* Experience with building large-scale batch and real-time data pipelines; ETL design, implementation, and maintenance.
* Experience with schema design and data modeling, and the analytical skills to QA data and identify gaps and inconsistencies.
* Experience supporting Machine Learning or Business Intelligence teams and products
* Computer Science or related degree required