Manager Data Engineering
New York, NY

Job Description

Posting Job Description

OVERALL SUMMARY

The Data and Engineering Services team within the Media Technology & Information Services (MTIS) group is looking to add a Data Engineering Manager to lead the architecture, design and implementation of projects focused on collecting, aggregating, storing, reconciling and making data accessible from disparate sources to enable analysis and decision making. As an independent contributor, this individual will collaborate within a team of technologists to produce enterprise scale solutions to meet our clients' needs.

In this role, responsibilities will touch on all stages of the software engineering lifecycle: understanding customer requirements, designing the application, reviewing code, and monitoring the system for issues. There will also be an opportunity to exercise highly developed collaboration abilities interacting with other technology teams on broader cross-organization initiatives.

PRIMARY RESPONSIBILITIES

* Be a thought leader across the Data Engineering & Services organization to build new and improved data tools and services that can scale with the company.


* Take loosely defined business questions and translate them into clearly defined technical/data specifications for implementation.


* Write complex ETL (Extract / Transform / Load) processes, design and develop solutions for real-time and offline analytic processing on AWS-hosted Data and Analytics Platform.


* Ensure that data pipelines are scalable, repeatable and secure.


* Deliver quality work on defined tasks with little oversight and direction.


* Ensure all deliverables are of high quality throughout the project by adhering to coding standards and best practices and participating in code reviews.


* Implement activities that generally impact multiple components / processes and the work of own and possibly other units / teams / projects.


* Interacts with internal and external peers and management to share highly complex information related to areas of expertise and/or to gain acceptance of new or enhanced technology / business solutions.


* Mentor less experienced members of the team.



REQUIREMENTS

* 6+ years of experience designing and implementing large-scale, complex, data-driven applications on the cloud, preferably on AWS.


* 6+ years of hands-on programming experience with Python, frameworks (Spark and Airflow) and streaming technologies (Kafka/Kinesis)


* 6+ years of hands-on experience using SQL to perform complex data manipulation


* 3+ years of experience modeling data warehouses (Kimball dimensional modeling)


* An understanding of technologies and design patterns in fields such as: microservices, streaming/queuing systems, key-value stores and high-performance solutions (vectorization, task and data parallelism)


* Ability to deal with ambiguity and work with rapidly-changing business data


* Appreciation of the need and ability to design for flexible systems that adapt to evolving user requirements


* Must have strong communication skills (written and verbal)



NICE TO HAVE (BONUS)

* 1+ year of experience working on Snowflake Cloud Datawarehouse on AWS


* 1+ year of experience with graph database technologies (e.g., Neo4j, TigerGraph)


* 1+ year of experience building Master Data Management (MDM) solutions


* 1+ year of hands-on experience with R, Scala, Hive and/or Pig



PREFERRED QUALIFICATIONS

* BS/MS/PhD in Computer Science or related field or equivalent experience.