Lead Big Data Engineer
New York, NY

Job Description

Business Unit:

FreeWheel, A Comcast Company, comprised of FreeWheel Publishers, FreeWheel Markets, and FreeWheel Advertisers empowers all segments of The New TV Ecosystem. We are structured to provide the full breadth of solutions the advertising industry needs to achieve their goals. We power the technology, data enablement, and convergent marketplaces required to ensure buyers and sellers can transact across all screens, across all data types, and all sales channels, in order to ensure the ultimate goal results for marketers. With offices in New York, San Francisco, Chicago, London, Paris, Beijing, and across the globe, FreeWheel, A Comcast Company, stands to advocate for the entire industry through the FreeWheel Council for Premium Video.

FreeWheel is hiring a New York based Lead Big Data Developer reporting to Director, Engineering.

FreeWheel helps the television industry generate revenue from their premium content through a robust technology platform, which enables consumers to watch entertainment on any devices from PC, mobile, set-top box to traditional TV.

New York Data Engineering Team at FreeWheel is part of a global engineering group that is building advanced and high quality solutions to process terabytes of raw set top box data on state of the art big data platform, and make it accessible in a secure and cost effective manner for other teams.

Responsibilities

Develop software to run on cloud native big data infrastructure built on AWS using Spark, Lambda, S3, and other cloud native services.

Design and implement high-performance, scalable and optimized data solutions.

Write applications to derive insights from terabytes of highly valuable data sets.

Develop quick proof of concepts for marketing teams and clients with big data needs.

Manage and improve existing code base and add new features to support business needs

Conduct design and code reviews

Work in a collaborative manner with a smart, diverse and fast-growing New York based Data Engineering scaled agile (SAFe) teams to rapidly deliver high quality solutions.

Interface with global engineering, product and operation teams in US/EU/China to incorporate their innovations and vice versa

Ability to mentor and provide direction in architecture and design to developers

Monitors industry trends and directions; develops and presents substantive technical recommendations to senior management

Be in tune with emerging trends Big data and cloud technologies and participate in evaluation of new technologies to improve our data processing architecture to improve our data processing architecture to improve our data processing architecture

About You

Bachelor's or master's degree in Computer Science or similar field of study

5+ years of experience building large scale big data applications

Proficient with SQL and database general concepts

Hands-on experiences with data processing are strongly preferred.

Expertise with at least one of the following languages Golang, Scala, Python, and or Java. You are flexible with languages and tools and are willing to learn whatever is necessary to get the job done.

Experience with CI/CD tools like Jenkins.

AWS certification is preferable: AWS Developer/Architect/DevOps/Big Data

Experience in AWS technologies such as Lambda, Kinesis, EC2, IAM, CloudFormation, Route 53, ELB, VPC, Cloudwatch, Lambda, EMR, S3 and Big data related technologies like HIVE, Presto, Kafka is strongly preferred.

Experience with bash, Linux utilities and environments.

Understanding of various columnar file formats like Parquet, ORC etc.

Strong problem-solver and analysis skills, and a team player. Have experience successfully introducing new patterns and frameworks.

Must be able to quickly understand technical and business requirements and can translate them into technical implementations

Solid understanding of secure application development methodologies

Excellent written and verbal communication skills. Ability to communicate well in a team.

Comcast is an EOE/Veterans/Disabled/LGBT employer