Senior Data Engineer
Los Angeles, CA

Job Description

TWO NIL is a growth consultancy that uses a holistic approach to create profitable go to market campaigns - at scale. We provide clients with vertically integrated, unbundled solutions across their acquisition marketing needs, from strategy and planning to execution, forecasting, and optimization. Our clients represent best of breed brands across a diverse range of verticals and geographies.

As a Senior Data Engineer, you'll help us build out our core product by designing and implementing high complexity ETL pipelines using PySpark and AWS Glue. You'll be responsible for delivering quality, scalable, and reusable software, and providing input on software architecture, data storage, API and user interface design. You'll be responsible for the design and implementation of serverless microservices using AWS Lamba and API Gateway.

Responsibilities:

* Work closely with Business Intelligence and Data Science teams to understand the data needs, define, scope and plan new product features
* Design, build, deploy new data models and complex ETL pipelines into production with PySpark and AWS Glue
* Design and implement serverless microservices with AWS Lambda, API Gateway and Fargate
* Be accountable for operational efficiency and be proactive in monitoring data pipelines
* Make smart platform and engineering decisions based on data analysis and collaboration
* Deliver quality scalable, efficient and reusable software
* Ensure unit test coverage and promote code quality
* Research new tools and technologies applicable to new and existing use cases
* Identify areas to improve architecture, application design and scalability
* Provide input on software architecture, data storage, API and user interface design
* Mentor more junior team members

Qualifications:

* Degree in Computer Science
* 6 + year's building enterprise software applications
* 5+ years' experience in custom ETL design, implementation, and maintenance
* 3+ years experience with Spark/PySpark or equivalent distributed processing systems
* 3 + years experience with AWS Services (Athena, Glue, Redshift, DynamoDB, ECS, S3) and python.
* Experience with Service Oriented Architecture and Microservices
* Experience in the complete Software Development Life Cycle (SDLC)
* Experience with Agile Software Development (SCRUM)
* Good understanding on relational databases (Postgres, MySql, etc.)
* Good understanding on NoSQL databases (DynamoDB, MongoDB, etc.)
* Familiarity with container orchestration services (ECS, Kubernetes, etc.)
* Familiarity with serverless compute platforms (Lambda, Cloud Functions, etc.)
* Drive to analyze data to identify deliverables, anomalies, and gaps
* Accountable, Curious and Organized
* Experience in data analytics tools such as Tableau a good to have
* Strong communication skills to collaborate with various teams

As a TWO NIL employee, you will enjoy:

* Competitive compensation package
* Unlimited paid time off policy
* Flexible working hours
* Benefits (Health, Dental, Vision, Life Insurance, 401k, Flexible Spending Account and more)
* Fitness reimbursement
* Catered lunches and stocked kitchen with fresh fruit, snacks, premium coffee & tea, and cold brew coffee
* Ongoing learning and classes for employees
* Team events and outings