Sr. Data Engineer Allstate
San Francisco, CA

Job Description

Drift is a new car sharing platform focused on improving mobility and reducing the cost of car ownership. We give car owners a way to earn extra income from their idle cars and connect drivers with a convenient, affordable way to drive a car when they want.

Job Description

As a Sr. Data Engineer, you will be part of a team of engineers working to manage our ever-growing collection of mobility and driver data from across the USA. You'll work primarily with Java and Apache Spark and a variety of data stores including MongoDB, Aurora PostgreSQL, Parquet, and Splunk. You will interact with engineering and product teams to design and optimize the schemas needed to support the products we dream up. Going forward, you'll support our evaluations of new tools and technologies to scale and analyze our data.

Job Qualifications

* 5+ years of experience in software engineering with experience as a senior contributor or team lead
* Good programming skills in Java, Scala, Python, Ruby On Rails
* Expertise in building the data pipeline with distributed and federated systems
* Experience with data visualization tools - Tableau/D3
* Expertise in building services to expose calculated data
* Experience with MongoDB, Aurora PostgreSQL and Data Lakes
* Experience delivering Spark-based data
* Strong background in SQL, Data Modeling, and Performance Tuning in both relational and no-SQL databases
* Experience Operating in AWS
* Experience with Scala or Python / PySpark
* Experience generating and evaluating data quality metrics
* Experience with PII data practices