Principal Software Engineer - Data Engineering
San Francisco, CA

Job Description

Job Description

Shipt is a membership-based marketplace that helps people get the things they need. Our friendly shoppers handpick fresh groceries and household essentials and deliver them to members in as soon as one hour.

Shipt has a wide variety of data partners and as our data volumes increase dramatically in a national rollout, we're looking to grow the Data Framework and Tools engineering group within Data Engineering to help expand and develop our large-scale data processing platform and tools.

Data Engineering at Shipt primarily focuses on retailer catalog and general product data for e-commerce purposes. You'll focus on developing pipeline frameworks, services, and tools that can scale to match the company's growth trajectory.

Your Responsibilities

* Develop and scale our data processing platform and services so that we can quickly and reliably process large amounts of data
* Help evolve our data model for new retailers and new retail verticals
* Work with other teams in the organization (e.g., Engineering, Catalog) to build tools and solutions that support and help manage data within the Shipt ecosystem
* Keep the big picture in mind so that our architectural patterns can better consume and validate source data
* Build and experiment with different tools and tech, and share your learnings with the broader organization

Requirements

* Candidate should have at least 5+ years of relevant work experience post college. Expert-level proficiency ( 5 - 7 years) in one or more of the following domains:
* Python programming & Ruby on Rails. Experience with Go is a plus.
* Good Systems design and architecture skills.
* Lead design reviews with other senior technologists across the company
* Building and managing ETL pipelines, data frameworks.
* Experience developing micro services that are deployed on Redis, AWS is required.
* Experience working with Elastic Search/Algolia is a plus.
* Experience with relational (SQL) and non-relational Databases.
* Bachelor's degree/ Masters Degree in Computer Science or similar technical field.
* Familiarity with Git, Jenkins. Contributions to any open source projects is a plus.
* Project Management Skills and experience leading engineers and projects in a previous role.
* Architect high performance batch and near real-time data pipelines
* Excellent written and verbal communication skills; able to effectively collaborate with diverse teams.



Nice To Haves

* Masters's Degree in a technical field or equivalent work experience.

We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.

Pursuant to the San Francisco Fair Chance Ordinance, we will consider for employment qualified applicants with arrest and conviction records.