Data Infrastructure Engineer - Algorithms Platform Partners Data Systems
San Francisco, CA

Job Description

About the team

In this role you'll be a member of the Scalable Infrastructure team which is part of the Algorithms Platform. This team provides frameworks and services to access and operate on our data, including Spark, Presto, and custom tools. This team also handles initial data ingestion: our data initially comes from our Kafka logging pipeline along with regular snapshots of transactional databases. Our ETL framework along with tools to track and monitor jobs helps to increase reliability while making it easier for data scientists to obtain and manipulate data.

About the role

In this role you'll be contributing heavily to our Spark infrastructure, creating and improving services to make it easier for the team to submit and monitor jobs and providing established patterns for data scientists to create their jobs. Occasionally you'll help a team member design or debug a more complex Spark job or perhaps a pipeline of multiple jobs. The interesting thing you'll find about our department is the diversity of applications we support, the varied sizes of our jobs, and the different ways that we use Spark. This role will be exciting for you if you enjoy building and monitoring infrastructure along with digging into complex tools and making them easier for others to use.

Some typical projects you might work on include (but aren't limited to!):

* You'll help us improve our Spark, Presto, and custom service deployments to function well under load and in AWS. You'll extend these services and create new ones to help make the experience better for our data scientists.
* We build our own versions of Spark along with custom libraries included in each Spark job, so you will contribute to our Spark/Presto customization efforts, builds, and deployments.
* You'll help us utilize various file formats (e.g. Parquet), and help create readers and writers that function well on S3 and with our metadata services.
* You'll build services to ingest data into our warehouse and ensure it's clean and consistent.
* Many of the changes we need would also benefit others in the big data community. You'll have the opportunity to contribute back.

We get excited about candidates who have:

* 5+ years of software development experience with significant contributions.
* Exceptional coding and design skills, particularly in Java/Scala.
* Strong distributed systems background, and have worked with Spark and other tools in the Hadoop ecosystem.
* Ability to work autonomously and take ownership of projects.
* Understanding of how big data infrastructure works in the public cloud.
* Natural curiosity and tendency to get excited to dig in and understand how things work.

Where to Apply?

If this role describes you, please apply directly here!

Do you want to work on the algorithms platform but you're not sure if this role is right for you? Not a problem -- apply to our general Algorithms Platform Engineer role instead and we'll sort you out.

About Stitch Fix

At Stitch Fix, we're about personal styling for everybody and we believe in both a service and a workplace where you can be your best, most authentic self. We're the first fashion retailer to combine technology and data science with the human instinct of a Stylist to deliver a deeply personalized shopping experience. This novel juxtaposition attracts a highly diverse group of talented people who are both thinkers and doers. All of this results in a simple, powerful offering to our customers and a successful, growing business serving millions of men, women, and kids. We believe we are only scratching the surface on our opportunity, and we're looking for incredible people like you to help us carry on that trend.