Data Engineer - Experience Mission
Stockholm, AB

Job Description

We are hiring software engineers who are very enthusiastic about data to focus on building structured, high-quality data solutions. These solutions will be used to evolve our products bringing better experiences to our 200+ million users and the global artist community alike. We are processing petabytes of data using tools such as BigQuery, Dataflow and Pub/Sub. When needed, we also develop our own data tooling such as Scio, a Scala API for Apache Beam and Luigi, a Python framework for scheduling.

The Playback tribe in Spotify Experience mission impact the lives of hundreds of millions, every day. Our teams manage the whole audio, video and image pipeline from ingestion and transformation to content delivery and player management in the Spotify clients while gathering billions of data points that can help bringing guidance and clarity to user behaviour, product roadmaps and the strategy for Spotify.

You will be working primarily in 3 areas:

* Supporting squads building aggregates of there raw data for analysis.


* Scale systems for anomaly detection and quality control.


* Productionizing statistical and machine learning models



Initially the distribution of work will be 60% pipeline, 30% scale anomaly detection and 10% ML productionisation, with the expectation to halve the basic pipeline support within a year. You will work in a squad with 2 data scientists with possibility for collaboration with data engineers in our sister tribes. For pipelining you will be working with backend engineers with basic data engineering skills, and one of your prime tasks is to enable them to write and maintain their own pipelines.

What you'll do

* Enable backend engineers to write and maintain their own data pipelines.


* Develop higher level aggregation of dataset to enable fast root case analysis.


* Develop and maintain datasets for anomaly detection and machine learning.


* Automate and scale anomaly detection and quality control metrics.


* Productionize and handover machine learning models to other squads


* Keep up to date with best data engineering practices within Spotify, acting as data engineering bridgehead to the Playback tribe.



Who you are

* You don't like leaving questions unanswered and you love exploring/understanding data


* You are passionate about crafting clean code and have a steady foundation in coding and building data pipelines


* You are interested in being the glue between engineering and analysis


* You want to share your knowledge by pair programming, common projects and lunch and learns.


* You are interested in scalability and quantitative reasoning around computation resources for data processing.


* Ideally you will also find it motivating to improve the playback experience through your support in pipelines construction, anomaly detection and productionization of models.



We are proud to foster a workplace free from discrimination. We truly believe that diversity of experience, perspectives, and background will lead to a better environment for our employees and a better product for our users and our creators. This is something we value deeply and we encourage everyone to comebe a part of changing the way the world listens to music.

Psst! If this job is your perfect match and you want some inside tips before you apply, read this blog post.