Software Engineer - Storage and Messaging Twitter
Seattle, WA

Job Description

Software Engineer - Storage and Messaging

Seattle, WA

Who We Are:

Twitter's globally distributed, real-time communications network generates trillions of events and petabytes of data per day. To handle this scale, As engineers on the Storage & Messaging team, we build world class distributed databases, caching, and messaging infrastructure.

What You'll Do:

You will be responsible for contributing to and helping guide the future of our systems that hold petabytes of data, serve millions of queries per second, and are used by hundreds of millions of Twitter users as they connect, explore, and interact with information and one another. You will develop highly-performant algorithms and data structures, solve complex distributed systems problems, and envision and innovate the next solutions that will bolster the foundation of Storage and Messaging infrastructure.

Operating our own systems at hyper growth levels is a rewarding challenge, it is what helps make us great. We are a tight knit and passionate group that loves working together, and we are looking for exceptional additions to our flock. You will empower dozens of engineering teams, hundreds of co-workers, and millions of users to dream of new insights and new possibilities.

The Storage team is hiring for the following areas:

* Blob Storage - storage for all the photos, videos, and static assets served by Twitter. Store petabytes of data for 100s of millions of users everyday, with a fan out of a single Tweet reaching 100M followers in real time. Find the right balance between cost, latency, and availability to serve Twitters unique use cases. Enable future directions such as ML over Blobs for scenarios such as recommendations, identifying abuse, and anomaly detection.
* Messaging and Coordination - publish and subscribe technologies to enable services and applications to stream information between one another. This team also owns the replicated log which is a foundation technology used to power other teams within Twitter.

https://github.com/twitter/distributedlog https://blog.twitter.com/2015/building-distributedlog-twitter-s-high-performance-replicated-log-service


* Real Time Storage Group - Real-time, multi-tenant distributed database for Twitter scale. Distributed in-memory storage solutions that offer low latency and high throughput at Twitter scale

https://blog.twitter.com/2014/manhattan-our-real-time-multi-tenant-distributed-database-for-twitter-scale


* Graph Storage - services to store and serve high-throughput queries on Twitter's social graph and indexes on our Tweets. This service gets 50 billion queries per day with a peak of 25 million queries per second each day.


* Observability - a comprehensive set of diagnostic tools for Twitter engineers to troubleshoot incidents. Includes a monitoring & alerting stack processing ~5B metrics/minute, MetricsDB (a highly optimized purpose-built time series database) to store metrics, a distributed logging service (handling ~30 PB of data) and a distributed tracing service (Zipkin). https://blog.twitter.com/engineering/en_us/a/2016/observability-at-twitter-technical-overview-part-i.html

Who You Are:

You enjoy new challenges and are passionate about continually pushing the envelope of scale. You take satisfaction in building resilient, high-performance, and thoroughly tested systems that can power the most business-critical applications. You want to learn, work with, and contribute to cutting-edge open-source technologies.

The ideal candidate has experience with distributed storage systems such as Cassandra, HBase, MongoDB or MegaStore, BigTable or experience with messaging technologies such as Kafka, Zookeeper, Memcached or Redis and actively makes contributions to open-source software.

Requirements:

* 2+ years experience with software engineering best practices (e.g. unit testing, code reviews, design documentation).
* BS, MS, or PhD in Computer Science, or equivalent work experience.
* Experience with Java, Scala, C, C++, or another modern systems language.

We are committed to an inclusive and diverse Twitter. Twitter is an equal opportunity employer. We do not discriminate based on race, ethnicity, color, ancestry, national origin, religion, sex, sexual orientation, gender identity, age, disability, veteran status, genetic information, marital status or any other legally protected status.