Backend Software Developer
San Mateo, CA

Job Description

A bit about Scalyr

Scalyr's mission is to build the best tool for engineers to understand their operational systems. Our founder, Steve Newman, cofounded Writely (aka Google Docs). Frustrated by the fact that visibility tools - even Google's in-house tools - weren't keeping up, Steve started Scalyr to create a better solution. It's lightning fast, feature-rich and customers love it. The opportunity in front of us is huge and we are still in the very early days. This is going to be one of those companies where people will look back and say "I wish I'd been there when…" well, this is your chance to be part of "when".

What the Distributed Systems team does

Scalyr's business revolves around ingesting a firehose of log data from our customers and making that data available for real-time search. The Distributed Systems team is responsible for the custom distributed system and storage system powering it all. Most companies use databases, we built ours to gain a competitive advantage. We are proud that many customers choose Scalyr because our search is orders of magnitude faster than our competitors.

And we have only gotten started. We've built a lot, and we need you to help us go farther and faster. Here's some of the projects the team has worked on in the last few months:

"I was tasked with adding encryption at rest for all customer data, and at first I knew almost nothing about the database that I was supposed to work on. It's amazing how much I've learned during this project. One of the best parts was performance tweaking, I'm still happy for what I've came up with using minimal memory copies/allocation."

* Chengming

"We rebuilt Scalyr's log storage engine on S3 to improve the efficiency and operability of our platform. We solved many interesting performance and scalability challenges to enable storing and querying gigabytes of data per node in seconds. This new architecture also enables us to converge towards more stateless machines. The end result is a more cost-effective and scalable solution."

* Jim & Oliver

"I worked with our founder, Steve Newman, to design a novel scatter-gather algorithm for query distribution that dynamically lets fast nodes compensate for slow nodes without any preconceived server weighting. We call it QueryMaster, and internally the work assignment logic is known as 'boustrophedon races'. At some point there will be an awesome blog post explaining just what that is supposed to mean..."

* John

Your skills

* You build high-scale distributed systems.
* You love to dogfood the product you work on. We are heavy users of our own product; it's fun!
* You want to work on high-volume analytics, but not with an end goal of serving ads.
* Can thread the needle between "our business logic dictates X" and "external clients will want Y" and come up with a minimal API that supports both
* You are looking for a challenge.

Bonus points

* You have delivered elegant distributed solutions to complex problems, at scale
* You can identify relevant algorithms in the literature, and bring them into production when they fit.
* You write functional Java, and prefer composition to inheritance

Here are some additional blog posts with more detail about what we've been up to:

Built for Speed: Custom Parser for Regex at Scale

Searching 1TB/sec: Systems Engineering Before Algorithms

Impossible Engineering Problems Often Aren't

Tonight We Monitor, For Tomorrow, We Test in Production!

We believe in showcasing transparency in our process. Check out the below blog posts that dive into the greater detail of your interview:

Scalyr Distributed Systems Engineer Hiring Overview

Being Transparent in Our Hiring Process

Your benefits:

* Competitive salary & equity at a fast growing startup
* Fully funded comprehensive medical, dental, and vision coverage
* 401k
* Free lunch
* Flexible schedule
* Unlimited vacations
* Paid parental leave
* Easygoing, low-stress, considerate and fun office environment
* Whatever equipment you need to get the job done

Our commitment to diversity

We are an equal opportunity employer and value diversity at our company. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.