Senior Software Engineer - Platform Databricks
San Francisco, CA

Job Description

Mission

You will build and extend the Databricks cloud platform, which is based on a micro service architecture and includes systems for managing thousands of Kubernetes clusters at scale, systems for streaming and consuming gigabytes of log data per minute, onboarding and managing thousands of data scientists on Databricks, scalable API gateway, rate limiting framework, network security and encryption, build infrastructure (we use Bazel), and scalable CI/CD framework among many others.

Outcomes

* Develop and extend the Databricks platform. This implies, among others, writing clean, efficient code in Scala or Python and/or interacting with: cloud APIs (e.g., compute APIs, cloud formation, Terraform), with open source and third party APIs and software (e.g., Kubernetes) and with different Databricks services
* Be responsible for the full software development lifecycle - designing, development, testing, and operating scalable systems in production
* Architect solutions to achieve a high level of reliability, scalability and security
* Communicate effectively with other engineers in the same team, with other teams and with various other stakeholders including product managers and compliance teams
* Operate in an Agile and fast moving environment

Competencies

* 5+ years of experience writing production code in one of: Java, Scala, JavaScript, C++ or Python
* Experience architecting, developing, deploying and operating large scale distributed systems at scale
* Experience with cloud APIs (e.g., a public cloud such as AWS, Azure, GCP or an advanced private cloud such as Google, Facebook)
* Experience working on a SaaS platform or with Service Oriented Architectures
* Good knowledge of Linux fundamentals
* Good knowledge of SQL (MySQL preferred)
* Experience with Continuous Integration/Continuous Deployment frameworks
* Experience with Docker and Kubernetes

Benefits

* Medical, dental, vision
* 401k Retirement Plan
* Unlimited Paid Time Off
* Catered lunch (everyday), snacks, and drinks
* Gym reimbursement
* Employee referral bonus program
* Awesome coworkers
* Maternity and paternity plans

About Databricks

Databricks' mission is to accelerate innovation for its customers by unifying Data Science, Engineering and Business. Founded by the original creators of Apache Spark™, Databricks provides a Unified Analytics Platform for data science teams to collaborate with data engineering and lines of business to build data products. Users achieve faster time-to-value with Databricks by creating analytic workflows that go from ETL and interactive exploration to production. The company also makes it easier for its users to focus on their data by providing a fully managed, scalable, and secure cloud infrastructure that reduces operational complexity and total cost of ownership. Databricks, venture-backed by Andreessen Horowitz, NEA and Battery Ventures, among others, has a global customer base that includes Viacom, Shell, and HP. For more information, visit www.databricks.com.

Apache, Apache Spark and Spark are trademarks of the Apache Software Foundation.