Software Development Engineer - Big Data, AWS Elastic MapReduce (EMR) Amazon
Seattle, WA

Job Description

Want to change the world with Big Data and Analytics? Come join us on the Amazon EMR team in Amazon Web Services!

Amazon EMR is a web service which enables customers to run massive clusters with distributed big data frameworks like Apache Hadoop, Hive, Tez, Flink, Spark, Presto, HBase and more, with the ability to effortlessly scale up and down as needed. We run large number of customer clusters, enabling processing on vast datasets.

We are developing innovative new features including our next-generation cluster management system, improvements for real-time processing of big data, and ways to enable customers to more easily interact with their data. We're looking for top engineers to build them from the ground up.

Here are sample features that we have recently delivered:

* Authenticate with Kerberos: https://aws.amazon.com/about-aws/whats-new/2017/11/now-enable-kerberos-authentication-and-emrfs-authorization-in-amazon-emr/
* EMR Per Second Billing: https://aws.amazon.com/about-aws/whats-new/2017/10/amazon-emr-now-supports-per-second-billing/
* Spark Application History in EMR Console: https://aws.amazon.com/about-aws/whats-new/2017/09/now-view-apache-spark-application-history-and-yarn-application-status-in-the-amazon-emr-console/
* Custom AMIs: https://aws.amazon.com/blogs/big-data/create-custom-amis-and-push-updates-to-a-running-amazon-emr-cluster-using-amazon-ec2-systems-manager/
* EMR Instance Fleets: https://aws.amazon.com/blogs/aws/new-amazon-emr-instance-fleets/
* Auto Scaling EMR Clusters: https://aws.amazon.com/blogs/big-data/dynamically-scale-applications-on-amazon-emr-with-auto-scaling/
* EMR Security Configurations: https://aws.amazon.com/blogs/big-data/encrypt-data-at-rest-and-in-flight-on-amazon-emr-with-security-configurations/
* Responding To State Changes with AWS CloudWatch Events: https://aws.amazon.com/blogs/big-data/respond-to-state-changes-on-amazon-emr-clusters-with-amazon-cloudwatch-events/

This is a hands-on position where you will be do everything from designing and building extremely stable components and cutting-edge features for the savviest customers in the business to help them get the best results.

You will have a chance to work with the open source community and contribute significant portions its software to open source projects possibly including Apache Hadoop, Spark, Pig and Hbase. You need to not only be a top software developer with excellent programming skills, an understanding of big data and parallelization, and a stellar record of delivery but also excel at leadership and customer obsession and have a real passion for massive-scale computing. If you want to truly test your mettle against the hardest challenges in distributed systems to build solutions for large scale problems in a wide variety of domains, come join our group.

Your responsibilities will include:

* Translation of complex functional and technical requirements into detailed architecture and design
* Deliver systems and features with top-notch quality, on time
* Develop new technologies for monitoring production clusters
* Own the software development process end-to-end, including: working with engineers and product managers to develop requirements; designing, architecting, planning, implementing, and testing new systems and features; deploying, and operating the production EMR systems.

In joining our team, you will get to work with a minimum of technical supervision, while playing a variety of roles as needed to respond efficiently to multiple program priorities. You will get to collaborate with some of the best and brightest minds in the industry. You'll enjoy a competitive salary, great benefits, a creative and agile work environment, and the exciting opportunity to be part of a fast-paced and growing team and one of the most innovative technology companies - but most of all, you will get the satisfaction of making products that millions use everyday to great effect!

For more information:

* AWS re:Invent 2017 Amazon EMR: https://www.youtube.com/watch?v=1CAWf9VDgFM
* AWS re:Invent 2017 Keynote - Andy Jassy: https://www.youtube.com/watch?v=1IxDLeFQKPk
* AWS Big Data Blog: https://aws.amazon.com/blogs/big-data/
* AWS EMR: https://aws.amazon.com/emr/