Senior Big Data (Spark) Engineer
Cambridge, MA

Job Description

1 Broadway (21026), United States of America, Cambridge, Massachusetts

At Capital One, we're building a leading information-based technology company. Still founder-led by Chairman and Chief Executive Officer Richard Fairbank, Capital One is on a mission to help our customers succeed by bringing ingenuity, simplicity, and humanity to banking. We measure our efforts by the success our customers enjoy and the advocacy they exhibit. We are succeeding because they are succeeding.

Guided by our shared values, we thrive in an environment where collaboration and openness are valued. We believe that innovation is powered by perspective and that teamwork and respect for each other lead to superior results. We elevate each other and obsess about doing the right thing. Our associates serve with humility and a deep respect for their responsibility in helping our customers achieve their goals and realize their dreams. Together, we are on a quest to change banking for good.

Senior Big Data (Spark) Engineer

As a Capital One Engineer, you will be part of an Agile or platform team where you will participate in detailed technical design, development and implementation of applications using existing and emerging technology platforms. Working within an Agile environment, you will provide input into architectural design decisions, develop code to meet story acceptance criteria, and ensure that the applications we build are always available to our customers.

You will have the opportunity to develop your technical knowledge and skills to keep your mind and our business on the cutting edge of technology.

Our mission is to develop products and technology that serves the needs of One Data Producers and Consumers. What you will bring to the table is more than just your technical skills -- your unique perspective, ideas, and cultures will help us create better products and services.

What you'll do:

* Work closely with team to design and develop features that will empower One Data users
* Write server-side code for Microservices/Cloud Applications and develop prototypes quickly
* Measure and improve application performance and scalability
* Create Scala/Spark jobs for data transformation and aggregation
* Produce unit tests for Spark transformations and helper methods
* Write Scala doc-style documentation with all code
* Design data processing pipelines
* Write a well-tested code that is resilient to heavy usage and iteration.
* Manage project priorities, deadlines and deliverables.

What we are looking for:

* A solid foundation in data structures, algorithms and software design with strong analytical and debugging skills.
* Strong programming skills at least one of the following languages: Scala, Python, Java, GO.
* Knowledge of open API specification(Swagger) and Strong understanding of Web APIs / HTTP / REST / JSON.
* Experience with design, development, deployment, versioning, and maintenance of secure RESTful APIs.
* Good understanding of public cloud services platform like AWS.
* Experience on Big Data and Apache Spark.
* Experience working with HDFS, S3, DynamoDB.
* Deep understanding of distributed systems (e.g. CAP theorem, partitioning, replication, consistency, and consensus).
* Ability to work independently and in groups.

Basic Qualification:

* Bachelor's Degree or military experience
* At least 3 years of Python or Java or Scala programming
* At least 1 year of public cloud services platform like AWS
* At least 2 years of experience on Big Data and Apache Spark

Preferred Qualifications:

* 5+ years' experience in software development with Python, or Java or Scala
* 5+ years' experience in Agile practices
* 4+ years' experience in Big Data, Apache Spark and Hadoop.
* 3+ years' experience working with cloud-based infrastructure in AWS or Azure or Google Cloud
* 3+ years' experience building with platform-as-code
* 3+ years' experience with Open source projects and/or contributions
* 3+ years' experience with GitHub, Artifactory, Jenkins, CircleCI

At this time, Capital One will not sponsor a new applicant for employment authorization for this position.