Sr. Data Engineer
Seattle, WA

Job Description

Overview

Beyondsoft Consulting, Inc., is a leading, technical solutions and consulting partner. We combine emerging technologies and proven methodologies to tailor elegant solutions that solve complex challenges and empower our customers to accelerate their business goals. Our services include end-to-end support for cloud, digital, data analytics, multi-language translation, and testing.

Our client is growing their Data Engineering team within a demanding and well recognized enterprise and information technology company. This role will be the core solution of the Strategic Analytics organization, ensuring both the reliability and applicability of the team's data products to the organization. This individual will have extensive experience with ETL design, coding, and testing patterns as well as engineering software platforms and large-scale data infrastructures. The Data Engineers will have the capability to architect highly scalable end-to-end pipeline using different open source tools, including building and operationalizing high-performance algorithms. Proven experience with technologies to solve big data problems with expert knowledge in programming languages like Java, Python, Linux, PHP, Hive, Impala, and Spark.

Responsibilities

Responsibilities:

* Translate complex functional and technical requirements into detailed design
* Hadoop technical development and implementation
* Loading from disparate data sets by leveraging various big data technology e.g. Kafka
* Pre-processing using Hive, Impala, Spark, and Pig
* Design and implement data modeling
* Maintain security and data privacy in an environment secured using Kerberos and LDAP
* High-speed querying using in-memory technologies such as Spark
* Following and contributing best engineering practice for source control, release management, deployment etc
* Production support, job scheduling/monitoring, ETL data quality, data freshness reporting

Qualifications

Qualifications:

* 5-8 years of Python or Java/J2EE development experience
* 3+ years of demonstrated technical proficiency with Hadoop and big data projects
* 5-8 years of demonstrated experience and success in data modeling
* Fluent in writing shell scripts [bash, korn]
* Writing high-performance, reliable and maintainable code
* Ability to write MapReduce jobs
* Knowledge of database structures, theories, principles, and practices
* Understand how to develop code in an environment secured using a local KDC and OpenLDAP
* Familiarity with and implementation knowledge of loading data using Sqoop
* Knowledge and ability to implement workflow/schedulers within Oozie
* Experience working with AWS components [EC2, S3, SNS, SQS]
* Analytical and problem-solving skills, applied to Big Data domain
* Proven understanding and hands on experience with Hadoop, Hive, Pig, Impala, and Spark
* Aptitude in multi-threading and concurrency concepts
* M.S. in Computer Science or Engineering

We are an equal opportunity employer and value diversity at our company. We do not discriminate based on race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability