Job Directory Big Data Cloud Engineer

Big Data Cloud Engineer
Herndon, VA

Companies like
are looking for tech talent like you.

On Hired, employers apply to you with up-front salaries.
Sign up to start matching for free.

About

Job Description

The Company

Hitachi Vantara, a wholly owned subsidiary of Hitachi, Ltd., helps data-driven leaders use the value in their data to innovate intelligently and reach outcomes that matter for business and society - what we call a double bottom line. Only Hitachi Vantara combines 100+ years of experience in operational technology (OT) and 60+ years in IT to unlock the power of data from your business, your people and your machines. We help enterprises store, enrich, activate and monetize data for better customer experiences, new revenue streams and lower business costs. Hitachi Vantara elevates your innovation advantage by combining IT, operational technology (OT) and domain expertise. Come join our team and our employee‐focused culture, and help drive our customers' data to meaningful customer outcomes.

As part of the Hitachi Vantara family, REAN Cloud, a major player in the AWS Partner Network as a Premier Consulting Partner is growing substantially. REAN Cloud works with enterprise level organizations on some of the world's largest automation and cloud deployment projects, and get to work with the best technologies throughout the process! At REAN, we believe in building high performance teams who are passionate about what they do. We enjoy learning from each other and our clients, and our spirit of collaboration permeates everything we do.

The Role

We are seeking a highly motivated, hands-on and talented Data Engineer to join our Big Data and Application Modernization team. The individual will be responsible for working closely with our team of architects and engineers for engineering efficient and cost effective Data Solutions, primarily in multi cloud environments but also in hybrid cloud environments. As a Data Engineer you will be responsible for building repeatable Data Lake solutions and services for different industries, analyzing and processing large amounts of data structured, unstructured and semi-structured. As a Data Engineer, you will be mostly working on agile projects in small to large scrum teams on different projects for various clients.

Responsibilities

* Develop and deploy fully automated and scalable Big Data Solutions including ingesting, transforming and persisting the data from various data sources including but not limited to SQL, NoSQL, NAS etc
* Develop ETL and ELT scripts using standard SQL or tools such as Pentaho, Informatica, Talend etc.
* Build large scale data architectures using Kafka, Kinesis, Spark, Flink and Cassandra in hybrid environments
* Develop Hadoop, MapReduce and/or Amazon EMR workloads for processing large volumes of data
* Develop data pipelines for orchestrating data movements between different stages of data lifecyle using Airflow, Luigi and AWS Data Pipeline
* Develop scripts for processing and persisting data to Graph Databases such as Neo4J, Janus Graph, Amazon Neptune etc
* Implement data cataloging, metadata, data quality checks, Master Data Management (MDM) in data lake architectures
* Build data discovery features by leveraging search technologies such as ELK, SOLR or Splunk
* Develop Business Intelligence and visualization dashboards using Tableau, YellowFin, Birst, Looker or QuickSight
* Develop unit tests wherever applicable
* Participate and present design and solutions in Architecture Review Board Meetings
* Participate in code review and peer-review meetings

Qualifications

* Over 5+ years of experience with at least 2 years in developing and solving Big Data solutions/problems
* BS/MS in Computer Science or equivalent field of study
* Hands on experience in writing SQL queries, stored procedures, functions etc
* Hands on experience in MapReduce, Spark, ETL
* Hands on experience in Python, Java or .NET programming languages
* Experience with as many tools as possible in Hadoop eco system and platforms (Apache, Hadoop, Falcon, Atlas, Tez, Sqoop, Flume, Kafka, Pig, Hive, HBase, Accumulo, Storm, Solr, Spark, Ranger, Knox, Ambari, ZooKeeper, Oozie, Phoenix, NiFi, Nifi Registry, HAWQ, Zeppelin, Slider, Mahout, MapReduce, HDFS, YARN
* AWS experience and certifications preferred
* Knowledge on working with Graph DB tools (Gremlin, SparkQL, Tinkerpop)
* Hands on experience working with Relational Databases (MSSQL, MySQL, PostgreSQL)
* Knowledge on working with data warehousing solutions (Oracle, SQL Server, Redshift etc)
* Have good analytical, communication and interpersonal skills
* Experience working in Agile environments and projects

We are an equal opportunity employer. All applicants will be considered for employment without attention to age, race, color, religion, sex, sexual orientation, gender identity, national origin, veteran or disability status.

Let your dream job find you.

Sign up to start matching with top companies. It’s fast and free.