EDL Big Data Engineer - Corporate- Information Technology (No Agencies) Jefferies
New York, NY

Job Description

Position:

We are looking for an accomplished big data developer with strong experience in the cloud AWS data implementation to help us build and integrate data-driven intelligent cloud solutions for EDL. This role will involve a close collaboration with our team of passionate and innovative big data specialists, application developers and product managers.

This is a unique opportunity to be a member of our corporate CRM and Analytics Team, tackling our toughest and most exiting data lake challenges across multiple divisions in Jefferies.

CRM & Analytics Team Overview:

The CRM & Analytics team is a highly strategic and cross-functional team responsible for leading the firm's global digitalization effort. This initiative, spanning all client-facing business units and corporate functions, will drive innovation and strategic change through technology, data science, and deep analytics. The team partners with key business leaders and industry experts to build transformational technology to drive revenue, maximize efficiency, and optimize the allocation of resources. The CRM & Analytics team is at the forefront of Jefferies' cloud initiative, leveraging best-in-class cloud-based technologies to replace legacy on-premises solutions to provide intelligent trend insights, actionable opportunities, decision support, and transparency into all client and business-related activities. This team is also responsible for Enterprise Data Lake.

Position Overview:

We are looking for an accomplished big data architect with strong experience in the cloud AWS data architecture and implementation.

This role will involve a close collaboration with our team of passionate and innovative big data specialists, application developers and product managers.

This is a unique opportunity to be a member of our corporate CRM and Analytics Team, tackling our toughest and most exiting data lake challenges across multiple divisions in Jefferies.

Enterprise Data Lake (EDL) Team Responsibilities:

The EDL team will oversee and support architecture and implementation of EDL for all Jefferies big data initiatives. It will drive the data governance and facilitate data onboarding. It will approve the design of data and software architecture, perform architecture review to pass EDL tollgates, evaluate and select cloud/AWS/Big Data tools for acceptance, and serve as a vendor liaison with data lake tool vendors and out internal infrastructure teams. Also, it will certify data for consumption, EDL patterns and processes, manage and govern data access controls, and will manage data lake and data governance training initiatives across enterprise.

The EDL team will become the center of excellence for the following EDL components and associated tools:

q EDL architecture and patterns

q EDL Data Stores

q EDL Governance

q EDL Data Discovery

q EDL Data Preparation

q EDL Reporting

q EDL Ingestion tools & other technologies and tools

q Educate teams to migrate and develop new cloud applications

Basic Requirements:

* A minimum of 5 years of hands-on technical experience with:
* big data implementation and technology offerings
* AWS/cloud big data modeling & data management
* analytics and ingestion architecture of big data
* data lake management and data architecture
* data lake design patterns & cloud best enterprise practices
* IoT and streaming, real time processing
* Big data related AWS technologies
* Experience in AWS technologies such as Kinesis, Lambda, EC2, Redshift, RDS, Cloud formation, EMR, AWS S3, AWS Analytics, Spark, Databricks
* Experience with at least one of the following languages Scala, Python, R and or Java
* Experience with designing, developing, and implementing complex integration for end-to-end solutions at a middleware and app level with focus on performance optimization
* Strong implementation skill in area of cloud development in AWS
* Demonstrated ability in implementing cloud scalable, real time and high-performance data lake solutions (AWS)
* Ability to quickly perform proof-of-concepts for validating new technology or approach
* Ability to exercise independent judgment and creative problem-solving techniques in a highly complex environment using leading-edge technology and/or integrating with diverse application systems
* Ability to lead and drive technology change in a fast-paced, dynamic environment and all phases of the entire software life cycle
* Strong experience with data catalog, data governance, Collibra, MDM and/or Data Quality (IDQ) toolset
* Strong experience with integration of diverse data sources (batch and real time) in the cloud
* Lead the design and sustainment of data pipelines and data storage
* Expertise in Structured, unstructured, SQL and No-SQL technologies
* Expertise with identifying and understanding source data systems and mapping source system attributes to the target
* Experience with design and automation of ETL\ELT processes
* AWS and cloud performance tuning and optimization experience
* Experience with effort estimation for new projects/proposals on an ongoing basis.
* Excellent communication skills across all levels; ability to communicate with ease the complex and technical concepts.
* Ability to work effectively in a fast-paced environment