Data Engineer
New York, NY

Job Description

The Role:

By market definition, a data engineer is responsible for creating, maintaining and understanding data and the resulting delivery infrastructure. They are the connection between smart business users and not-so-smart data repositories. They are capable (through a solid command of various scripting languages (Python, R, SQL)) of taking any source of data and performing an EVL(ST) Extract, Validate, Load, Standardize, Transform to the correct data store, in the form agreed upon by the Data Engineer and the end-user. Data Engineers are often responsible for the efficacy, quality and elegance of their solutions. They are business savvy and understand the importance of the data they are piping in - to an extent.

Responsibilities

* Contribute to the design and development of our Python data workflow management platform
* Design and develop tools to wrangle datasets of small and large volumes of data into cleaned, normalized, and enriched datasets
* Build and enhance a large, scalable Big Data platform (Spark, Hadoop)
* Refine processes for normalization and performance-tuning analytics

About you

* You love building elegant solutions that scale
* You bring deep experience in the architecture and development of quality backend production systems, specifically in Python
* You love working on high-performing teams, collaborating with team members, and improving our ability to deliver delightful experiences to our clients
* You are excited by the opportunity to solve challenging technical problems, and you find learning about data fascinating
* You understand Server, Network, and Hosting Environments, RESTful and other common APIs, common data distribution, and hosted storage solutions

Must Have

5+ years of full-time experience in a professional environmentExpertise in PythonExperience with ETL and/or other big data processesExperience with at least 2 popular big data / distributed computing frameworks, eg. Spark, Hive, Kafka, Map Reduce, FlinkExperience working independently, or with minimal guidanceStrong problem solving and troubleshooting skills Ability to exercise judgment to make sound decisions Proficiency in multiple programming languagesStrong communications skills, interpersonal skills, and a sense of humor

Even Better

Data skills: RDBMS SQL and NOSQL, structured and unstructured data, BigQueryProficiency in Jupyter, C24; familiarity with ETL, CDC, and workflow toolsExperience working in a cloud-based environment, such as GCP or AWS

About Crux:

Crux is a data delivery and operations company that takes on the critical, yet commoditized, tasks of ingesting data and getting it ready for analysis by financial institutions. We are experiencing rapid growth from financial firms turning to Crux to process and onboard the data they need.

Crux is transforming how the financial services industry works with data. For years, data discovery, operations, and delivery have been expensive, time consuming, and frustrating for financial firms. We are solving this challenge by creating a new technology solution: the go-to platform for data delivery. Crux is in high demand and we have a clear path to success.

Founded in 2017 by a seasoned team of data professionals, Crux has grown to a team of 70 employees in San Francisco and New York. It is backed by leading industry players, including Citi, Goldman Sachs, and Two Sigma.

At Crux, diversity is valued and and treatment of employees and applicants are based on merit, talent and qualification. We encourage people from underrepresented groups to apply. We believe the key to success is bringing together unique perspectives and we do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status. For qualified applicants with criminal histories, consideration will be consistent with the requirements of the San Francisco Fair Chance Ordinance. All your information will be kept confidential according to EEO guidelines.