Spark Insights - Big Data Engineer
Boston, MA

Job Description

Overview

Spark Insights is an early stage stealth mode company based in Boston on a mission to develop a broad range of solutions for the insurance industry using big data, advanced modeling, machine learning, and artificial intelligence.

We are looking for an experienced Big Data Engineer to join the Spark Insights team. As a Big Data Engineer, you will be implementing scalable data platforms & solutions to ingest, process and expose computer vision related data to and from different data sources and implement data driven models to generate insights into this data, coordinating R&D batch-oriented data needs as well as production data-based web services that will have an impact in the financial services and insurance industries.

Responsibilities

As a key role in Spark Insights Big data platform, you will:

* Collaborate with team members on developing computer vision systems starting from prototype to production.
* Research, design, implement, and deploy full-stack scalable data platform to support both machine learning scientists for R&D and product designers for production needs.
* Build a strategy to re-invent systems and tools to create a continuous cycle of Innovation
* Create data monitoring models for each product and work with R&D team and product design team to create models ahead of new releases
* Ability to build data models supporting complex transformation
* Identifying and ingesting new data sources and performing feature engineering for integration into models

Qualifications

* 5+ years of experience in programming in Java, Scala, or Python.
* 5+ years of working in Linux environment, ability to interface with the OS using system tools, scripting languages, integration frameworks, etc.
* 2+ years of hands-on experience in implementing batch and real-time Big Data integration frameworks and/or applications, in private or public cloud, preferably AWS, using various technologies (Hadoop, Spark,, etc)
* Experienced in debugging, identifying performance bottlenecks and fine-tuning those frameworks.
* Excellent written and verbal communication skills and ability to communicate effectively to both technical and non-technical audiences.
* Technical fluency; comfortable understanding and discussing architectural concepts and algorithms, assessing tradeoffs and new opportunities with technical team members.
* Bachelor's degree or higher in a technology related field (e.g. Engineering, Computer Science, etc.) required.

Desired Skills:

* Working knowledge and hands-on experience for database systems
* Knowledge in computer vision, CNN, image classification, semantic segmentation.
* Experience with machine learning / deep learning tools or frameworks like Scikit learn, XGBoost, Spark, Tensorflow, Keras, or PyTorch.
* You must be authorized to work in the US.

Spark Insights is committed to providing equal opportunity for all employees and applicants without regard to race, color, religion, sex, sexual preference/orientation, gender identity or expression, age, marital status, national origin, physical or mental disability, veteran status, or any other protected classification under applicable law.