Senior Business Intelligence Data Engineer Databricks
San Francisco, CA

Job Description

Mission

The BI Data Engineer plays a key role in technological decision making for business teams future data, analysis, and reporting needs. The role supports the business's daily operations inclusive of troubleshooting of the business's data intelligence warehouse environment and job monitoring. It is also the role of the Senior Business Intelligence Data Engineer to guide the business in identifying any new data needs and deliver mechanisms for acquiring and reporting such information as well as addressing the actual needs. The Business Intelligence Data Engineer is also tasked with gathering and maintaining best practices that can be adopted in big data stacking and sharing across the business. The Business Intelligence Data Engineer provides expertise to the business in the areas of data analysis, reporting, data warehousing, and business intelligence.

Outcomes

* The Business Intelligence Data Engineer designs and supports the business's database and table schemas for new and existing data sources for the data warehouse. Additionally creates and supports the ETL in order to facilitate the accommodation of data into the warehouse. In this capacity, the Business Intelligence Data Engineer designs and develops systems for the maintenance of the business's data warehouse, ETL processes, and business intelligence.
* The role that the Senior Business Intelligence Data Engineer plays is highly collaborative and, as such, works closely with data analysts, data scientists, and other data consumers within the business in an attempt to gather and populate data warehouse table structure, which is optimized for reporting. The Business Intelligence Data Engineer also works closely with other disciplines/departments and teams across the business in coming up with simple, functional, and elegant solutions that balance data needs across the business
* The Senior Business Intelligence Data Engineer plays an analytical role in quickly and thoroughly analyzing business requirements for reporting and analysis and subsequently translating the emanating results into good technical data designs. In this capacity, the Business Intelligence Data Engineer establishes the documentation of reports, develops, and maintains technical specification documentation for all reports and processes.

Competencies

* 8+ years of experience in a Data Engineer role, who has attained a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field.
* Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
* Experience building and optimizing 'big data' data pipelines with Spark, architectures and data sets. In depth knowledge of Model and Design of DB schemas for read and write performance.
* Extensive working knowledge of API or Stream based data extraction process like Salesforce API and Bulk API is must.
* Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
* Build processes supporting data transformation, data structures, metadata, dependency and workload management.
* A successful history of manipulating, processing and extracting value from large disconnected datasets.
* Working knowledge of message queuing, stream processing, and highly scalable 'big data' data stores.
* Experience with building data pipeline from various business applications like Salesforce, Marketo, NetSuite, Workday etc.
* Experience with big data tools: Hadoop, Spark, Kafka, Spark & Kafka Streaming, Python, Scala, Talend etc.
* Working knowledge of BI Tools like Tableau, Looker etc is plus

Benefits

* Medical, dental, vision
* 401k Retirement Plan
* Unlimited Paid Time Off
* Catered lunch (everyday), snacks, and drinks
* Gym reimbursement
* Employee referral bonus program
* Awesome coworkers
* Maternity and paternity plans

About Databricks

Databricks' mission is to accelerate innovation for its customers by unifying Data Science, Engineering and Business. Founded by the original creators of Apache Spark™, Databricks provides a Unified Analytics Platform for data science teams to collaborate with data engineering and lines of business to build data products. Users achieve faster time-to-value with Databricks by creating analytic workflows that go from ETL and interactive exploration to production. The company also makes it easier for its users to focus on their data by providing a fully managed, scalable, and secure cloud infrastructure that reduces operational complexity and total cost of ownership. Databricks, venture-backed by Andreessen Horowitz, NEA and Battery Ventures, among others, has a global customer base that includes Viacom, Shell, and HP. For more information, visit www.databricks.com.

Apache, Apache Spark and Spark are trademarks of the Apache Software Foundation.