Senior Data Engineer
Manassas, VA

Job Description

SWIFT provides the platform, messaging, standards, and products & services to over 10.500 customers in 215 countries and territories.

The Data Analytics & Integration team works in conjunction with other teams to deliver an Operations And Support Intelligence System ("OASIS"), providing insight into customer and operational data, using a variety of big data and search technologies. The team is adding a sub-team to integrate existing information flows into OASIS. A key priority for this sub-team in 2019 is to migrate information flows currently consumed by a proprietary CMDB-based visualisation application to OASIS.

Responsibilities

Working with application development teams who write applications which generate logs, and with Operations teams who operate those applications, and through development of an understanding of the business of SWIFT customers, the successful candidate will drive and oversee the following:

* understanding of content and value of logs generated by messaging or service management tools
* creation of syslog-ng configuration to transport logs from generating system into pipeline
* development of parser(s) for the logs, using logstash or custom parser developed in Java
* development of configuration for a generic metrics engine defining fields to be used for calculating metrics of interest e.g. transit times
* development of dashboards for useful visualization of insights found in those logs
* working with development & operations staff, conduct end-to-end tests of the entire pipeline from generating system to dashboard

Qualifications

* Bachelor's degree in IT/Engineering or equivalent
* 3+ years of total experience, including experience with at least three of the following:
* Providing valuable insights from data
* Syslog-ng and its configuration
* Elastic stack
* Java and/or Scala development
* Apache Spark
* Apache Kafka
* Testing and test automation
* Leadership and/or scrum mastering of an agile team
* Implementation of a CI/CD pipeline
* Delivery within a CI/CD pipeline context