Scientific Workload Management / Job Scheduler Engineer
South San Francisco, CA

Job Description

As a core member of the Roche Scientific Infrastructure (RSI) team the Scientific Workload Management / Job Scheduler Engineer will be responsible for developing and maintaining all aspects of executing scientific applications in the shared computing environments of RSI. These environments will support capabilities for shared computing including managing the intelligent scheduling service, fair-share policies, guaranteed resource pools, global resource management and dynamic host provisioning. Working closely with other members of the RSI team and Infrastructure Services, the successful candidate will be expected to call on their experience and knowledge, working collaboratively with partner IT groups. The successful candidate is expected to be familiar with at least one high performance job scheduler and have familiarity with common workflow tools and awareness of resource requirements for scientific applications. The candidate would be responsible for contributing to the evolving Elastic Compute (EC service), providing technical consultancy to ensure effective utilization and tight integration with containerized applications and data access via object storage.

Your main responsibilities are:

* Contributing to the strategy and engineering of the job scheduling and workflow integration service and overseeing the operation of the service
* Consulting and partnering with those supporting and using the EC service to modify and optimize their workflows to efficiently compute at scale in a shared resource environment
* Responsible for engineering and maintaining the API interfaces and scheduler configurations for the RSI EC service including fair-share resource pools and dynamic provisioning templates. Familiarity with at least one common large-scale compute job scheduler e.g. PBS, SLURM, UGE, LSF, etc.
* Working closely with other members of the RSI team to ensure that scientific workflows can be effectively executed at scale using the RSI EC service and effectively integrate with other core services i.e. network, data management services, identity & access management (IAM), automation, etc.

The senior Scientific Workload Management / Job Scheduler Engineer will be an experienced scientific and/or IT professional. With a Bachelor's degree (advanced degree preferred) in a relevant field of technology, science or business and possessing the following qualifications:

* 5 to 10 years' experience with engineering shared scheduling systems and their integration and use in scientific environments, including multiple storage tiers, such as high-performance/parallel file systems (e.g. Lustre, IBM Spectrum Scale etc) and RESTful object storage
* Experienced in use of containerized applications for large scale HPC execution
* Experienced in the configuration and running of one or more large-scale compute schedulers and familiarity with workflow tools in one or more scientific disciplines
* Experienced in tuning and optimizing high volume workload scheduling, including fair-share job prioritization and configuring guaranteed resource pools in a shared environment
* Experienced in one or more scripting languages (e.g. Python, Ruby) and or workflow tools (e.g. WDL/CWL) used to automate job flow and control for complex job execution
* Experienced in the use of IAM technologies for credential generation and their management of user credentials as it pertains to queued and executing jobs via the job scheduler and familiarity with configuration and orchestration technologies used with job schedulers (e.g. Bright cluster manager, Ansible, Puppet, Terraform)

If you have questions then please check our FAQs and videos on careers.roche.ch/faq.

#LI-CGMT1

Roche is an equal opportunity employer.

Engineering, Engineering > Design Engineering & Architecture