Principal Production Engineer (PE) Oracle
Redwood City, CA

Job Description

Solve complex problems related to infrastructure cloud services and build automation to prevent problem recurrence. Design, write, and deploy software to improve the availability, scalability, and efficiency of Oracle products and services. Design and develop designs, architectures, standards, and methods for large-scale distributed systems. Facilitate service capacity planning and demand forecasting, software performance analysis, and system tuning.

Work with Site Reliability Engineering (SRE) team on the shared full stack ownership of a collection of services and/or technology areas. Understand the end-to-end configuration, technical dependencies, and overall behavioral characteristics of production services. Responsible for the design and delivery of the mission critical stack, with focus on security, resiliency, scale, and performance. Authority for end-to-end performance and operability. Partner with development teams in defining and implementing improvements in service architecture. Articulate technical characteristics of services and technology areas and guide Development Teams to engineer and add premier capabilities to the Oracle Cloud service portfolio. Understand and communicate the scale, capacity, security, performance attributes, and requirements of the service and technology stack. Demonstrate clear understanding of automation and orchestration principles. Act as ultimate escalation point for complex or critical issues that have not yet been documented as Standard Operating Procedures (SOPs). Utilize a deep understanding of service topology and their dependencies required to troubleshoot issues and define mitigations. Understand and explain the affect of product architecture decisions on distributed systems. Professional curiosity and a desire to a develop deep understanding of services and technologies.

A BS or MS in Computer Science, or equivalent. Identifies and implements complex solutions to knowledge of server hardware and software configuration, networking, standard internet services, scripting languages, cloud computing patterns, technology security and compliance. Experience running large scale customer facing web services. Identifies and implements complex solutions to understanding of load balancing technologies and experience with development in programming languages, databases and big data stores, and container technologies. Work involves defining and documenting technical architecture of complex and highly scalable products. A minimum of 8 years experience of running large scale customer facing web services.

Oracle is an Equal Employment Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability and protected veterans status or any other characteristic protected by law.

Job Title: Principal Production Engineer (PE), Oracle Cloud Service Center

Purpose: As a Principal Production Engineer (PE), you will be focused on improving service reliability, performance and operability of Oracle Cloud Services. You will have your hand on the pulse of the service and will play a key role in responding to live service issues. Additionally, you will have the opportunity to create future automation and tooling that will allow us to continuously improve our service. One of the most critical roles that the PE plays is the Incident Commander, where you will be accountable for command and control of major incidents that are impacting Oracle's Cloud services.

Location: California-Redwood City/Santa Clara

Description: Working as part of a Production Engineering Team, you will be part of Oracle's Cloud Operations with the mission to maintain and enhance services that generate over 270 billion platform transactions weekly. You will have a pulse on the live service at all times and be able to communicate the "health of the service" while in command. As part of our global SaaS Engineering team, you will be the technical lead, driving major incident response while working with subject matter experts (SMEs) across various cloud services teams (Service Operations, OCI Infrastructure including Compute, Storage and Network, Feature Development, etc.) toward rapid restoration of service. You will be empowered to make the call on some of the most significant live service events impacting our business.

You will leverage excellence in communication, technical/business analysis, problem solving and attention to detail to methodically resolve issues. Technically, you will understand the full stack of the services you support (Network to Application) and are able to establish risks/benefit in order to drive the action that delivers the best result for our customers. Further, you will drive improvements through the development and introduction of tools and engage partner teams to drive down incident counts, reduce severity of events and minimize TTx. You will help create a "Site Up" culture across all Oracle Cloud teams. We will look to the PE to continually review and enhance systems, methods and applications to enable the delivery of a positive customer experience to highly acclaimed hosted products including Oracle Service Cloud, Fusion Apps, Taleo, TOA, Responsys, and Eloqua.

As a member of the Oracle SaaS Engineering team, you will be surrounded by "willing to help" individuals representing some of the brightest and most innovative minds in the industry. You will be a part of an organization that prides itself on providing training, empowerment, and career progression.

Responsibilities:

* Be a technical leader within SaaS Engineering who serves as command and control of the most complex incidents.
* Develop automation, self-healing and propose configuration changes and architectures necessary for tactical and strategic solutions to mitigate service risk, improve efficiency, reliability and performance.
* Define and refine monitoring and response toolset to rapidly detect and respond to service impacting events.
* Work upstream with Service Operations and Apps/Feature Development to clearly understand change plans, risks and rollback to reduce impact of change.
* Execute approved changes during change windows. Remotely deploy and configure services.
* Participate in project delivery aimed at increasing capabilities around monitoring, notification, configuration and deployment of servers and applications within the Oracle Cloud Platform.
* Assist in the training and development of more junior team members.

Knowledge, Skills, Abilities, and Background

* College diploma or university degree in the field of computer science or equivalent or 5 years equivalent work experience in infrastructure, systems, engineering or development environment.
* Excellence in verbal and written communication. Ability to communicate with all levels during critical events and be a bridge for technical discussion with non-technical people.
* Strong Technical background with an ability to troubleshoot issues impacting large scale service architectures and application stacks.
* Demonstrable experience in one or more scripting/programming languages such as Powershell, Python, Perl, Java, BASH, GO…
* Familiarity with large scale cloud networking infrastructure, including network architectures, TCP/IP protocols, firewall management, routing, switching, ACLs, SSL/TLS
* Database administration with technologies such as Oracle RAC, MySQL/Maria, Postgres, MongoDB
* Expertise utilizing Cloud Infrastructure such as Oracle Cloud Infrastructure, Azure, AWS, OpenStack, GCP
* Experience installing, configuring, and maintaining Linux, Linux services, and Linux networking, preferred Oracle Linux.
* Configuration and maintenance of applications such as WebLogic, Apache, NGINX, Envoy, Memcached, Squid, NFS, DHCP, NTP, SSH, DNS and
* Master of source code management for large teams using frameworks such as GIT
* Automated configuration management using Puppet, SALT or similar technologies
* Expert skills in all aspects of incident management (Incident Commander, Scribe, Troubleshooter, etc)
* Familiarity with JAVA application troubleshooting (JFR, Stack Traces, Weblogic and EM metrics, etc)
* Good interpersonal skills with ability to present ideas in business-friendly and user-friendly language. Team oriented with a passion to share knowledge and teach best practices
* Highly self motivated and directed with keen attention to detail.
* Proven analytical and problem-solving skills with strong customer service orientation.
* Ability to effectively prioritize and execute tasks in a high-pressure environment.
* Able to work as part of a shift in a 24x7x365 DevOps team. Must be willing to work non-standard work hours (though primary is US daytime working hours) including being part of an on-call roster for holidays and weekends.