City & County of San Francisco
San Francisco, CA

Job Description

Introduction

Applicants are encouraged to apply immediately as this recruitment may close at any time but not earlier than March 21, 2019. The position is open until filled.

Position:

The Department of Technology is the centralized technology services provider within San Francisco City & County government, delivering technology infrastructure and services to approximately 33,000 employees and 800,000 residents. The department has an annual operating budget of over $130M and contains over 220 employees. Core service areas include: Infrastructure and Operations, Technology Security, Service Delivery & Management, Enterprise Applications, Project Management Office, Public Safety Systems & Wiring, Technology Administration, Policy & Governance and IT Public Communications.

The primary responsibility of the Senior Systems Engineer is to effectively manage & administer a complex multi-tenant private cloud infrastructure. Aspects of this position include backup, recovery, capacity and disaster recovery planning, provisioning, installation/configuration, operations and maintenance of systems hardware and software, report creation and distribution, service level support.

The emphasis for the Senior Systems Engineer position will manage the architecture, administration, maintenance, development, and implementation of policies and procedures for providing recovery of data and systems.

In this position the incumbent must have the ability to determine, plan & understand the complexities of VDCs, the systems or networking components & requirements to plan for or enhance the needs of the Enterprise system or network. Troubleshoot and resolve VDC problems with minimal disruption to the system or network; exercise judgment, decisiveness and creativity in designing, planning, troubleshooting and integration of large complex systems or networks based on the potential benefits and/or consequences; develop alternative solutions within established guidelines and standards; communicate effectively orally and in writing; establish good working relationships with other IS professionals, vendors, management and departments; read and understand professional journals, manuals and literature; and prioritize competing requests for service.

The 1043 Senior Systems Engineer assignments are in a highly technical area which supports VDCs residing in geo diverse data centers which host multiple City client departments. The Senior Systems Engineer will provide functional and technical expertise; develop and/or modify procedures to solve VDC issues; perform multiple software functions and support hardware platforms.

Essential Duties:

* Design, plan, install, test, integrate, implement, document, maintain and enhance the physical and logical components of a large complex system of virtual servers, virtual server platforms and storage as well as 3rd party cloud offerings (Azure, AWS, Google) & Disaster Recovery Architecture. This may involve the analysis and evaluation of systems software, hardware and communications strategies.
* Manages disaster recovery and resumption plans.
* Collaborate with other City departments in datacenter disaster recovery plan, including remote hot site replication of critical infrastructure
* Experience with best practices, and procedures within the VM platforms, VDC Operations, & disaster recovery
* Plan, implement and provide senior level support pertaining for Cloud technologies related to PaaS, Saas & IaaS.
* Providing Linux OS and Windows Server administration skills to maintain, manage, administer access, control and perform upgrades of the server infrastructure supporting all business and infrastructure host systems on various virtual environment, and traditional implementations
* Performing system modifications to accommodate hardware and OS version upgrades
* Installing and hardening Linux & Windows operating systems and planning for applying security and recommended patches on a regularly scheduled basis
* Performing system modifications to accommodate hardware and OS version upgrades
* Managing storage allocation and back-up/recovery operations
* Troubleshooting of system performance in collaboration with application support, database administration and network administration groups.
* Working and troubleshooting network infrastructure with the Department of Technology staff including switches, routers and firewalls
* Working in close collaboration with City Wide technology staff in the support and maintenance of the network, security and hardware support teams to accomplish desired results
* Participating in on-call rotation and maintenance of server support and performing other duties as assigned
* Ability to thrive in time-sensitive, results oriented environment
* Relies on experience and judgment to plan and accomplish goals.
* Leads and directs the work of others.
* A wide degree of creativity and latitude is required.
* Manages daily activities for ensuring the security and integrity of company data, databases, information systems, and technology.
* Cross-train in various support areas in order to provide assistance during afterhours on-call rotation
* Assist in developing best practices and procedures for Oracle Engineered systems operations
* Help support Level 1, 2, 3, troubleshooting with 24x7 support
* Document procedures and troubleshooting steps related to all aspects of our hybrid cloud infrastructure, including but not exclusive to underlying hardware comprising compute (blades), storage (SAN/NAS), storage and data networks, and software, Azure cloud; develop and review documentation prior to general distribution.
* Assist in establishing policies, procedures and standards. Develop alternative solutions to problems within established guidelines and standards.
* Provision or install all underlying virtual platform components, including but not exclusive to compute, storage, and network platforms.
* Ensure system recoverability and performance stability of all aspects of software and hardware comprising our hybrid cloud environment encompassing on premise virtual platform and virtual machines running on the platform, both management and tenant.
* Perform departmental and system-wide virtual and storage system assessments, including "physical to virtual" and "virtual to virtual" server migrations over dissimilar platforms and technologies. Optimize software and hardware parameters for smooth integration/migration and assist in determining network efficiencies.
* Act as the liaison between vendors, technical support hot-lines and departments to resolve complex system problems as needed. Assist in the coordination and implementation of corrective measures; this may involve site visits, telephone assistance, remote systems management, and participation in technical committees. - Participate and/or drive the establishment of standard engineering practices for service, project delivery and documentation.
* Respond to service request and incident assignments, validating incident and request information, executing and closing assigned tickets, engaging Operations and Support organizations as needed to provide a positive support experience for the client or tenant.
* Perform other related duties as requested.

Job Type:

The Temporary Exempt- Full Time position is excluded by the Charter from the competitive civil service examination process and shall serve at the discretion of the appointing officer. The anticipated duration of this project position is twenty-four (24) months and will not result in an eligible list or permanent civil service hiring.

Nature of Work:

Incumbent must be willing to work a 40-hour per week schedule and may be required to work additional hours when necessary as determined by the department.

Work Location:

Incumbent will conduct the majority of work at the Department of Technology, 1 South Van Ness. However, there may be situations where the incumbent will be required to work at other sites throughout the City of San Francisco as necessary.

Minimum Qualifications

These minimum qualifications establish the education, training, experience, special skills and/or license(s) which are required for employment in the classification. Please note, additional qualifications (i.e., special conditions) may apply to a particular position and will be stated on the exam/job announcement.

Education:

An associate degree in computer science or a closely related field from an accredited college or university OR its equivalent in terms of total course credits/units [i.e., at least sixty (60) semester or ninety (90) quarter credits/units with a minimum of twenty (20) semester or thirty (30) quarter credits/units in computer science or a closely-related field].

Experience:

Three (3) years of experience analyzing, installing, configuring, enhancing, and/or maintaining the components of an enterprise network.

Substitution:

Additional experience as described above may be substituted for the required degree on a year-for-year basis (up to a maximum of two (2) years). One (1) year is equivalent to thirty (30) semester units/ forty-five (45) quarter units with a minimum of 10 semester / 15 quarter units in computer science or a closely related field.

Desirable Qualifications:

Previous as an IS Engineer Senior or related experience in a large enterprise environment that demonstrates the knowledge and ability to perform essential duties; experience working with Linux and Windows Server administration skills to maintain, manage, administer access, control and perform upgrades of the infrastructure supporting all business and infrastructure host systems on private cloud virtual environment, experience working as a VDC support engineer, Solid understanding of load balancing, horizontal scaling, UNIX-related network services, TCP/IP networking, DNS and HTTP(s); Experience in project management; Familiarity with Windows Server OS, WSH, Perl, Visual Basic or batch file commands; Experience with Visio; Ability to interact effectively with team members and all levels of users; and to balance customer needs while adhering to corporate and department policies; Attention to detail and commitment to quality; Ability to develop and follow written procedures and guidelines; Project planning and execution; and Ability to work well with limited supervision; and to multi-task and deal with changing and competing priorities.

* Disaster Recovery/Business Continuity Program Development expertise


* Experience with disaster recovery policy and management, and process development and improvement.


* Senior engineer level understanding of Windows Unix/Linux Operating Systems: Red Hat, Solaris, CentOS


* Virtualization: VMware


* Storage: Network Appliance, EMC, Oracle Pillar


* Strong SAN, NAS knowledge


* Experience with virtual/Cloud data center consolidations, relocations, or moves.


* Experience with public cloud workload migrations


* Experience with DR/BC strategy


* Experience with DR Plans/Playbook/Orchestration


* Experience with DR Testing and Exercise


* Experience in government and/or public safety environment.


* Experience working with and configuring VMware vCloud Networking and Security (VCNS), VMware vShield and associated components and appliances.


* Hands on experience implementing and supporting VMware vCloud Director 5.x, or vCloud Automation Center 6.x.


* Experience implementing and regularly utilizing VMware Operations Center 5.8/6.


* Hands on experience supporting other Virtual Machine Technologies such as Microsoft Hyper-V or AWS.


* Experience with vCenter Orchestrator (VCO) or Puppet orchestration tools.


* In depth knowledge of Cisco Unified Computing System (UCS) Blade Servers and their respective management software platforms.


* n depth knowledge of Storage Area Network management tools/software, such as EMC's Unified Infrastructure Manager or NetApp's On Command system, and associated replication methodologies.


* Experience with implementing disaster recovery and high availability architectures utilizing SAN and VMware based replication technologies and tools such as VMware Site Recovery Manager (SRM).


* Experience with Windows Server 2008/2012, Active Directory, and Red Hat Linux provisioning in a multi-domain environment.


* Current Microsoft Certified Systems Engineer certification (MCSE) on Security and Windows Server 2012 Active Directory.


* VMware Certified Systems Engineer certifications such as VMware VCAP-DCA, VCAP-CIA and VCP-NV Certification or higher.



Special Requirements:

Criminal Justice Information Services (CJIS) Security Clearance may be required. (See Security Clearances and Background Investigations below).

Note:

1. Medical Testing: Prior to appointment, eligible candidates must successfully pass the TB testing process.

2. Security Clearances & Background Investigations: Positions in this classification may require that successful candidates who become eligible for appointment may be required to go through a background investigation to determine the candidate's suitability for employment in this classification. Factors considered in the investigation may include employment history, use of illegal/controlled substances. Reasons for rejection based on this investigation may include, but not limited to: applicable convictions, repeated or serious violations of the law, inability to accept supervision, inability to follow rules and regulations, falsification of application materials and/or other relevant factors. Failure to obtain and maintain security clearance may be basis for termination.

How To Apply

Applications for City and County of San Francisco jobs are only accepted through an online process. Visit www.jobaps.com/sf to register an account (if you have not already done so) and begin the application process.

§ Select the desired job announcement

§ Select "Apply" and read and acknowledge the information

§ Select either "I am a New User" if you have not previously registered, or "I have Registered Previously"

§ Follow instructions on the screen

Computers are available for the public (from 8:00 a.m. to 5:00 p.m. Monday through Friday) to file online applications in the lobby of the Dept. of Human Resources at 1 South Van Ness Avenue, 4th Floor, San Francisco.

Applicants may be contacted by email about this announcement and, therefore, it is their responsibility to ensure that their registered email address is accurate and kept up-to-date. Also, applicants must ensure that email from CCSF is not blocked on their computer by a spam filter. To prevent blocking, applicants should set up their email to accept CCSF mail from the following addresses (@sfgov.org, @sfdpw.org, @sfport.com, @flysfo.com, @sfwater.org, @sfdph.org, @asianart.org, @sfmta.com, @sfpl.org, @dcyf.org, @first5sf.org).

Applicants will receive a confirmation email that their online application has been received in response to every announcement for which they file. Applicants should retain this confirmation email for their records. Failure to receive this email means that the online application was not submitted or received.

All work experience, education, training and other information substantiating how you meet the minimum qualifications must be included on your application by the filing deadline. Information submitted after the filing deadline will not be considered in determining whether you meet the minimum qualifications. Resumes may be attached to the application; however, resumes will not be accepted in lieu of a completed City and County of San Francisco application.

Applications completed improperly may be cause for ineligibility, disqualification or may lead to lower scores.

Note: Falsifying one's education, training, or work experience or attempted deception on the application may result in disqualification for this and future job opportunities with the City and County of San Francisco.

If you have any questions regarding this recruitment or application process, please contact the exam analyst, Carol Wong, by telephone at 652-628-5045, or by email at carol.x.wong@sfgov.org.

Selection Procedures

Supplemental Questionnaire (Qualifying and Informational):

Applicants will be prompted to complete a supplemental questionnaire as part of the online employment application. It is essential that applicants provide complete information in identifying their education, experience, training and licensure, consistent with the information provided on their application. The supplemental questionnaire will be used to evaluate whether applicants meet the minimum qualifications. The Questionnaire will also be used to assess the applicant's knowledge, skills, and abilities as they relate to the this position. Applicants must also complete the official application. The information in the supplemental questionnaire will not be scored and is collected for informational purposes.

The Department may establish and implement additional screening mechanisms to comparatively evaluate qualifications of candidates. If this becomes necessary, only those applicants whose qualifications most closely meet the needs of the Department will be invited for an interview.

Note: Applicants who meet the minimum qualifications are not guaranteed to advance through all of the steps in the selection process.

Verification:

Applicants may be required to submit verification of qualifying education and experience at any point during the recruitment and selection process. If education verification is required, information on how to verify education requirements, including verifying foreign education credits or degree equivalency, can be found at http://sfdhr.org/index.aspx?page=456

THE CITY AND COUNTY OF SAN FRANCISCO IS AN EQUAL OPPORTUNITY EMPLOYER. MINORITIES, WOMEN AND PERSONS WITH DISABILITIES ARE ENCOURAGED TO APPLY.

Conviction History

As a selected candidate for a job, you will be fingerprinted, and your fingerprints will be sent to the California Department of Justice (DOJ) and the Federal Bureau of Investigation (FBI). The resulting report of your conviction history (if any) will be used to determine whether the nature of your conviction (or arrest, in limited circumstances) conflicts with the specific duties and responsibilities of the job for which you are a selected candidate. If a conflict exists, you will be asked to present any evidence of rehabilitation that may mitigate the conflict, except when federal or state regulations bar employment in specific circumstances, such as:

* Candidates for positions with the Unified School District and the Community College District may be disqualified from consideration should their conviction history not meet the standards established under the California Education Code.


* Candidates for positions with the Recreation and Park Department may be disqualified from consideration should their conviction history not meet the standards established under California Public Resources Code 5164.



Having a conviction history does not automatically preclude you from a job with the City.

If you are a selected candidate, the hiring department will contact you to schedule a fingerprinting appointment.

Disaster Service Workers

All City and County of San Francisco employees are designated Disaster Service Workers through state and local law (California Government Code Section 3100-3109). Employment with the City requires the affirmation of a loyalty oath to this effect. Employees are required to complete all Disaster Service Worker-related training as assigned, and to return to work as ordered in the event of an emergency.

Conclusion

Terms of Announcement:

Applicants must be guided solely by the provisions of this announcement, including requirements, time periods and other particulars, except when superseded by federal, state or local laws, rules or regulations. Clerical errors may be corrected by the posting the correction on the Department of Human Resources website at www.jobaps.com/sf.

Requests:

Applicants with disabilities who meet the minimum eligibility requirements for this job announcement can find information on requesting a reasonable ADA Accommodation at: http://sfdhr.org/information-about-hiring-process#applicantswithdisabilities

General Information concerning City and County of San Francisco Employment Policies and Procedures: Important Employment Information for the City and County of San Francisco can be obtained at http://sfdhr.org/information-about-hiring-process or hard copy at 1 South Van Ness Avenue, 4th Floor.

Copies of Application Documents:

Applicants should keep copies of all documents submitted, as these will not be returned.

Right to Work:

All persons entering the City and County of San Francisco workforce are required to provide verification of authorization to work in the United States. Please be informed that the Department of Technology will not sponsor visa applications/transfers.

Issued: March 7, 2019

Micki Callahan

Human Resources Director

Department of Human Resources

Recruitment ID Number: TEX-1043-092412

DT/CW/628-652-5045

Benefits

All employees hired on or after January 10, 2009 will be required (pursuant to San Francisco Charter Section A8.432) to contribute 2% of pre-tax compensation to fund retiree healthcare. In addition, most employees are required to make a member contribution towards retirement, ranging from 7.5%-13.25% of compensation. For more information on these provisions, please contact the personnel office of the hiring agency.

For more information about benefits, please click here.

Click on a link below to apply for this position: