Service description
The Operations Center is the Network and Security Operations Center of UNC’s Information Technology Services (ITS), and is staffed by technical professionals around the clock. It operates 24 hours a day, 365 days a year, regardless of any University closure.
Monitor and ensure the reliability and security of campus data communications and services at all times and to assist the campus community in receiving emergency alert communications.
Communications networks and the services supported by them are increasingly critical to UNC Chapel Hill’s educational and research initiatives. In support of this technological backbone, the job of the Operations Center is to ultimately minimize network and service downtime.
- Network Operations Center (NOC): Provide 24×7-status awareness of core network infrastructure. Alleviate off-hours, on-call workloads – requests handled within one hour 24/7/365.
- Security Operations Center (SOC): Provide 24×7-status awareness of IT security infrastructure. Alleviate off-hours, on-call workloads – requests handled within one hour 24/7/365.
- Assist in coordinating professional and timely resolution of network and security problems.
- Monitoring Operations: Provide 24×7 system and service monitoring for services provided by ITS and campus departments – requests handled within one hour 24/7/365.
- Identify problem areas and coordinate corrective action before services are affected.
- Remote KVM / Serial Consoles (Avocent DSView): Provide remote connectivity to out-of-band serial and KVM consoles. Manage software instance on three servers and an inventory of ~85 switches with ~1250 connected target devices across three data centers – requests handled within one business day.
- CS Gold User Access Permissions – requests handled within one hour 24/7/365.
- xMatters: Provide customized, automated individual and team escalation of events originating from various sources such as ServiceNow – requests handled within one business day.
- Data Center Operations Support: Support the ITS DCOps group with various on-site, hands on services as well as coordinate after hours facility responses – requests handled within one hour 24/7/365.
- ITRC after hours & Holidays: The Operations Center covers ITRC service desk responsibilities during off peak after hours and non-peak University holidays – requests handled within one hour 24/7/365.
- Incident Coordination: Facilitate rapid communication of service affecting outages to service owners, the service desk and the Campus committee – requests handled within one hour 24/7/365.
- Alert Carolina Emergency Notification System: The Operations Center is the primary IT provider for the University’s Alert Carolina Emergency Notification System. The Operations Center staff perform weekly drills to maintain proficiency with the multiple steps of the Alert Carolina system activation – requests handled within 15 minutes 24/7/365.
- Support and serve as backup for UNC Public Safety siren activation.
- Publish change management information and status updates to itsstatus.unc.edu
- Provide customers with timely updates on service requests.
- Provide timely, consistent and meaningful communications regarding the availability of campus network and system services.
- ITS Emergency Management Representative to UNC and Chancellor’s Emergency Response Group – requests handled within 15 minutes 24/7/365.
- Internal / External Customer Escalation Point: Provide 24×7 answering point and issue escalation for groups upon request – requests handled within one hour 24/7/365.
- Incident Coordination: Facilitate rapid communication of service affecting outages to service owners, the service desk and the Campus committee – requests handled within one hour 24/7/365.
- Serve as a central point of contact within ITS for addressing campus wide networking, security and system issues.
- Notify ITS Service Desk and campus IT departmental contacts of systems for unplanned outages.
- Assist in dissemination and triage of trouble tickets throughout UNC’s ServiceNow system.
- Facilitate the maintenance of ITS contact and escalation information via xMatters.
- Serve as customer advocate with external service providers.
- Keep abreast of emerging technologies and related management tools.
UNC’s Alert Carolina System
If there is a threat to the safety of individuals on campus, the UNC Alert Carolina System Protocols are followed. This involves the use of all available contact devices:
- Updating the Alert Carolina and UNC home pages.
- Distribution Alert Carolina notifications via SMS text messaging and email.
- Publishing Alert Carolina slides on UNC CATV.
- Creating a University-wide broadcast voicemail.
ITS Service Emergency
If there is an unexpected problem with an ITS service, the Emergency Change Notice Procedures are followed. This involves:
- Creating an unplanned change plan notice and subsequent change notices.
- Collaborating with the ITS Service Desk.
- Updating the itsstatus.unc.edu site, which automatically updates the ‘Alerts and notifications’ section of help.unc.edu.
- Broadcasting a mass voice mail to key IT groups with hourly updates during a major system or network outage.
The policies and procedures outlined below are for all ITS-related issues, whether reported by one of our monitoring tools or a customer. Additionally, information regarding our system registration policy is included below.
Critical Tickets Information
ITS has historically defined a critical request as:
- An incident that affects an individual’s health or safety.
- A computing incident that affects multiple users such that University business is disrupted and an Unplanned Change Plan Change Notice is sent to the campus community.
- A computing incident that affects one person to the extend that they are unable to complete critical work.
- An IT Security incident affecting sensitive data.
Our goal is to assign the request to the responsible group and ensure its acknowledgement within 15 minutes. If the individual handling the critical incident cannot resolve the issue, other individuals or groups may need to assist and the resolution time may vary.
A customer may request or submit a critical trouble ticket for ITS services that do not provide 24/7/365 support. In this circumstance, the Operations Center will contact the customer and relay when they can expect a response to their request.
If there is any uncertainty about a critical ticket, regardless of the time of day, the Operations Center or a Manager On Call is available to discuss the matter.
Customers may check and/or update the status of their request(s) by visiting help.unc.edu, or by calling 962-HELP at any time.
System/Service Administrator Notification Procedures
The Operations Center will attempt to contact the administrator responsible for an ITS system or service if one of our monitoring tools identifies an incident.
Will the Operations Center contact an administrator after normal business hours?
If the system/service administrator has identified a service as critical and the monitoring parameters that have been provided indicate the service is operating abnormally, the Operations Center will contact them.
Before contacting an administrator, the Operations Center will check Change Plans and any special notices to see if there is a known reason for the service impacting event.
How does the Operations Center know who to contact?
Each service group maintains their contact and escalation information in xMatters. This information typically includes at least 3 levels of contacts (with ITS groups going to the AVC level). xMatters is an automated notification engine that notifies the user and group on their user-defined devices at the user-defined interval. Typical escalation from person to person is 15 minutes.
Will the Operations Center create a ticket each time an administrator is contacted?
For a confirmed incident (one in which both the Operations Center and system/service administrator agree) the Operations Center will create a ticket and assign it an appropriate severity. The ticket will be assigned to the administrator that was contacted. The Operations Center will then set the status to ‘Pending Analyst’ so the administrator can work the incident without further interruption. The system/service administrator is responsible for updating the ticket worklog and making any other status changes, such as marking the ticket solved.
Will the Operations Center post a change notice?
The Operations Center will create an unplanned change plan at the request of the service owner for any incident identified as critical by the system/service administrator. The itsstatus.unc.edu website will be updated from the unplanned change plan.
What steps are taken when an incident is detected?
- Check the itsstatus.unc.edu site and Change Plans for scheduled events that might account for the problem.
- Check any recent exceptions to standard operating procedure that might apply to the problem.
- If the above steps yield nothing relevant, notify the system/service owner to confirm the problem and post an unplanned change plan if requested.
- Create a ticket with the appropriate severity (critical or important) and assign it to the administrator that was contacted. After the ticket has been created, the Operations Center will change the status to ‘Pending Analyst’ so the appropriate administrator can work the incident.
- The Operations Center will contact the ITRC directly to relay current information regarding the incident so the Service Desk can address customer issues or concerns appropriately.
- The Operations Center will post the issue to the ITS Status page. Critical incident information will also be posted to help.unc.edu and its.unc.edu.
What is the Operations Center?
The Operations Center is responsible for the day-to-day management of the central computer and networking hardware facilities. By monitoring these systems and devices the Operations Center is able to ensure that all monitored equipment is functional and responsive to user needs.
What are the hours of operation for the Operations Center?
The Operations Center operates 24 hours a day, 7 days a week, 365 days a year. There is always a human presence in the Operations Center, and this schedule is maintained by three shifts.
What systems does the Operations Center monitor?
The Operations Center monitors IT Security services, the University network, ITS campus wide services. To request monitoring of your systems/services, submit a ticket to ITS-OPERATIONS-CENTER.
Who manages the Operations Center?
Neil McKeeman, Operations Center Manager nmckeeman@unc.edu
How many employees staff the Operations Center?
At full staffing, there are 12 members of the Operations Center staff.
How do I contact the Operations Center staff?
How do I institute the ITS Operations Center Conference Bridge?
Contact the Operations Center to request use of the bridge, at 962-6503.
University of North Carolina at Chapel Hill
3100 ITS Manning, 211 Manning Drive, CB#3420
Chapel Hill, NC 27599-3420
919.962.6503
Fax: 919.962.2120
Resources