Best Practices: Be Prepared with a Good Security Incident Response Plan

by Steven Weil, Senior IT Security Auditor, Coalfire

Imagine the following scenarios. Late on a Tuesday night, your company’s database administrator performs some off-hours maintenance on several production database servers. The administrator notices some unusual directory names on one of the servers. The administrator also discovers an unfamiliar account with high-level privileges on the server.

While creating a weekly usage report, one of your network administrators notices that off-hours bandwidth utilization has been significantly higher than usual.  The administrator sees that an unusually large percentage of the activity involves an internal FTP server. The network administrator contacts the FTP server administrator who determines that the server is hosting unauthorized materials, which appear to include pirated software, songs, and movies.

Do you panic and start "shooting from the hip" or do you take a calm, methodical approach to managing these security incidents? If you have a formal, well-thought-out security incident response plan (SIRP), your organization can effectively and promptly respond to security incidents that could compromise the confidentiality, integrity or availability of your information systems, data and/or network resources.

SIRP Phases

Let’s discuss what should be in an effective SIRP and how having a plan can benefit your organization. In general, a good SIRP should formally assign responsibilities to appropriate employees or teams and establish a step-by-step approach to identifying, responding to and managing security incidents. More specifically, a good SIRP should include the following phases.

Preparation phase:
The goal of the preparation phase is to ensure that your staff is appropriately trained and has formal procedures and tools for detecting and responding to a security incident.  In this phase, you should identify and define a security incident response team (SIRT) that will respond to and manage security incidents. You should also create a current external contact list of service providers and other organizations that may need to be contacted during a security incident. Additionally, you should create a current list of your organization’s legal, public relations, and management employees that also may need to be contacted.

Benefit: You won’t waste valuable time determining who should respond to an incident and trying to find critical contact information.

Identification phase:
This phase is focused on determining whether or not a security incident has occurred and, if one has occurred, determining the type and severity of the incident. You should assign responsibility to specific employees or teams for reviewing and documenting possible security incidents and develop an incident classification system (e.g. low, medium, high severity). Additionally, you should formally define when the SIRP will be activated and when your organization’s management is notified that an incident has occurred. 

Benefit: Your organization won’t waste time and money overreacting to “false positive” incidents; instead, it will appropriately respond to real security incidents. Your management will be appropriately notified about security incidents.

Containment phase: The containment phase is focused on limiting the scope and magnitude of a security incident.  You should define where your SIRT will gather and how they will initially respond to an incident. Establish formal processes for determining whether or not law enforcement should be contacted about an incident and whether or not systems impacted by an incident should be allowed to operate. You should also define how and when to notify those impacted by a security incident (e.g. information system owners and administrators, third-party organizations who use impacted services or data).
Benefit: Bringing in law enforcement and/or turning off services or systems are critical decisions with many significant effects. Instead of having to make “on-the-fly” decisions under stressful circumstances, you will have a formal process to follow. Key people and business partners impacted by an incident will be appropriately and promptly notified.

Eradication phase: The goal of the eradication phase is to eliminate all adverse impacts caused by a security incident and mitigate the vulnerability(s) that led to the incident. You should define the general processes the SIRT will use to determine how security incidents occurred (e.g. log reviews, camera data review) and how they will mitigate the vulnerability(s) that led to an incident.  Additionally, you should establish a formal process for deciding whether information systems impacted by a security incident should be rebuilt or repaired.

By formally identifying and mitigating vulnerabilities that led to a security incident, further incidents can be prevented.

Recovery phase:
The goal of the recovery phase is to return all data and/or services impacted by a security incident to full operational status. You should define a formal process for validating that rebuilt or repaired information systems are functioning as expected. You should also have a process for determining when impacted data or services are made available again if they have been turned off because of an incident. This phase should also include a formal process for following appropriate state data breach laws if the security incident has resulted in unauthorized access of “non-public” personal information.

Benefit: Systems and data impacted by a security incident are returned to “production” status in a planned manner rather than "on the fly".  This can prevent costly mistakes.

Follow-up phase:
The focus of the follow-up phase is to identify lessons that will enable your organization to more effectively respond to and manage the next security incident. You should require that a “post mortem” meeting be conducted promptly after the incident response is complete. The meeting should result in documentation of lessons learned by employees who responded to the incident along with recommendations for better responding to and managing the next security incident.

Benefit: By formally getting employee feedback, you can modify your SIRP as necessary and ensure that your next security incident will be even better managed. 


It is important that your SIRP work in the real world and be customized to your organization rather than being a general template plan.  At least annually, you should carefully test your SIRP to make sure it is still current and appropriate for your organization. Having a good, formal SIRP can save your organization time and money and allow it to more effectively respond to and manage security incidents.