Six Steps to an Effective Incident Response Exercise

Incident Response drills shouldn’t be viewed as overhead, or a discretionary spend, because a poorly executed response to a cyber security incident translates to material financial, regulatory, and reputational costs for your organization.

Effective incident response requires preparation; this includes not only preventing incidents, by ensuring that systems are secure, but also establishing an incident response capability so that you’re confident your organization is ready to respond.  Practicing your incident response procedure is as critical as creating the incident response plan in the first place.

  1. Design and plan your exercise around a real-world scenario
  2. Establish the exercise objectives and identify participants
  3. Define success criteria to judge your organization’s performance
  4. Brief the facilitator, scribe, and judging panel in advance
  5. Evaluate your organization’s performance in a Hotwash
  6. Capture recommendations in an After-Action report

Step One: Design and plan your exercise around a real-world scenario

Make the scenario as realistic as possible. To facilitate an effective incident response exercise, begin with a design containing three essential elements:  a hypothetical event; triggering clues; and success criteria.  Plan the test so that it requires all participants to work together to achieve the response. This way the exercise will validate that the documented incident response playbooks, process roles and responsibilities, and stakeholder contact lists are accurate and current. Possible real-world scenarios for a tabletop exercise include:

  • Spear Phishing
  • Ransomware
  • Insider Misuse
  • Distributed Denial of Service
  • Theft or Loss of Devices

Regardless of the scenario, a description of a hypothetical incident provides the catalyst for the exercise. For example:

Your CIO receives a potentially malicious phishing email request today, from a sender appearing to be the CFO, asking for confidential banking information.

“I need you to process a same day wire transfer to one of our domestic clients.  What is the required information needed to get it done?  Get back to me to know how soon you can do this.”

Similar emails requesting wire transfer details were reported as received by at least four employees; although it is not yet known whether other employees received this email, or if anyone responded to it.

Create the scenario and include a mix of challenges and tasks so that every participant has an opportunity to contribute to the solution. You could manufacture log entries and screen capture alerts, perhaps from a test environment. Alternatively, prepare a series of questions and answers that guide the participants. For example,

  • The administrator searches log entries on the web server but is only able to determine that there is a gap in the logs.
  • A security admin should search authentication logs for that time frame and determine whether a particular account was compromised. Consequently, the facilitator would share a clue that a specific user ID was compromised.
  • Twenty minutes into the exercise the team will be informed that the web service crashes and the log records are deleted. This clue can be shared verbally with the team, or provided as a screenshot or data logs.

Step Two: Establish the exercise objectives and identify participants

Identify and invite the participants based on the primary aims of the exercise. If the main goal is to validate operational procedures, invite operational-level personnel. If the main goal is to validate the decision-making and oversight processes within the IR plan, include senior-level staff.

Make a list of everyone who will be participating in the exercise and which participant will interact with each scenario element; paying careful attention to balance involvement, so you don’t overload one person or department while others are sitting around observing.

Step Three: Define success criteria to judge your organization’s performance

Define success criteria and write this into your scenario so you’ll know if the team completed sufficient tasks to judge the response to have been successful. For example:

  • Did the team use available playbooks and other documentation, or just “wing it”?
  • Were they expected to determine whether an administrative password was compromised, and then invoke procedures to begin changing all administrative passwords of a particular type?
  • Do they need to document the clues and hand them over to a forensics team?
  • Do they need to restore service?

Step Four: Brief the facilitator, scribe, and judging panel in advance

With so much information passed quickly between the tabletop’s participants, it isn’t practical for one person to facilitate and document observations. Two-person teams, preferably external and objective security consultants, lead the most effective incident response exercises. A Facilitator leads the activity and delivers clues, and a Scribe records all necessary information about the team’s performance. Also, for complex incident response exercises that include multiple groups, judges will be required to assess how the various teams perform. Brief the Scribe and judging panel on the scenario in advance, so they understand their roles and which triggering events to watchout for.

Step Five: Evaluate your organization’s performance in a Hotwash

Participants typically engage in a facilitated debrief, also known as a ‘Hotwash’, immediately following the tabletop exercise to discuss areas that went particularly well and document areas where there may be deficiencies in the current incident response plan. How did your IR team do?

  • Did they follow the documented incident response plan?
  • Did they designate an Incident Commander and engage the CSIRT?
  • Did the team need to make modifications to the response plan during the exercise?
  • What elements of the response didn’t go as expected?
  • How well did communications flow between the participating teams?
  • Where should procedural improvements be made?

Step Six: Capture recommendations in an After-Action report

Capture the observations raised during the event, key findings discussed during the Hotwash, and lessons learned in an After Action Report. To ensure this information is actionable, the Scribe is briefed before the exercise, so they know what type of information to capture and document in the report. In the report describe:

  • The exercise objectives
  • The scenario tested
  • Critical observations made
  • Evaluation of the overall response process
  • Recommendations
  • Actions, accountable owners and due dates

Send the final report to all tabletop participants and the executive with overall responsibility for incident response. Once the evaluation has is completed, use the information to improve the incident response plan and drive process improvements where necessary.

Realize the benefits of exercising your incident response plan

Tabletop exercises such as this builds trust and confidence in your organization’s ability to respond to potentially damaging cyber security incidents because it prepares both operational teams and senior management ahead of an event. They are a cost-effective way to improve incident response programs and are critical to reducing the damage caused by an event by opening up communications channels between managers of IT and other business functions.

Knowledgeable facilitators and scribes will know what the right questions are and when to ask them. External experienced and objective two-person teams lead the most effective incident response exercises that when comprehensively documented, provide evidence for auditors and regulators. It demonstrates that your company has a defined incident response process, and it is subject to continuous improvement. Save this documentation according to your organization’s document retention policy for future audit reference.

Back to Blog

Before you leave…
want to sign up for our newsletter?