DevOps Incident Postmortem Generator
Generate structured postmortems with root cause analysis (5 Whys) and action items. Enter values for instant results with step-by-step formulas.
Formula
Priority = (Severity × Impact) + DetectionGap
We triage the incident based on Severity (Sev1 being highest) and Detection Source. Incidents found by customers get higher priority for 'Detection' action items (improving TTD) than those found by monitoring.
Worked Examples
Example 1: Sev1 Outage
Problem:60 mins down, Customer reported, Config change
Solution:Actions: 1. Add Config Validation. 2. Create Alert. 3. Public Apology.
Result:High Priority Plan
Example 2: Sev3 Latency
Problem:Slow queries, Internal alert, Capacity issue
Solution:Actions: 1. Optimize Index. 2. Tune Auto-scaling.
Result:Maintenance Plan
Frequently Asked Questions
What is a Postmortem?
A written record of an incident: what happened, why it happened, and how to prevent it from happening again. Also called an RCA (Root Cause Analysis).
Why prioritize Action Items?
Postmortems often generate 20+ ideas. You can't do them all. You must prioritize the ones that significantly reduce Risk or TTD/TTR.
Who should write the postmortem?
The person who knew the most about the incident (often the one who fixed it), but reviewed by the whole team.
Do I need a postmortem for Sev3?
Usually no. A lightweight log entry is enough. Save the deep analysis for incidents that actually hurt the business.
Background & Theory
The Incident Lifecycle
- Detection: Monitoring or User Report.
- Response: Assessing severity, paging, mitigating (stopping the bleeding).
- Analysis: Finding the root cause (5 Whys).
- Remediation: Fixing the underlying defect (Action Items).
Action Item Taxonomy
- Fix: Repair the immediate bug.
- Prevent: Make it impossible to recur (e.g., type checking).
- Detect: Know about it faster next time (e.g., better alert).
- Mitigate: Reduce the impact if it happens again (e.g., graceful degradation).
Practical Tips
- Assign Owners: An action item without an owner and a due date is just a wish.
- Link to Tickets: Create JIRA/Linear tickets immediately. Don't leave them in the doc.
- Review Past Items: Before closing a postmortem, check if previous action items (from past incidents) would have prevented this one. If yes, why weren't they done?
History
Aviation and Safety Culture
Postmortems originated in aviation (NTSB) and medicine. When a plane crashes, an independent body investigates not to punish the pilot, but to update the safety manuals for *every* pilot. This "Just Culture" made aviation the safest mode of transport.
Google and SRE
Google adapted this for software in their Site Reliability Engineering (SRE) handbook. They popularized the "Blameless Postmortem." The logic: You cannot fire your way to reliability. If you fire the engineer who made the mistake, you lose the person with the most knowledge about how to fix the system.
The Modern Standard
Today, tools like PagerDuty, Incident.io, and JIRA have built-in postmortem workflows. The process has shifted from "Who did it?" to "How did the system allow this to happen?" It is the primary mechanism for organizational learning in tech.
Common Misconceptions
- Myth: "Root Cause" is singular. Reality: Complex systems usually fail due to a combination of factors (Swiss Cheese Model), not one single bug.
- Myth: "Human Error" is a root cause. Reality: Human error is a symptom of bad tool design. The button shouldn't have been that easy to click.