Library→Third Way: Continual Learning→Blameless Postmortems
Blameless Postmortems
How to turn production failures into organizational learning. Why blame stops learning — and how to create an environment where problems are surfaced, not hidden.
Video Lesson
A video lesson for this topic is in development. The library articles and mission exercises cover the same material in the meantime.
Why blameless?
When a production incident occurs, there are two possible organizational responses. A blame culture asks: who caused this? A learning culture asks: what conditions made this possible?
Sidney Dekker's research in safety engineering shows that in complex systems, failures are never caused by a single person. They are the result of multiple contributing factors — inadequate tooling, poor documentation, missing monitoring, time pressure, incomplete testing. The human who triggered the failure was the last link in a long chain.
Blame is seductive because it is simple. It gives the illusion of a fix: remove the person, prevent the failure. But in complex systems, if one person could cause that failure, so can the next person put in their place. The system has not changed.
The postmortem format
A postmortem is a document, not a meeting — though a meeting is used to produce it. The document is the artifact that persists and can be shared. A well-structured postmortem has five sections:
1. Incident summary
What happened, when, for how long, and the user impact. Written for engineers who were not on call. One paragraph.
2. Timeline
Exact timestamped events from first signal to resolution. Detection time, escalation time, mitigation time, resolution time. Include gaps.
3. Contributing factors
Not causes — factors. What conditions made this incident possible? Each factor is a potential action item. Avoid 'human error' as a factor.
4. What went well
What helped detect and resolve faster? Good runbooks? Fast rollback? Functioning alerts? These should be reinforced and made more reliable.
5. Action items
Concrete tasks with owners and due dates. Each removes a contributing factor. SMART: specific, measurable, achievable, relevant, time-bound.
Psychological safety in postmortems
A postmortem is only as good as the information people are willing to share. Engineers who fear blame will omit the details most useful for learning. The facilitator's job is to make the room safe for complete honesty.
Facilitator role
The facilitator is a neutral party — not the manager of the engineers involved. Their job is to keep the conversation on contributing factors, redirect blame, and ensure everyone's perspective is heard.
Language matters
"Who deleted the database?" vs "What sequence of events led to the database deletion?" The second question generates a timeline. The first generates silence.
No punishment rule
Engineers must know in advance that honest participation in a postmortem will not result in disciplinary action. This rule must be demonstrated, not just stated.
Separate learning from performance
Performance issues — repeated incidents caused by the same individual — are a management conversation. Not a postmortem topic. Conflating them destroys postmortem culture.
Action items
A postmortem with no action items is a historical document, not an improvement mechanism. Action items are what convert learning into change. They must be:
"Add alert for connection pool exhaustion" — not "be more careful about migrations"
One named owner, not "the team" or "DevOps"
A due date in the next sprint, not "when we have time"
Linked to a ticket in the team's backlog. Reviewed in the next retrospective.
"Improve our deployment process" — no owner, no deadline, no definition of done
Sharing postmortems
The organizational value of a postmortem multiplies when it is shared. An incident that affected one team may contain a lesson for every team. A shared postmortem library is a documented organizational memory of failures — and how the system improved after each one.
Internal wiki
Every postmortem published to a searchable internal wiki within 48 hours of the incident review. Tagged by service, root cause category, and severity.
Weekly digest
A weekly email or Slack post summarizing recent incidents and their key learnings. 5 minutes to read. Keeps the whole organization aware without requiring everyone to attend every review.
Quarterly trend review
Aggregate incident data to find recurring patterns. What root cause categories appear repeatedly? Where should the next reliability investment go?
Google publishes its postmortems internally across all SRE teams. An engineer in London can learn from an incident in Tokyo they were not involved in. This is organizational learning at scale.
Further reading
Google SRE Book — Chapter 15
Postmortem Culture: Learning from Failure. The full Google postmortem format with real examples.
DevOps Handbook — Chapter 26
Blameless postmortems and organizational learning. Integration with the improvement kata.
Sidney Dekker — Just Culture
The human factors research behind blameless cultures. Why human error is a label, not an explanation.
John Allspaw — Blameless PostMortems
The Etsy engineering blog post (2012) that brought blameless postmortems to the DevOps community. Still the clearest statement of the idea.