How to Build An Engineering Team Culture That Survives Incidents
Two teams at the same company can share the same values page yet behave completely differently at three in the morning. That's one of the clearest ways to understand engineering culture.
For example: one team declares an incident twelve minutes after the first unusual graph, brings in three people familiar with the subsystem, and openly admits that they don't yet know what's happening.
The other waits because declaring an incident feels like escalating, and escalating feels like admitting they couldn't handle it alone. By the time anyone else joins, the first engineer has already spent forty minutes investigating the incident alone.
Nothing in either team's stated values tells you which one you're dealing with. What it tells you is how they work on an ordinary Tuesday: what their tools make easy, what their processes quietly reward, and what happened the last time someone raised a false alarm.
What Engineering Team Culture Means
Engineering team culture is the set of behaviors a team falls back on without thinking. It shapes how people communicate, make technical decisions, handle mistakes, and respond when something breaks. In other words, it's what the team does by default, which isn't always the same as what it says it values.
It's important to separate this from company culture because the two are often confused. Company culture is shaped by hiring, leadership, and organizational values. But engineering culture shows up in code reviews, service ownership, and whether people feel comfortable admitting they're stuck during an incident.
Two teams inside the same organization, reporting to the same VP, working from the same values document, can still routinely develop opposite norms because their day to day working conditions differ.
- Company culture sets organization-wide values, while engineering team culture sets the operating behaviors a single team defaults to
- Company culture shapes who gets hired, while engineering team culture shapes how those engineers actually work together once they arrive
- Company culture is measured through employee experience, while engineering team culture shows up in how technical decisions get made
- Company culture is expressed through leadership principles, while engineering team culture is expressed in how incidents get handled
- Company culture points at the customer mission, while engineering team culture determines whether reliability is protected in practice
This matters because engineering culture has real technical consequences. It influences whether a near miss is reported or quietly fixed and forgotten, whether an engineer asks for help after five minutes or fifty, and whether the same failure happens three times in a year.
Those are reliability outcomes, and they often come down to team behavior rather than system design.
What Good Engineering Culture Looks Like in Practice
DORA's work on generative organizational culture found that a high-trust culture that prioritizes the flow of information predicts both software delivery performance and organizational performance, building on Ron Westrum's typology of organizations as pathological, bureaucratic, or generative.
Their analysis of the 2019 State of DevOps Report went further and found that psychological safety specifically predicts delivery performance, organizational performance, and productivity.
Google reached a similar conclusion from a different direction. Project Aristotle identified psychological safety as the first of five dynamics behind effective teams, alongside dependability, structure and clarity, meaning, and impact, and the researchers were clear that how a team worked together mattered more than who was on it.
None of that turns into a values statement you can hang on a wall. It shows up in how the team behaves under pressure. The most useful engineering values are the ones you can recognize when the system is on fire.
| Value | How it shows up during an incident |
|---|---|
| Ownership | Engineers take responsibility for the outcome without anyone assigning fault |
| Transparency | Status updates stay honest and timely, including when nobody knows the cause yet |
| Collaboration | People pull others in early rather than working the problem alone |
| Continuous learning | Findings change the system, not just the document |
| Reliability | Preventing recurrence carries the same weight as restoring service |
Ownership is the one most often misread. A team with strong ownership is not a team where one person is accountable for every failure in their service. It is a team where the person closest to the problem drives the response and knows they will not be the one holding it alone.
Engineering Team Culture Examples You Can Actually Observe
Most descriptions of engineering culture rely on adjectives like: Collaborative, innovative, transparent, and customer-focused. They sound good, but they tell you very little about how a team actually works. If you want to see a team's real culture, watch how it behaves during an incident.
Below are what each one looks like in practice:
| Healthy incident culture | Fragile incident culture |
|---|---|
| Shared ownership | Individual blame |
| Clear communication | Confusion and duplicated work |
| Calm coordination | Panic-driven decisions |
| Fast escalation | Delayed reporting |
| Learning after incidents | Looking for someone at fault |
| Uncertainty stated openly | Hero engineers carrying it alone |
Three Situations That Can Reveal Which Culture You Have
1. The engineer who declares an incident before they're certain: On a healthy team, saying, "I think something is wrong with checkout latency, and I'd rather raise it early than be wrong too late," is completely normal.
Nobody is criticized if it turns out to be a false alarm. But on a fragile team, that same engineer spends fifteen minutes collecting evidence first because the social cost of being wrong feels too high.
2. The one person who knows the service best: Every team has someone who knows the payments service better than anyone else. The real question is whether that knowledge has been shared through documentation, pairing, and incident response, or whether the team has quietly become dependent on one person's availability.
Teams that rely on heroes are one resignation away from an outage they can't resolve.
3. The review that changed something: Plenty of teams write thorough post-incident reviews. Fewer can point to a specific change in how they operate that came out of one.
Why Your Daily Workflow Defines Your Culture
Culture is often seen as something shaped by values, hiring, and leadership, all of which focus on people. But under pressure, team behavior is influenced just as much by what their everyday tools and processes make easy.
DORA makes this point directly, arguing that culture changes when the way people work changes. They draw on John Shook's view of organizational transformation, which argues that lasting cultural change starts by changing what people do, not by trying to change how they think.
Under Pressure, People Do Whatever is Easiest To Do
An incident leaves little room for deliberate decisions. Nobody stops to read the values document at 3 a.m. People fall back on habit, and those habits are shaped by whatever the team's tools and processes make easiest. If the easiest option is to stay quiet and keep investigating alone, that's what a tired engineer will do, regardless of what the team says it believes about collaboration.
Every Workflow Decision Produces a Cultural Outcome
These mechanisms are important because each one comes from a workflow decision, and every one can be changed.
- Declaring an incident requires approval: Escalation carries a social cost, so people wait until they're certain, which is exactly when it is too late to be useful.
- The timeline is reconstructed from memory: Reviews become debates over what happened, and the person with the strongest memory often wins instead of the person who is right.
- Action items live in a separate document: Improvement work ends up competing with roadmap work while sitting outside the system where roadmap work is tracked, so it rarely gets prioritized.
- One person always leads the response: Everyone else waits for instructions, and the team never develops the habit of acting independently.
None of these decisions appear on a values page, but all of them shape how the team behaves.
Which is Why Incidents Test Culture Rather Than Create It
An incident doesn't create culture. It reveals the habits your team has already built under the pressure of real work. That's why teams are often surprised by their own behavior during a major outage, and why postmortem action items so often fade away within a few months. The team fixes what happened during the incident, but the everyday habits that shaped the response stay the same.
Three Practices Every Team Needs Before Anything Else
Three practices appear in every serious discussion of engineering culture, and any mature team should already have them.
They're the foundation, though, not the strategy.
- Blameless postmortems: Google's SRE book argues that postmortems should assume everyone acted with good intentions based on the information they had. When blame becomes the focus, people stop reporting problems for fear of being punished.
- Structured post-incident reviews: Use a consistent format with a timeline based on when customer impact began, contributing factors instead of a single root cause, and a small set of clearly owned action items.
- Defined incident roles: One person coordinates, one communicates, one investigates, and everyone knows their responsibilities.
A team can do all three well and still work in an environment that pushes against them. That's where the rest of this discussion begins.
How to Build Engineering Team Culture That Survives Incidents
Make Declaring an Incident Cheaper Than Deliberating
Remove any approval step. Let engineers declare incidents even when confidence is low, and treat cancelled incidents as a healthy outcome rather than a mistake. A team that regularly cancels incidents is often a team that catches problems early, and those cancellations are valuable learning opportunities.
Capture the Timeline as It Happens
A timeline reconstructed from memory is just a document. But a timeline captured during the incident is a record. The difference becomes obvious during the review, when a decision made at 03:14 is either documented or remembered differently by four people. Tagging key moments during the response takes seconds and eliminates hours of debate later.
Put Improvements Where the Work Already Lives
Following through on post-incident actions is one of the biggest challenges teams face, and the reason is simple. Improvement work stored in a separate document has to be copied into the system where work is actually planned, and that's often where it gets lost. Action items that become real backlog tickets with owners and sprint assignments get done. Action items left in a bulleted list usually don't.
Build the System So Nobody Has to Be the Hero
Document the information your team will need before an incident happens, including:
- Service ownership and who to contact for each system
- Key dependencies and known failure modes
- Runbooks and recovery steps for common incidents
- Escalation paths, including when to involve teams outside engineering
- Communication templates for customer and stakeholder updates
Common Mistakes That Undermine Engineering Culture
- Rewarding heroics: Reward preparedness instead, or the team learns that the way to be visible is to be indispensable at 4am.
- Blaming individuals: Improve the system that allowed the failure, because the next person in that position will make the same call.
- Skipping reviews when the incident was small: The cheap incidents are where learning costs least, and they are usually rehearsals for an expensive one.
- Writing documentation after the outage: Write it before you need it, since nothing written at hour three of an incident gets written well.
- Delaying communication until the cause is known: Communicate early and include the uncertainty, because silence gets interpreted as something worse than the truth.
- Tracking action items outside the backlog: Keep improvements in the work queue where they compete for priority like everything else.
The first example is both the most damaging and the easiest to miss. When leadership publicly praises the engineer who stayed up until 6 a.m. fixing an incident, but says nothing about the engineer whose runbook helped resolve the next one in twenty minutes, the team learns what is truly valued.
People copy what leaders reward far more quickly than what documentation tells them.
The Signals That Tell You It Is Working
None of these changes matter if you can't tell whether they're working. Fortunately, the operational side of engineering culture leaves measurable signals.
Four metrics especially are worth tracking:
| Signal | What it tells you |
|---|---|
| Near-miss and low-severity reporting rate | Whether people feel safe raising small things |
| Time from impact to incident declared | Whether escalation carries a social cost |
| Action item completion rate in a fixed window | Whether reviews change anything |
| Repeat incidents against contributing themes | Whether learning compounds |
The first metric often surprises people. How is an increase in reported minor incidents a good sign?
What it means is your monitoring has improved and engineers are reporting issues earlier instead of filtering them out. Over time, that tends to reduce the number of major incidents.
A team that reports almost nothing either has an exceptionally stable system or has created a culture where raising issues doesn't feel worthwhile.
These metrics complement the reliability measures most teams already track. MTTR tells you how quickly you recover from incidents. These metrics tell you whether your team is getting better at preventing them in the first place.
Most of these are trackable in whatever you already use. In Phoenix, they surface on the same dashboard as the recovery metrics, so the behavioral signals and the reliability numbers are read side by side rather than in separate reports.
Where Phoenix Fits
Phoenix Incidents runs incident response inside Jira and Slack, so the tools that shape your team's behavior are the same ones they already use every day.
Here's how specific features support the behaviors this post has discussed.
1. Timeline tagging:
This feature captures key events as they happen, so no one has to reconstruct the incident later from memory. React to any Slack message in an incident channel with the ⌚ watch emoji and Phoenix logs that moment to the incident timeline with its exact timestamp.
This solves the timeline capture problem, since nobody has to stop mid-response to write anything down, and the review a week later works from a record rather than four people's recollections.
2. RCA Review Status:
This is an optional status that keeps the technical write-up complete but still open for review by security, compliance, or leadership. The incident remains editable while feedback is incorporated, without prematurely marking it as finished, and operational metrics remain accurate throughout the process.
3. Monitoring Status:
Creating a formal stage between fixing the problem and calling it resolved, this stops the active resolution clock and signals to the business that the fire is out, while giving engineers room to watch the dashboards and confirm the fix actually held before the ticket closes.
Teams that skip this stage tend to record incidents as resolved that were not.
4. Lifecycle charts:
Recovery time is broken out by stage: Detect, Acknowledge (Ack), Verify, Fix, and Monitor. The data is broken down by product, tracked month over month, and reported as medians and distributions instead of averages so unusually long incidents don't distort the picture.
This makes it easy to see whether improvements are actually happening. For example, a team that has made incident reporting fast and low-cost will see Detect and Acknowledge times consistently fall over time.
5. Executive Summary:
Phoenix reads the completed RCA along with the incident details and generates a clear, non-technical summary directly in the same Jira issue. Engineers can keep the RCA as detailed as it needs to be, while leadership gets a concise briefing they can quickly understand.
That means the technical analysis no longer has to be simplified just to make it readable for non-technical stakeholders.
6. /phoenix recap:
An engineer joining the incident two hours after it began can type /phoenix recap in the incident channel and instantly see what has happened so far, what has already been investigated or ruled out, and where the response currently stands.
Instead of spending time catching up on Slack messages, they can start helping immediately. That makes bringing additional engineers into an incident much less disruptive, making early escalation and shared ownership practical instead of something teams only aspire to.
Explore these features in more detail on the Phoenix Incidents What's New page →
Build An Engineering Team Culture Before You Need It
The way your team responds during the next outage is decided long before the incident begins, through everyday decisions about how you work.
- Does declaring an incident carry a cost?
- Does the timeline capture itself?
- Did last quarter's action items actually get done?
None of that appears on a values page, but all of it becomes obvious at 3 a.m. The teams that perform best under pressure are the ones that built the right habits before the system went down.
Frequently Asked Questions
1. What does a healthy engineering culture look like during an incident?
A healthy culture keeps people focused on fixing the problem, not finding someone to blame. Engineers share facts early, ask for help, and follow clear roles while pressure is high. Afterward, the team reviews the system and decisions without using the meeting to punish people.
2. How can we build trust before the next incident?
Build trust through daily habits, not a speech after something breaks. Use clear ownership, shared runbooks, safe practice drills, and regular one-to-one talks about workload and risk. Leaders must admit mistakes, reward honest reporting, and act on issues raised by the team.
3. Do small engineering teams really need an incident process?
Yes, but a small team needs a simple process rather than a large manual. Define who leads, who communicates, how work is tracked, and when the issue is escalated. A short checklist can prevent confusion without adding needless meetings or paperwork.
4. How much does it cost to build incident-ready team culture?
The main cost is regular time for practice, reviews, training, and better tools. Most teams can begin with existing chat, ticket, and monitoring tools instead of buying new software. The larger cost comes from ignoring weak systems, repeated outages, staff burnout, and lost customer trust.
5. What are the most common mistakes teams make after an incident?
The biggest mistake is treating the person closest to the failure as the whole cause. Other common errors include skipping the review, writing action items no one owns, and fixing symptoms while leaving the system unchanged. A useful review names system gaps, assigns owners, and checks whether each fix worked.
6. What should we do after our next production incident?
Start with a calm review while the facts are still fresh, then record the timeline, impact, decisions, and open risks. Ask what made the incident harder to spot or fix, and give each useful action one clear owner. Share the lessons with the wider team, including changes that do not require code.