Carrier-Grade Is a Culture, Not a Spec Sheet
Redundant hardware fails in redundant ways when the team behind it isn't ready. High availability is a habit before it is a number.
Untested redundancy is a rumour
Dual power, dual uplinks, dual routers — none of it matters until you have failed it over on purpose. Schedule the failure. Watch what breaks. Fix the runbook. Repeat until it is boring.
Runbooks over heroes
If an outage needs one specific person awake, you have a single point of failure with a phone number. Write it down, rotate on-call, and make sure the second-best engineer can fix it at the worst possible hour.
Blameless, not toothless
Post-incident reviews should find the system failure behind the human one — and still end with owners and dates. Psychological safety and accountability are not opposites; mature teams need both.
Monitoring is part of the design
Every new service ships with its alarms, dashboards and SLA definitions on day one. If we can't see it, we haven't finished building it.
Velocity at the edge, stability at the core
Move fast on products, portals and automation. Change the core only with a maintenance window, a peer review and a rollback plan. Knowing which is which is most of the job.
- #SRE
- #On-call
- #Runbooks
- #Teams
Working on something similar?
I'm always open to comparing notes with operators, partners and boards.