Picture a ground-control room at three in the morning. A telemetry line goes quiet, a few heads turn, and within a minute somebody says out loud that this is now a situation. Nobody argues about whose job it is to speak next. That quiet choreography is what engineers mean by IncidentOps, the craft of handling unplanned service disruptions from the first declaration to the last follow-up. Space people have lived by these habits for decades, and software teams borrowed them on purpose. Here are eight of them, told through the stories astronomy and spaceflight readers already know.
1. Say the word early
Observatories do not wait for a telescope mirror to crack before they log a fault. A small anomaly gets written down while it is still small. IncidentOps asks for the same reflex: write down criteria for what counts as an incident, and call it when the criteria are met rather than when everyone feels brave. One example from the Google-style playbook is a trio of triggers, namely that a second team is needed, that customers can see the problem, or that an hour has passed without a fix. A declared incident that turns out minor wastes a few minutes. A late declaration can burn hours.
2. Give one person the wheel, and keep their hands off the engine
On a launch day the flight director does not tighten bolts. They decide, coordinate and keep the room in sync. In incident work the equivalent seat is the incident commander, and the Atlas is blunt that this person is not a resolver. The people allowed to change anything in production are the operations lead and the specialists working under them. Everyone else watches, advises or stays out. It feels slow for about ten minutes and then it feels like the only sane way to run a night.
3. Treat roles as hats, not titles
An amateur astronomy club might have a treasurer who also runs the star party. For one evening she wears the host hat, and nobody confuses that with her job description. Incident roles work the same way. A junior engineer can be the commander for an hour if they are the highest-ranking person actually on the call, and a director can happily be a spectator. Rotating the hat spreads the skill around, which is exactly what you want before a really bad night arrives.
There is a gentle parallel with a satellite parked in a fixed slot. A simple explanation of how geostationary orbit keeps a craft over one spot shows why a position stays useful only while someone keeps nudging it back into place. A role is similar. The seat stays put, but the person sitting in it changes with each shift, and the seat only works if everyone knows its duties by heart.
4. Keep the team small enough to hear each other
Emergency-management guidance suggests a supervisor should oversee around five people. Past that, split into sub-teams with their own leads. One public incident-response guide even pre-names its sub-teams with the phonetic alphabet so nobody wastes time inventing labels under pressure. Think of the way a crowded star party breaks into small groups around each eyepiece. Nobody can narrate a whole sky to forty people, but a handful will follow along just fine.
5. Fix the symptom before you argue about the cause
When a satellite tumbles, the first move is to stabilise it. The autopsy comes later. IncidentOps calls this mitigating before explaining. Rolling back a change, draining traffic from a sick region, switching off a feature or shedding load are all generic moves that need only a sense of where the fault lives, not a proven root cause. Teams that insist on a full explanation before touching anything are choosing a longer outage and calling it rigour.
6. Speak on a schedule
Space agencies hold regular press briefings during a mission because silence breeds rumours. The incident equivalent is a communications lead who posts on a clock. The Atlas outlines a rhythm in which the first public note lands within about five minutes of the call starting, a scoping update follows a few minutes later, and fresh posts keep arriving at least every twenty minutes during the first two hours. A closing note states plainly whether any data was lost. Status pages usually walk through four simple states: investigating, identified, monitoring and resolved. The cadence numbers here come from the Ops Atlas entry on IncidentOps, which gathers them from several public playbooks.
7. Hand over out loud
Mission control runs in shifts, and the hand-off between flight controllers is a ritual with a spoken confirmation. IncidentOps borrows it. When one commander leaves, the successor hears a clear sentence such as “you are now the incident commander, okay?” and answers yes. Silence is not consent. Without that exchange you can end up with two people steering, or none at all, which is the kind of mistake that tends to surface at the end of a long overnight shift.
8. Learn without blaming
After every anomaly, astronomy and spaceflight teams run a review that stays fixed on conditions rather than culprits. The Atlas quotes the view that hindsight biases how we judge human performance after the fact, and it recommends stopping at system conditions instead of writing “human error” in the summary. Keep the action list short, give each item one owner and a due date, and name the review owner before the incident call even ends. Several well-known teams set deadlines of a few days for the review meeting itself and a few weeks for the priority fixes.
What these eight habits have in common
Look back over the list and a theme appears: every habit trades a little ceremony for a lot of calm. Declaring early costs a sentence. Naming a commander costs a moment of awkwardness. Scheduled updates cost a few minutes of typing from someone who is not fixing anything. In return the room stops duplicating effort, outsiders stop pinging the responders for news, and the person who finally wakes up to read the thread can work out what happened without a phone call.
That is also why the discipline is described as coordination rather than technical wizardry. The Atlas places incident handling between the world of alerts and monitoring, which decide what gets noticed, and the world of long-term repairs, which remove causes over weeks. IncidentOps lives in the stretch between those two: the hours in which humans must organise themselves around a live problem, and it is the part most teams never rehearse.
One volunteer livestream, replayed
Suppose a volunteer team that streams a live meteor shower to thousands of viewers. Ten minutes into the peak the feed freezes. In the old days three people would have rushed to the same laptop, each restarting something different. In the new way, one volunteer calls it, a second becomes the commander, a third posts to the chat every few minutes to say what is happening, and only one person touches the encoder. They roll back to the backup stream before anyone knows why the first one died. The viewers barely notice. The explanation, which turns out to be a dull configuration slip, gets written up the next afternoon.
If you want the fuller background on where these ideas come from, including the lively disagreement about when the firefighting command model was first developed, the linked entry is worth an evening, and our own beginner pieces such as this clear tour of the stages of a spacecraft mission show how much of space operations already depends on phases, roles and handoffs. The pattern is old. IncidentOps simply gives it a name for the people who keep websites and services running.