Set service recovery objectives
Define recovery time and recovery point objectives with service owners. Map dependencies such as identity, DNS, keys, networks, data and third parties.
Restore order matters. A database cannot serve customers if secrets, identity or network policy are unavailable. Build a dependency-aware runbook.
- Named business service owner
- Critical dependency map
- Acceptable data loss and downtime
- Manual continuity path
Separate backup trust
Use independent administrative roles, protected accounts and immutability or retention locks where appropriate. Monitor changes to backup policy and deletion attempts.
Keep recovery documentation accessible when normal collaboration platforms are unavailable. Protect copies from casual access because they contain sensitive data and architecture.
- Dedicated recovery roles
- Multi-person approval for destructive actions
- Offline or logically isolated copies
- Protected key recovery
Restore into a clean environment
Do not automatically return compromised systems to the same environment. Provide a clean-room pattern with known-good infrastructure templates, trusted identities and controlled connectivity.
Validate data integrity and security before reconnecting customers or partners. Recovery pressure is not a reason to remove change control; it is a reason to simplify it.
- Known-good infrastructure code
- Malware and integrity checks
- Temporary restricted access
- Evidence preserved for investigation
Exercise the business outcome
Run component restores frequently and full service exercises on a risk-based schedule. Include decision-makers, communications and suppliers—not only infrastructure engineers.
Record actual recovery time, missing dependencies and manual steps. Fund the fixes. A lesson that stays in the exercise report is not improvement.
- Random sample restore
- Annual or semi-annual service exercise
- Executive decision rehearsal
- Tracked remediation actions
A practical 30-day field plan
Week one — Protect. Versioned, immutable or isolated copies. Put one accountable owner in the room, agree which business service or decision is in scope, and record the assumptions the team is making. Resist the urge to begin with a technology purchase; the first deliverable is a shared view of the problem and the authority to change it.
Week two — Detect. Backup tampering and unusual deletion. Walk through the current process with the people who operate it. Compare the written design with real access paths, data flows, exceptions and on-call practice. Mark every point where an owner is missing or where the team cannot produce evidence that a control works.
Week three — Restore. Clean environment, identity and data. Choose a narrow pilot that can be observed safely. Define the expected result, the rollback path and the person who may accept a trade-off. Capture operational friction as product feedback; controls that are difficult to use will eventually be bypassed.
Week four — Prove. Business service test and measured objectives. Review the pilot with engineering, operations, security and the service owner. Close urgent gaps, assign longer work to a funded backlog and set the next evidence review. The month should end with a repeatable operating rhythm, not a one-time presentation.
Evidence worth keeping
Good evidence is understandable outside the team that created it. Keep a concise record that connects the decision, owner, technical implementation and observed result. Screenshots can support evidence, but configuration, logs, test output and approved records are stronger because another person can reproduce or challenge them.
- Named business service owner — owner, current state, last validation and any open exception
- Dedicated recovery roles — owner, current state, last validation and any open exception
- Known-good infrastructure code — owner, current state, last validation and any open exception
- Random sample restore — owner, current state, last validation and any open exception
- Decision log showing who approved residual risk and when it will be reviewed
- Test or exercise result with the actual outcome, not only a pass label
Questions for the leadership review
Use these questions to keep the discussion connected to operating risk rather than tool activity. A useful answer names a person, a service and evidence.
- Who is accountable for the cloud resilience outcome when teams disagree about delivery and risk?
- Which critical service or customer promise would be affected by a failure in this area?
- What evidence would tell us the design is working in production today?
- Which exception creates the largest concentration of access, dependency or recovery risk?
- What would the team contain first, and how would it restore a trustworthy service?
- Which improvement can be completed in the next 30 days without waiting for a large programme?
Common questions
Are snapshots enough for ransomware recovery?
Not by themselves. They may share production identities and control-plane risk; use isolation, immutability and tested clean recovery.
Who owns recovery?
Technology teams operate it, but the business service owner sets acceptable outcomes and participates in exercises.