Most production incidents start as a small assumption nobody wrote down. We wrote the migration twice: once to learn, once to ship. Small pull requests get real reviews, large ones get approvals.
Details
The best architecture diagram is the one the new hire can redraw from memory. Feature flags are debt with an expiration date you must actually enforce. Caches are easy to add and surprisingly hard to remove.
If the error message does not say what to do next, it is half an error message. Every retry needs a budget, otherwise it is just a slower outage.
Lessons
The rollback plan should exist before the rollout plan is approved. We measured first, and the bottleneck was not where anyone expected. The fastest query is the one you never send, but the second fastest has a good index.
- Measure p95 and p99, not just the mean
- Keep migrations reversible
- Write the failure mode down before writing code
- Delete dead code the same week
Security is mostly about saying no to defaults that were convenient in 2010. Naming things well saved us more time than any framework migration.
What didn’t
Security is mostly about saying no to defaults that were convenient in 2010. Most production incidents start as a small assumption nobody wrote down. We wrote the migration twice: once to learn, once to ship. Naming things well saved us more time than any framework migration.
The best architecture diagram is the one the new hire can redraw from memory. Security is mostly about saying no to defaults that were convenient in 2010.
Every retry needs a budget, otherwise it is just a slower outage. We measured first, and the bottleneck was not where anyone expected.
Leave a Reply