Our on-call rotation improved the day we deleted half of the alerts. The best architecture diagram is the one the new hire can redraw from memory. Naming things well saved us more time than any framework migration.
Getting started
Feature flags are debt with an expiration date you must actually enforce. Small pull requests get real reviews, large ones get approvals. Logs answer questions you knew to ask; traces answer the ones you did not. If the error message does not say what to do next, it is half an error message.
Most production incidents start as a small assumption nobody wrote down. Feature flags are debt with an expiration date you must actually enforce.
Lessons
The best architecture diagram is the one the new hire can redraw from memory. Security is mostly about saying no to defaults that were convenient in 2010. A boring deploy pipeline is a feature, not a lack of ambition.
Our on-call rotation improved the day we deleted half of the alerts. The rollback plan should exist before the rollout plan is approved.
Trade-offs
We measured first, and the bottleneck was not where anyone expected. Every retry needs a budget, otherwise it is just a slower outage. The fastest query is the one you never send, but the second fastest has a good index. Caches are easy to add and surprisingly hard to remove.
Logs answer questions you knew to ask; traces answer the ones you did not.
Most production incidents start as a small assumption nobody wrote down. The best architecture diagram is the one the new hire can redraw from memory.
Feature flags are debt with an expiration date you must actually enforce. Logs answer questions you knew to ask; traces answer the ones you did not.
Leave a Reply