Our on-call rotation improved the day we deleted half of the alerts. Security is mostly about saying no to defaults that were convenient in 2010. Tests are documentation that fails loudly when it goes stale.
Context
Caches are easy to add and surprisingly hard to remove. The fastest query is the one you never send, but the second fastest has a good index. Feature flags are debt with an expiration date you must actually enforce. Security is mostly about saying no to defaults that were convenient in 2010.
Tests are documentation that fails loudly when it goes stale. Latency percentiles tell a story that averages politely hide.
The short version
Our on-call rotation improved the day we deleted half of the alerts. The fastest query is the one you never send, but the second fastest has a good index. Every retry needs a budget, otherwise it is just a slower outage. Tests are documentation that fails loudly when it goes stale.
- Pin versions, update on purpose
- Write the failure mode down before writing code
- Delete dead code the same week
- Keep migrations reversible
Latency percentiles tell a story that averages politely hide. If the error message does not say what to do next, it is half an error message.
A small example
Our on-call rotation improved the day we deleted half of the alerts. Caches are easy to add and surprisingly hard to remove. If the error message does not say what to do next, it is half an error message. Latency percentiles tell a story that averages politely hide.
Security is mostly about saying no to defaults that were convenient in 2010. Latency percentiles tell a story that averages politely hide.
A boring deploy pipeline is a feature, not a lack of ambition. Every retry needs a budget, otherwise it is just a slower outage.
Leave a Reply