Observability: Mistakes I Keep Seeing

Abstract code gradient artwork

If the error message does not say what to do next, it is half an error message. A boring deploy pipeline is a feature, not a lack of ambition. The fastest query is the one you never send, but the second fastest has a good index.

Going further

Naming things well saved us more time than any framework migration. The best architecture diagram is the one the new hire can redraw from memory. If the error message does not say what to do next, it is half an error message. We wrote the migration twice: once to learn, once to ship.

We wrote the migration twice: once to learn, once to ship. The fastest query is the one you never send, but the second fastest has a good index.

Lessons

Naming things well saved us more time than any framework migration. We measured first, and the bottleneck was not where anyone expected. Security is mostly about saying no to defaults that were convenient in 2010.

The best architecture diagram is the one the new hire can redraw from memory. We measured first, and the bottleneck was not where anyone expected.

Context

Logs answer questions you knew to ask; traces answer the ones you did not. Tests are documentation that fails loudly when it goes stale. Our on-call rotation improved the day we deleted half of the alerts. Feature flags are debt with an expiration date you must actually enforce.

Tests are documentation that fails loudly when it goes stale. If the error message does not say what to do next, it is half an error message.

If the error message does not say what to do next, it is half an error message. Most production incidents start as a small assumption nobody wrote down.

Comments

Leave a Reply

Your email address will not be published. Required fields are marked *