AI & Machine Learning

Instrumenting a Distributed System You Can Actually Debug

James Chen 1 min read

Traces, structured logs and the few metrics worth alerting on – and why dashboards are not observability.

Executive Summary

  • Dashboards answer questions you already thought to ask; traces answer the ones you have not.
  • Structured logs with a shared request id are worth more than any number of counters when an incident is already running.
  • Alert on symptoms your users feel - latency, error rate, saturation - and leave everything else as evidence, not a page.

Where teams usually start

Most teams arrive at this problem from the same direction: something that worked at one scale stopped working at the next, and the fix that used to be obvious no longer is. This guide walks the path we take with clients, in the order we take it.

The approach

Work in slices that can each ship on their own. A slice that cannot be deployed independently is not a slice, it is a phase – and phases are where migrations go to die.

What to measure

Decide the numbers that say it worked before you start, and instrument them first. Retrofitting measurement onto a finished migration tells you nothing about what the migration changed.

Conclusion

None of this is exotic. It is the discipline of small, reversible steps applied to a problem large enough that skipping the discipline is tempting.

Resources & Insights

Technical research and guides.

Whitepaper
Guide
White Paper

Heimler CRM

February 04, 2026 Read Now →
Report