Observability in the Development Pipeline

How Slack Used Observability to Increase Trust in Software Testing

Table of Contents

  1. Setup and Benefits Summary
  2. Challenge or Opportunity
  3. Solution
  4. Process/Approach
  5. Result
  6. Retrospective and Lessons Learned
  7. About Jon Collins

1. Setup and Benefits Summary

After very fast growth, collaboration tools provider Slack found its application testing teams weighed down under increasingly complex infrastructure, impacting test reliability. Up to 50% of tests were deemed inconclusive—or, in the parlance, flaky—which meant development teams could no longer rely on the outputs of the testing process.

When Slack brought its observability solution to bear on the infrastructure used for software development and testing, the company found it an effective way to pinpoint, diagnose, and solve individual infrastructure issues one by one. But, given the lack of trust in testing results, gaining buy-in across multiple teams was a challenge. This was addressed following a results-driven, show-don’t-tell approach that involved the teams in decision-making.

Overall benefits of deploying observability pre-deployment were:

  • A reduction of flaky test results from 50% to 5%
  • A removal of 45,000 hours of wasted server time per week
  • Increased developer confidence and productivity

Full content available to GigaOm Subscribers.

Sign Up For Free