Open handbook navigation

Situation playbooks

The team doesn't trust the data.

How to restore team confidence in data: metric inventory, definitions, event quality, owners, and transparent checks.

The phrase “data cannot be trusted” usually hides several different problems: divergent definitions, loss of events, opaque transformation, or an inconvenient conclusion. Choose one critical metric, trace it from event to report, and publicly record the limitations.

How does distrust of data manifest itself?

Typical signals

The most common is that the team argues with the results of the analytics or asks for a recheck. Phrases like:

Let’s check the calculations again, it seems that the data do not match our experience.

Also noticeable are the ignoring of dashboards, parallel manual calculations in Excel, or discrepancies between different sources.

Example

In the e-commerce project, the analyst prepared a conversion report. The team did not accept the conclusions – in the discussion, most referred not to the numbers, but to the feelings of the customers. The decision was postponed and the “safe” path was chosen.

Causes of the problem

Mistakes and anti-patterns

Most often, trust falls for these reasons:

  • No transparency: it is unclear how the data was collected and processed.
  • Different sources show different numbers.
  • Low quality data or artifacts (skips, duplicates, strange emissions)
  • Poor communication – data does not explain the reason for the connection with the business.

Example

We launched a new feature flag and analyze its effectiveness. The old analytical report shows one thing, the new tool shows another. No one can explain the difference. The result is that no one believes the old or the new result.

Quick Action to Restore Trust

Check the base.

First, explain in simple words where the data came from and how it was collected. Check the metrics for validity: there are no gaps, strange jumps, out-of-synchronization between systems. Show the process of obtaining data to others, do not hide the details.

Example of pattern action

Instead of a general table, disassemble the structure: queries from the SQL analyst, step by step check the sample, explain where and how the information comes from. Show it to the team, so there’s less room for speculation.

Get a quick cross check.

Try to check the conclusions in another way: check with raw logs, add time charts, look at the metrics in the context of time. This helps to quickly identify distortions.

How to create an atmosphere of trust for a long time

Transparency and regular review

Explain not only what the numbers show, but how they are collected. Make regular checks, especially if new tools or updates to the collection logic appear.

Simple rules for the team

Have regular data analysis meetings where anyone can ask a question. Figuring out discrepancies between sources, keeping a low profile on mistakes, is the key to long-term trust.

Mini-practice

In retrospect or in daily terms, show how a team can quickly reconcile the calculation with multiple independent sources.

What to Avoid: Frequent Mistakes

Isolation of the process

If analysts are closed, don’t show the process, trust collapses. Calculations and formulas should not be for only one person.

Concealing inconsistencies

If you hush up the discrepancies in the metrics or just digest the errors on the sidelines, it will quickly notice – the team will start doing double work and wasting time.

What to look for in metrics and where to look for benchmarks

Key indicators

The specific values are highly dependent on the market and the product. It is usually useful to monitor: completeness of data, number of collection errors, delays/lag, number of disputed situations. Benchmarks and best practices for data quality can be found in the directories dbt Labs, Google Analytics Help.

Open approaches

Many companies publish their data auditing techniques and checklists. For example, Amplitude Data Taxonomy Playbook or Google Data Quality Guidelines.

FAQ

Why does the team stop trusting the data?

opaque collection and processing, frequent errors, and contradictions between sources. This is usually noticeable when metrics are not explained to a business in simple words.

How quickly can you restore credibility to the report?

Show me how the sample is formed, let the team step by step verify the data. Check the results with another independent source.

What tools help verify data?

SQL queries with comments, data audit scripts, data quality monitoring (e.g. dbt, Amplitude, Google Data Studio).

What if there is no single version of the data?

Make a summary meeting, collect all versions of the reports, discuss the reasons for the discrepancies, implement a single compliant logic of collection.

How to reduce the number of controversial situations in analytics?

Regular analysis, transparent methodology, open-source reports, and independent validation (e.g., manual checks or parallel logs).

What are the signs of a good level of trust in data?

Rare disputes on basic metrics, discussion of conclusions and decisions, the presence of a unified methodology for data collection.