Five Things I Check Before Trusting a Dashboard

Somebody shares a dashboard. The numbers look clean, the charts are pretty, the executive is nodding. Everyone seems to agree it’s telling the truth.

I’ve learned to be skeptical.

Not cynical — skeptical. There’s a difference. Cynical is “dashboards are always wrong.” Skeptical is “let me check a few things before I act on this.” Over the years I’ve developed a short checklist I run before I let any report inform a real decision. It takes maybe twenty minutes and has saved me from embarrassing mistakes more than once.

Here are the five things I check.

1. When Was the Data Last Updated?

This sounds obvious. It isn’t. I’ve seen dashboards presented in Monday morning meetings that were last refreshed on Friday afternoon, or quarterly reviews built on data that stopped updating three months ago because an API key expired and nobody noticed.

Most BI tools show a “last updated” timestamp somewhere — often tucked in a corner or visible only if you hover. Find it. Verify it makes sense for the data you’re looking at.

If the dashboard shows “updated 14 days ago” and you’re about to make a staffing decision based on it, you need to know that. The data might still be valid. It might not. But you should know.

What to do: Check the refresh timestamp. If it’s stale, trace back to the pipeline and figure out why it stopped. Don’t present stale data as current without flagging it explicitly.

2. Does the Total Match a Source I Trust?

Pick one key metric on the dashboard — total orders, total revenue, total users, whatever the central number is. Now go find that same number in a source you have direct access to: the database, the platform’s own reporting interface, an export you pulled yourself.

Do they match?

They often don’t, and the reason is always interesting:

Different date filters (dashboard uses order_date, source uses shipped_date)
Different definitions (dashboard counts “completed” orders, source counts all orders)
Timezone handling (dashboard is UTC, source is EST)
Deleted records (dashboard pulls from a soft-delete table that includes canceled orders)

None of these are necessarily wrong — but they need to be understood. A dashboard where “total revenue” means something subtly different from what the stakeholder thinks is a problem waiting to surface at the worst possible moment.

What to do: Cross-check one number against a trusted source. Trace any discrepancy to its definition. Document the difference.

3. What Happens at the Edges?

Look at the chart’s extremes: the earliest dates, the most recent dates, the smallest values, the largest outliers. Edge cases are where data quality problems live.

Specific things I look for:

Flat lines at the beginning or end of a time series — often means a pipeline didn’t start collecting until a certain date, but the chart extends back to a default start date and fills with zeros or NULLs that got cast to zero
Suspiciously round numbers — 100,000 users is almost certainly a default value or a cap somewhere, not a real count
Sudden spikes followed by immediate drops — could be real (a promotion, a launch) or could be duplicate data that got partially cleaned
Missing recent data — if the chart goes to yesterday but you’re looking at it on Tuesday and Monday looks empty, the pipeline might have a lag

What to do: Zoom into the edges. Click into the last few data points. If the chart tool lets you see the underlying data, look at it.

4. Is the Metric Definition Written Down Somewhere?

“Active users.” “Qualified leads.” “Completed orders.” “Revenue.”

These words sound like they have obvious meanings. They don’t. Every organization has its own private definition of each, and those definitions evolve over time in ways that rarely get documented.

Ask: where is this metric defined? Is there a data dictionary, a README, a comment in the SQL view that powers this chart? If the answer is “in Steve’s head,” or “I think it means X,” that’s a problem.

The most dangerous dashboards are the ones where everyone thinks they understand what the numbers mean, but nobody has written it down. Two people can look at the same chart, nod at the same number, and walk away with completely different understandings of what it represents.

What to do: Find or create a definition for the key metrics. If no documentation exists, write a one-liner and put it somewhere visible. Future you will appreciate it.

5. Has Anyone Validated It Against Known Ground Truth?

At some point, every dashboard should have been validated against reality. Not just “it looks about right” — actually spot-checked against a source of ground truth.

For an ecommerce revenue dashboard: does the total match the accounting system at month-end close? For a lead pipeline: does the count match what the CRM exports directly? For a product analytics dashboard: does the funnel conversion rate match what you see when you manually walk through the product?

If the answer is “we’ve never actually checked,” that’s not the same as “it’s correct.” It’s “it’s unchecked.” Those are different things.

What to do: Pick a historical period and validate the dashboard numbers against your most trusted source for that period. Document what you find. If there’s a discrepancy, trace it. If it validates clean, document that too — future auditors will want to know.

The Meta-Point

I’m not saying dashboards are bad. I use them constantly. Good dashboards, with well-defined metrics and reliable data pipelines, are genuinely powerful tools.

But a dashboard is an abstraction. It hides the underlying data behind charts and numbers and color coding. That abstraction is useful — until you forget it’s an abstraction and start treating the chart as the source of truth.

Twenty minutes of verification before a big decision is cheap. Building a strategy on a stale, misdefined, unvalidated metric is expensive.

Check before you trust. Then trust with confidence.

— Matthew