Every Data Tool Has Misled Me at Least Once
That afternoon, I had just finished a data script. It hit a 429 on the third URL and I froze.
429 is the HTTP status for too many requests. The server is rate-limiting you. I assumed I had made the script too aggressive, added a sleep, and tried again. Still 429. Three layers down in the documentation, in the small print, I found it: Clarity’s API, the interface that lets programs talk to each other, allowed only 10 calls a day. The two failed runs before this one had already used the day’s quota.
For a second, I had a ridiculous thought: again.
This was the fourth time in six months that a tool had led me astray. The first three were GA4, GSC, and my own head.
Give me eight minutes and you will know how to spot a tool’s blind spots, why one tool alone makes a bad basis for a decision, and the three-step verification routine I use now.
The four times I got it wrong
I looked back over the previous six months. At least four times, I nearly made the wrong call because I trusted a tool’s output.
The first was GA4. It fed me a pile of Singaporean bot traffic as if it were real visitors, and I nearly rewrote a blog post because bots had inflated its traffic. Its “automatically filter bots” switch only catches well-behaved crawlers that identify themselves from a known list. A crawler that can impersonate Chrome gets through. Worse, GA4 gives you no warning that it is unsure about that traffic. I wrote separately about spotting Singaporean bot traffic in three seconds, so I will not repeat that part here.
The second was GSC, Google Search Console, which is basically Google’s dashboard for site owners. URL Inspection told me several pages were missing a meta description, the short text under a title in search results. I was ready to add them. Out of habit, I used curl on the source. curl is a command-line tool that asks a page directly for its raw HTML, the source before the browser renders it. The meta description was already there, correctly formatted. GSC was looking at an older version because its data lagged by several days. Without that extra curl check, I would have spent time fixing a field that was already fixed.
The third was the most absurd. In another post, I had written it plainly: if GA4 sees 100 sessions and Clarity records 30, the remaining 70 are likely bots. When AI helped edit the draft, it compressed that judgment into GA4 ÷ Clarity < 50%. During review, I substituted the numbers: 100 ÷ 30 = 333%. It could never fall below 50%. The correct rule is Clarity ÷ GA4 < 50%.
I stared at the screen for a while. It was embarrassing. The prose above it was clear, so how had the formula been reversed when AI condensed it? Later I realised that prose follows the order in which terms appear, while a formula has to follow the order that produces a meaningful threshold. Those orders can differ. Nobody substitutes the numbers for you, and AI will not do it for itself. It can turn a sentence into a formula in rhetorical order and still miss that the direction is wrong. If I had not run the numbers, that mistake would have gone live.
The fourth was the Clarity quota from the opening.
A normal-looking summary is the dangerous part
These four tools were nothing alike. GA4 analyses traffic. GSC reports search performance. Clarity records sessions. The third error happened entirely in my own reasoning.
They misled me in the same way: the output looked normal, so I did not think to check it.
- GA4 does not volunteer that it is unsure whether a batch of traffic is human.
- GSC does not put “this is a five-day-old snapshot” at the top of a summary.
- Clarity did not show a line saying “the API stops at 10 calls a day.”
- When I write a formula, nobody appears to tell me that I have reversed the direction in my head.
It felt like cooking in a spotless kitchen. Everything is lined up neatly and the counter is polished. Yet every knife has a dull spot you cannot see, and every burner runs a little hot or cold. You think you are cooking, but you are negotiating with the kitchen’s blind spots.
There is another problem. The shape of these blind spots often matches your own blind spot. You do not question what you cannot see. They fit together like two puzzle pieces, and the join is exactly where you fail to notice the gap.
The routine I use before trusting a data conclusion
I now treat any data-based conclusion as something that needs a verification step. If you only look at one tool’s output, you are not seeing the truth. You are seeing the side of reality that tool is willing to show you. I usually do three things.
First, I remove bot traffic before drawing a conclusion. When traffic looks strange, I break it down by device resolution, country, and bounce duration. The earlier GA4 post explains the exact filters in more detail. My rule is simple: rejecting questionable traffic is safer than accepting it.
Second, I verify structural claims in the source with curl instead of trusting a tool summary. For a conclusion such as “meta is missing,” “canonical is wrong,” or “hreflang is absent,” I run curl -s <url> | grep -iE "meta|title|canonical" first. grep is another old command-line tool. It pulls out the lines in a large body of text that contain a keyword. GSC and tools such as WebFetch can lag, cache results, or strip script tags. Their summaries are useful reference points. The source is the fact I need to inspect.
There is a limit to that check too. curl retrieves the source before the page runs its scripts. If your meta tags are injected by JavaScript, curl can return an empty result and falsely report a missing field. My site was statically generated, so its meta tags were written directly into the source and this method was enough. Even the tool I use to verify other tools has its own blind spot.
Third, I substitute real numbers into every formula as soon as I write it. This rule is for me and for AI. Clear prose does not guarantee a correct formula, because compression into symbols can make people and models follow rhetorical order rather than mathematical order. When a sentence contains a ÷ or a > threshold, I plug in the nearest real numbers from the preceding text. A reversed direction becomes visible immediately. I now keep a separate review step called “substitution check” and do not skip it.
This takes about half an hour. Without it, every content decision rests on the slice of reality a tool happens to show you.
What I would do in the first week
If you run a content site and have just started looking at data, I would hold off on trusting any single tool.
- In the first week, run a basic report in each tool, then use another tool to inspect the same question. The larger the gap, the more you need a verification routine rather than another tool.
- Do not connect five analytics platforms on day one. Start by getting GA4, GSC, and Clarity working together. Their cross-checks are enough for the first six months.
- Before writing any conclusion that “X rose” or “Y fell,” ask whether the same number can be reproduced from another angle. If it cannot, hold the conclusion.
Back to the Clarity quota. I combined every dimension into one API call: URL, device, and country together. I then saved the raw JSON, the structured data returned by the API, locally. During later debugging, I read the local cache instead of using another call from that day’s quota. The change took twenty minutes. The job finally ran through, and I stopped working under the pressure of having only 10 attempts a day.
My view now is straightforward: tools do not tell you where they are unreliable. Using data well depends less on how many tools you connect than on whether you leave room for a verification step.
If a tool gives you a number that conflicts with your instincts, pick one of these checks and use it first. Run it for a week, then decide whether to add the other two. I have probably been caught by the same tool already.