Score It

When a data partner tells you their results are hard to judge, they've just told you exactly what to measure.

This year we looked at a provider that matched people to the companies they work for. It's a useful capability. The pitch was good, and the team behind it was sharp.

Before anything goes into our product, though, every provider goes through some kind of test. We send them a file we already know the answers to, and we score what comes back. Not their sample. Ours.

The results weren't strong enough to carry into the product, so we passed.

Their reply was fair. This kind of matching is never perfect, they said. You'll always lose some records along the way.

Our data lead's answer is the line I've kept: we know. That's why we scored it instead of assuming it.

The test isn't about catching anyone out or making a partner jump through hoops. It's the only way I've found to see what's actually there.

On the surface, this market is hard to read. Every week, companies reach out saying they can name every visitor to your website, or that they're a cheaper version of the big contact databases. Every provider says they're better or different. Some tell you another data company already uses them, and then that company tells you it's the other way around. From the outside, you can't tell who has data that fills a real gap, who is reselling someone else's, and who actually has the rights and the closest connection to the source.

Both sides lose if you guess wrong. We both want a deal, but more than that, we both want a fit. If we sign on the strength of a good pitch and find out a couple of months in that the data doesn't hold up, the relationship ends. The buying process was wasted, and we're back to finding another provider to partner with. The test is how both sides avoid doing the work twice.

So we test before we commit, whether that's a file built around a gap or a trial run of the integration. The market and a provider's marketing are how we hear about them. They're not how we decide. When we were looking at any type of data, a similar file could go out to three shortlisted providers at once, scored the same way, so we compared results side by side instead of pitch against pitch. This week, one file went to four. One ruled itself out within a day: it couldn't cover a region we need, and we never got as far as price.

Here's what I think is going on

A methodology debate is a conversation the partner will always win. They know their pipeline far better than you do, and they can explain any weak result. A scoring test is one you can't lose, because you know your own file.

Score it, don't argue it.

It matters because of where it sits. Fit decides whether a partnership is worth exploring. Terms decide what it costs. Proof is where you decide whether to believe it. Skip it, and every stage after inherits an assumption: you size a commitment, design a screen and set a price on data you never actually tested.

It also makes the hard conversations easier. When a score decides, nobody has to win an argument. The partner sees the same result you do, and a no lands as a fact, not an opinion. You don't need to justify why you didn't move forward, and they don't need to justify it to their team either.

I do the same thing outside work. Some afternoons I have a little dark chocolate. Not the 100% kind, just something less sweet than milk chocolate. I didn't keep the habit because of anything I read about it. I kept it because I'm sharper on the days I have a small amount and sluggish on the days I overdo it. My own results, on my own days, decided it.

It has limits, and it's easier said than done. Building a file you already know is the real work. A sample shows you the floor, not everything: freshness over time and rare segments need a second look after launch. And a strong score doesn't fix bad terms or a feature nobody can find. It earns the next conversation, nothing more.

So before an evaluation now, I build the file from what our users actually search for, send the same rows to every provider in that gap, and decide what passes before anything comes back. Then I let the score have the argument.

Which brings me back to that first partner. They were right that matching is never perfect. But the question was never whether it's perfect. It was whether it's good enough, on a file you know.

Marc

P.S. What test do you run before you trust someone else's data? Reply if this reached your inbox, or write to me through the Contact page. I read every one.