Lies, Damned Lies, and Statistics

Don’t believe everything that companies tell you.

Very impressive email, HubSpot. You obviously worked hard on it. One small problem – it’s not true. Those numbers are fabricated. Someone outed you in post entitled Lead Generation : How Some Marketing Software Firms Stretch the Truth.

Laura Ramos, formerly with Forrester Research and now with Xerox actually downloaded and read the survey. She was impressed with the quality of the study – better than most she had seen. “But what bothered me was how the vendor-sponsor exaggerated the results to a degree that I felt were unnecessary.”

Laura recommends people look at three things when reviewing vendor claims:

What the base? – How many participated in the study. The HubSpot headline claims 5,000, but the survey had 214 respondents and most were small companies.
Where did 5,000 come from HubSpot?
Does the headline stand up when you scrutinize the data? – You need to compare the headline to the details. For instance, small numbers and lack of time frame can skew the data. The study results (page 12) show that 83% of the respondents and 32% of those show their leads increased 50%. But who are those respondents and how long did it take to get those results? That important data was omitted.
How credible is the source/authors? – Who wrote the report? Are they experts or not. In HubSpot’s case, it appears that the report was done by two MBA graduate degree candidates. While they did a fine job, does this give Hubspot the right to link the MIT brand to the report? That is quite a stretch.

The bottom line is that marketing automation is highly competitive and vendors find very tempting to highlight customer results to catch the eye of busy marketing executives. But don’t believe everything you read. Successful B2B marketing today is hard work, and requires great processes and awesome content. As Laura says, software should be a distant fourth on the list. And we at Find New Customers agree with Laura.

What do you think? Do you know of vendors who skew the data? We love comments.

Jeff Ogden (@fearlesscomp) is the President of the B2B lead generation consultancy Find New Customers. Find New Customers helps companies dramatically improve revenue results by transforming the way they attract, engage and win new customers. Contact Find New Customers by calling (516) 495-9350 or sending an email to sales at findnewcustomers.com.