Sorry, Wrong Numbers: Yet Another Flawed Study On Mobile Marketing

by Freddy J. Nager, Founder of Atomic Tango LLC + Guy Who Likes To Question The Answers

I used to read and recommend eMarketer, simply because they have numbers and pie charts and other examples of statisticulation. That made their articles seem more rational and realistic than 98.76%* of the blogposts written by social media gurus, who like to turn their personal whims and opinions into “10 rules for this” and “20 best practices for that” with no data to substantiate their arguments whatsoever.

But this morning I came across this chart of “US Internet users who use their mobile devices to purchase products online.”  It showed that a majority of the survey respondents had made a purchase (not just apps) using their mobile phones.

Does this pie look a little half-baked to you?

Hmmm. Interesting. But that sounded kind of high to me given the number of kids with mobile phones but no credit cards, and the number of senior citizens with mobile phones and no desire to use them for anything but phone calls.

So I made the same request I always make when someone throws percentages at me: Show me the raw numbers!

Oddly, there were no raw numbers in the eMarketer article. So I went off to play detective…

The article referred to some company called AYTM Market Research. Sounds impressive, but then I hit the website and read that AYTM stands for “Ask Your Target Market.” This wasn’t Nielsen or McKinsey – it’s a survey service. Like SurveyMonkey, whose name is more fun and indicative of what they really do.

I couldn’t readily find the blog on the AYTM site (navigation not being their strong suit), so I had to go back to Google to find “AYTM Blog.” I finally found it and the story about mobile users.

And there were no raw numbers.

But WAIT: in a graphic was a button to “Open Full Report.” So I did and finally found the dirty details.

And they were positively filthy.

I went straight into the respondent profiles, and this is what I found:

• First, the survey had only 200 respondents, which is not statistically significant for a market as large as all mobile users in the United States.

• Then, it turns out 61.5% of the respondents were women. From my thankfully-long-gone days as a single guy, I know the ratio of women to men in the United States is not 3 to 2, so I’d find it hard to believe that 61.5% of cellphone owners are female.

• Then it stated that 63.5% of the respondents fell between the ages of 18-44. Likewise, the Millennials are a large generation and they do love their phones, but even if you throw in the tail end of Gen X, they’re not 2/3 of the U.S. population (they account for about 25%) or of mobile users. And the survey didn’t cover anyone younger than 18. So I guess that means high school kids in the U.S. must not have mobile phones.

• Other data from their survey respondents:
  2.4% were Hispanic/Latino (the U.S. population is about 16% Hispanic/Latino)
  40% were employed full time and 25% were unemployed (not counting students). I know the U.S. economy is still hurting, but not to that degree.

So in other words, this was not a survey of U.S. mobile phone users. It was a survey of mostly underemployed young women who take surveys online, likely for the money they get paid to do so.

If that’s your target market, this is the study for you.

The survey also erred by omission: it didn’t say what these young women purchased. An MP3 or a vacation to Hawaii? A movie ticket or a big screen TV? A pair of socks at Target or a suit from Prada?

Now perhaps it is true that the majority of Americans have bought something online using their mobile devices, but I wouldn’t use this study to support that argument. And I feel sorry for any business that bases its strategies on such a survey… “Hey, Bob, we should shift more of our budget to mobile, because according to this chart, look at all the Americans buying stuff using their phones!”

Most of all, I’m disappointed in eMarketer. I don’t expect application providers like AYTM to have the best marketing reports. But I do expect a filter and influencer such as eMarketer to scrutinize everything before they pass it on as “truth.” To their credit, they responded to my nitpicking on their Facebook page, so they may clean up their act in the future.

In the meantime, question anything that anyone posts online. It’s true only about 16.28%* of the time.

*Numbers generated by statisticulation and pure whimsy.