Does anyone really know how many games are sold?

Last week, market research firm the NPD Group released its figures for video game sales in December and all of 2006. This resulted in a flurry of posts and reports on gaming sites and in the mainstream press, as always happens when NPD numbers are released. I’ve been wondering for a while why a market research firm is the basis for the industry’s tallies, and some questions raised by a video game analyst make me wonder even more.

Michael Pachter of Wedbush Morgan Securities tells GamesIndustry.Biz that the NPD numbers for Wii sales don’t make sense:

“There was a discrepancy between the number of Wii units claimed to have been shipped by Nintendo to North America — approximately 2 million — and the cumulative sell-through measured by NPD — approximately 1.1 million. This difference alone is almost impossible to reconcile.”

He says the same thing in interviews with other Web sites, and adds this in his comments to GameDailyBiz that he’s skeptical of NPD’s December numbers:

“The NPD data suggests that combined November and December software sales (including PC software) were up 6%. In contrast, retailer GameStop reported last week that its software sales were up 18% for the November-December period. While it is possible that GameStop gained significant market share during the holidays, comments from competitors Best Buy and Circuit City suggest to us that GameStop gained only slight market share. If GameStop did not gain significant share, the NPD’s reported figures may be far below the actual sales generated in December,” he remarked. “The discrepancy between the NPD’s reported figures and GameStop’s sales report is difficult to reconcile, and we will be forced to wait for earnings reports from the major publishers before we can explain the difference.”

Even if Pachter is right, it’s possible that NPD was off only for one month’s worth of numbers (though it is the biggest month of the year in terms of game sales). But I have a feeling this isn’t a first. According to Next Generation, NPD’s numbers “are based on reports from around 60 percent of US retailers.” Sixty percent doesn’t seem like a big enough sample to give accurate numbers. And are those “reports” anecdotal surveys or hard numbers based on receipts? In other words, does the way NPD tracks video game sales give an accurate account of actual sales, or is it just an estimate that’s not altogether helpful?

The music industry’s numbers come via Nielsen SoundScan, which tracks actual sales and is the source for the Billboard charts. According to Nielsen, “Sales data from point-of-sale cash registers is collected weekly from over 14,000 retail, mass merchant and non-traditional (on-line stores, venues, etc.) outlets.” The company doesn’t say what percentage of music sellers 14,000 outlets is, but that sure seems like a lot. The Recording Industry Association of America also keeps detailed track of sales to certify gold, platinum and diamond records through an “independent sales audit of each title by Gelfand, Rennert & Feldman, a highly respected accounting firm. … The RIAA®’s certification levels are based on unit shipments (minus returns) from manufacturers to a wide range of accounts, including non-retail record clubs, mail order houses, specialty stores, units shipped for Internet fulfillment or direct marketing sales, such as TV-advertised albums.”

The movie industry gets its North American box-office figures from movie theaters’ actual sales. Two companies, Rentrak and Nielsen EDI’s Flash service, get the numbers directly from theaters and give them to the movie studios, according to this Slate article. The trackers apparently cover 90 percent of theaters, which is a lot more comprehensive than NPD’s 60 percent (and again, if it’s 90 percent of theaters’ actual sales versus 60 percent of game retailers’ survey responses, that’s a big difference). The box-office figures we see on Sundays are studio estimates based on those Rentrak and Nielsen numbers, but on Mondays more accurate figures are released.

The television industry has a less comprehensive tracking system. Nielsen Media Research tracks daily viewing through “25,000 metered households” — the meters capture what channel is being watched, so those are pretty fail-safe accurate — and also gathers data through “1.6 million handwritten paper diaries from households across the country during sweep periods.” I’m sure Nielsen has statistical analyses that say 25,000 metered homes is a large enough sample size, though I’d be more confident if there were meters in every TV or if the networks made deals with cable providers to track viewership (in turn giving viewers guarantees of anonymity and some incentive to allow themselves to be tracked). But the paper diaries are extremely suspect, as the New Yorker’s James Surowiecki has pointed out.

The question is, does the video game industry rely on NPD’s numbers for a specific reason or just because video games are still young and there aren’t industry hegemons like the big six movie studio/entertainment conglomerates or a powerful umbrella organization equivalent to the MPAA or RIAA that would want more specific data (there is the Entertainment Software Association, but that hardly has the clout of the MPAA that Jack Valenti built)? Surowiecki notes that the TV numbers derived from paper diaries greatly benefit local affiliates. It’s not clear who would benefit from squishy video game numbers, so I’m guessing it’s not a purposeful decision. But as the industry continues to grow, it’s a question that deserves an answer.

— January 16, 2007

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s