Nielsen admits it miscalculated Web traffic by 22%

One of the top Web traffic data companies in North America has admitted it has been underestimating the amount of time people in the United States spent on the Internet.

The Nielsen Co. says a technical glitch is responsible for its undercounting time spent by 22 percent. Nielsen says the problem stems from its system having difficulty recognizing long URLs.

In a statement obtained by Adage [PDF], Steve Hasker, President of Media Products for Nielsen, said:

We are writing to inform you that we are ctively investigating an erroneous decline in our Internet use data…The primary cause is an increasing incidence of websites using very long URLs that are not properly recognized by our systems. When our system attempts to process session data including the very long URLs, which are more than 2,000 bytes in size each, it intermittently does not recognize the session, causing an estimated average 22% decline in time spent year-over-year, which can vary at the domain level.

The company says there are other factors that led to incorrect data to be shared, but those factors were not disclosed. Nielsen is now looking for potential errors in reports it issued in other countries, as well as reports that track online video consumption and Web search traffic.

Nielsen also said:

This investigation, covering every element of our Internet measurement methodology, including the panel, collection capabilities, and processes, and nature of the root cause indicates that we need to do a better job keeping pace with the rapid evolution of the Internet. We are putting new processes in place to add greater rigor to the continuous testing of our methodology and expanding our monitoring for anomalous events. We have also engaged the Media Rating Council (MRC) to review our findings and the additional processes.

The company says the problem will be fixed by the time its December report is published in January 2011.

[Via Chris Hogg]