CASRO

CASRO’s first-ever Panel Conference, which took place in Miami on February 5-6, was a dynamic forum in which 165 attendees discussed important new studies in the area of online research. The presentation of such salient research on research in the presence of the industry’s top strategists proved a valuable milestone in what many acknowledge is a crucial knowledge-gathering process.

“We were pleased to host this important event and want to thank everyone who participated in the dialogue,” stated Diane Bowers, president of CASRO. “Clearly the thirst for additional knowledge is intense and CASRO is committed to challenging our member companies to fulfill this need by sharing research on panel recruitment, panel usage, hyperactive respondents, sample quality, survey design and much more.”

“I encourage all of you to be wide open in these discussions. Don’t view these issues as competitors,” Jim Nyce, Senior Vice President, Global CIS, Kraft Foods, Inc., urged attendees during an open discussion on panel integrity. “Cost, timing – no variables or restrictions should get in the way and compromise data integrity. The validity of the results is crucial to the entire industry,” continued Nyce, who shared the dais with Kevin Nuffer, Senior Marketing Consultant at McKinsey & Company, Steve Coffey, Chief Research Officer at The NPD Group, Inc., and Frank Kelly, Managing Director at TNS EAP. The session was moderated by Efrain Ribeiro, Chief Operating Officer, Ipsos Interactive Services.

“Clients used to give a lot of direction to us, but not any more,” noted Kelly. “We are left to figure it out.” His colleagues on the panel echoed his concern about the lack of research training and education on the client side.

While some of the research presented and much of the conversation focused on issues surrounding fraudulent respondents, many in attendance asserted that the biggest overall concern is poor survey design.

Answering a question seeking a rundown of the key issues related to the quality of online research, Nuffer recommended working with a company you know well, reviewing sampling plans, keeping surveys open for a sufficient duration, and taking note of what traps are in place to take out speeders and flatliners. In many ways, he noted, it is the due diligence required of all survey methods.

The call for more research seemed unanimous. We don’t have answers to many of the questions that are now being bandied about in regards to online research quality, Nyce noted. “We need to get those answers so we can put a floor under the quality issue,” he said.

Outside the session room, attendees enjoyed breaks in a packed exhibit room, which provided an opportunity to connect with established research companies and emerging international niche players.

Other sessions included:

- Renée Smith, Vice President, Panel Quality, Harris Interactive, Inc., discussed research which utilizes a propensity score based on attitudes and behaviors of likely panel joiners and survey completers to minimize potential biases created by “cheaters.”

- Andrew Jeavons, Managing Director, Nebu USA LLC, and Jeff Miller, Chief Operating Officer, of Burke Inc. presented an experiment which showed how the problem of satisficing and respondent “fraud” can be ameliorated, and completion rates in web surveys can be increased, through the use of survey “intervention” of covert or overt means.

- Joe Giacobbe, Vice President, Innovation & New Product Development and Annie Pettit, Director of Online Panel Analytics at Ipsos Interactive Services, explored such survey-taking behaviors as speeding, random clicking and qualifying, and their effect on survey results.

- Enticed by a large incentive, how many non-qualified respondents will fudge their answers to get the prize? Terence McCarron of Greenfield Online Inc. shared results of a test survey that provided insight on the correlation of incentives and cheating.

- The behavior differences of self-recruited panelists and panelists selected by researchers was the focus of research presented by Bob Fawson, Director, Online Services, at Western Wats.

- The development of a new propensity score system to normalize the potential self-selection bias in panel sampling was discussed by Lisa Luth-Pickard, Chief Knowledge Officer at Luth Research.

- Marjette Stark, Senior Vice President, and Melanie Courtright, Vice President, of DMS Research, reviewed the differences and similarities between river respondents and panelists and clarified how these might influence the research design and conclusions resulting from the use of a particular sample for an online study.

- Michael Gazala, CEO, North America, at Lightspeed Research, provided an analysis that helped attendees understand the impacts of panel usage—specifically the role of exclusion periods and category controls—on research quality.

- Frederic-Charles Petit, CEO of Toluna plc., provided an in-depth look at the ever-changing online habits of potential respondents.

- Su Ning Strube, Vice President, Fusion Services at Vision Critical and Yola Zdanowicz, Senior Vice President at Angus Reid Strategies, combined on research that explored the impact of visual and interactive design factors on respondent fatigue and overall satisfaction with the survey process.

- The impact of different question layouts and designs, particularly in relation to the quality and quantity of responses obtained, was investigated by Trixie Cartwright, Director at TNS Global Interactive.

- Dissecting the results from ongoing methodological experiments funded by the NSF, NIH, the ISR at the University of Michigan and elsewhere, Reginald Baker, Chief Operating Officer at Market Strategies International, provided a set of best practices to help ensure that respondents find survey questionnaires easy and comfortable to use.

Special thanks to Conference Co-Chairs Frank Kelly (Managing Director, TNS EAP) and Efrain Ribeiro (Chief Operating Officer, Ipsos Interactive Services) as well as CASRO University Chair Roseanne Luth (CEO, Luth Research) and CASRO Board Chair Chet Zalesky (President, CMI Research) for their efforts in planning a most successful event. CASRO also acknowledges the support of all exhibitors and sponsors, particularly Survey Sampling International for their sponsorship of Tuesday’s dinner.