2013 Online Research Conference
With an expanded program and additional networking opportunities, CASRO’s popular Online Research Conference heads to San Francisco for 2013. This event attracts leading practitioners to present and discuss new developments that improve the core methods of online sampling, respondent access, and online data collection. Substantial attention will be paid to exciting new methods now gaining traction as either complements to or substitutes for what we consider traditional online research.
After an industry-wide "call for papers” CASRO and the Conference Committee are pleased to announce the selected presentations on the pages that follow. This event also will feature panel discussions from practitioners on critical topics.
We are thrilled to continue the tradition of providing a forum for researchers to understand current and emerging methods through evidence-based research. We hope you’ll join us to participate in the discourse.
Wednesday, March 6, 2013
Thursday, March 7, 2013
|Pre-Conference Gathering—sponsored by reInvention|
Meet up with your fellow conference attendees. Drinks are on us.
At Local Edition (691 Market Street at Third)
A short walk from the Westin, Local Edition is one of San Francisco's newest and most talked about gathering places. Join us in this historic underground space - formerly the printing room for William Randolph Hearst's renowned San Francisco Examiner. Step back in time as you are surrounded with vintage typewriters and historical issues of the Examiner and The Call. The drink list includes a bevy of new concoctions named for people, events, and things associated with the 1950s.
|8:00 a.m.||Breakfast in the Exhibit Area|
|8:45 a.m.||Opening Remarks|
• Diane K. Bowers, President, CASRO
• Bob Fawson, Chief Access, Supply & Engagement Officer, SSI, Conference Co-Chair
"Reinventing and Re-imagining Online Markets, Competition and Leadership"
Given the success of non-research insights firms (e.g. web analytics), and the speed at which they have created value for marketers, researchers must understand how to remain relevant to decision makers. Researchers are trained to view problems through the lens of traditional methods. Proliferation of techniques and tools that do not approach problems with that framework has greatly increased the complexity of our space. So, the question is, what framework should a researcher use when deciding which method to apply to a given problem/project? And how do you stay abreast of new developments? This presentation provides a provocative and practical approach to getting ahead of the "Complexity Curve" created by the confluence of Big Data, DIY, Consumer Influence Networks, digital and mobile venture startups.
• Gayle Fuguitt, Former VP, Consumer Insights, General Mills, Inc. (Founder, Tempus Futures)
|9:45 a.m.||Introduction: Data Quality |
Sampling bias, respondent engagement techniques and other issues impacting data quality have been discussed extensively in recent years. We’ll hear about recent research, review the findings and discuss what holds promise and what requires further study.
• Bob Fawson, Conference Co-Chair
|9:50 a.m.||"Do I Have Your Full Attention?”|
When respondents fail our attention traps we like to think it has been proven that they were not paying
attention. How do we know that inattention is what we are trapping? This original research will demonstrate that what we consider ‘inattention’ might better be explained through the concepts of cognitive under specification. It is the trap itself that causes the failure—the respondent’s response is entirely normal and human.
• Pete Cape, Global Knowledge Director, SSI
|10:25 a.m.||"Does Sample Size Still Matter?”|
The advent of online research, in particular online panels, promised to make very large samples affordable. Alas—while online panels have driven down CPI, small samples are commonplace, especially for B2B and healthcare research. In this paper we explore the impact of inferences from very small samples on the outcome of management decisions and employ a Bayesian approach to improve our confidence in managerial decisions based on small samples. In addition to the presenter, this paper was authored by Megan Bond (KJT Group, Inc.).
• David Bakken, Chief Operating Officer, KJT Group, Inc.
|11:00 a.m.||"Battle of the Scales: Understanding Respondent Scale Usage in the US and Abroad” |
What is the impact of using 4-, 5-, 7-, 10-, and 100-point scales on the internal consistency and reliability of survey data? Does the inclusion or exclusion of a neutral point have an impact on respondents’ evaluation of products or constructs? The presenters extend earlier research by comparing differences in response styles and scale reliabilities using multiple measurement scales. They also examine the impact of dropping or retaining the neutral point in Likert-type scales, and address guidelines for designing global surveys that yield more reliable results. In addition to the presenters, this paper was authored by Kartik Pashupati (Research Now).
• Melanie Courtright, Vice President, Research Services, Research Now
• Annie Pettit, Vice President, Research Standards, Research Now
|11:35 a.m.||Break with Exhibitors|
|12:05 p.m.||Succinct or Extinct: Can Your Panel Company Survive in 2014?|
2013 is the turning point for panel companies to become more efficient or face extinction in a fiercely
competitive marketplace. Learn how full service companies, end clients and others are leading panel
companies to a framework that increases quality and reliability, streamlines processes and controls risk. Follow the latest trends in productivity, supply chain management, and quality metrics before you face extinction with your end clients.
• Craig Overpeck, COO, M3 Global Research Services
|12:20 p.m.||Do It Yourself (DIY) Research Panel Discussion|
Much has been written about the recent proliferation of companies offering DIY/Instant Research
methodologies, which are typically inexpensive. Are these methodologies able to produce representative results? What conditions are necessary to use such offerings effectively? What are best practices regarding the application of statistical accuracy measures to qualitative research? Our panel of experts will engage with the audience to address these issues.
• John Bremer, session moderator
• Bob Fawson, session moderator
• Phillip Garland, Vice President, Methodology, SurveyMonkey
• Efrain Ribeiro, Chief Research Officer, Lightspeed Research
• Ryan Smith, Co-founder and CEO, Qualtrics
• George Terhanian, N. A. President and Group Chief Strategy Officer, Toluna
• Randall Thomas, Vice President - Online Research Methods, GfK
|1:25 p.m.||Lunch - Sponsored by GMI |
|2:30 p.m.||Introduction: Data Integration|
Nabbing actionable information from constantly growing streams of data can be thrilling. But deciphering, integrating and managing this infusion of facts and figures also is daunting and often gives rise to unforeseen issues and challenges. What are researchers and others doing to extract meaning in a manageable, responsible way?
• John Bremer, Chief Research Officer, Toluna
|2:35 p.m.||"Data Privacy and Ownership: What Consumers Think”|
This presentation will report findings from a cross-age (15 to 65+) and cross-country (US, China, India) investigation of consumer’ attitudes and behaviors regarding privacy. While laws determine what market researchers can/cannot do, these results help us understand what we should and should not do. In addition to the presenters, this paper was authored Jay Meyers (J.D. Power & Associates), Edward Paul Johnson (SSI) and Peter Van Mondfrans (SSI).
• Gina Pingitore, Chief Research Officer, J.D. Power and Associates
• Kristin Cavallaro, Knowledge and Data Analysis Specialist, SSI
|3:10 p.m.||"Combining Mobile, Social & Survey Research to Obtain Unparalleled Insight”|
While many companies simply look to offer online surveys multi-modally on mobile devices, this approach fails to recognize the true benefits of mobile surveying. The best data collection occurs where respondents are most comfortable in responding and leveraging all available information, which on the mobile device is most likely, social media participation information. Mobile data collection may be self-reported behavior and attitudes from structured survey interactions or passive, collecting data about respondents by ‘listening’ to their interactions within the social media space. Augmenting a structured quantitative survey with social media data from your respondents can strengthen your typical survey analysis with rich qualitative and quantitative insight, and help you to develop sophisticated segmentation based on social media narratives.
• Carol Haney, Vice President Product Marketing, Toluna
|3:45 p.m.||"Optimize Windows 8 Campaign Website Using Big Data and Online Survey Results”|
Microsoft wanted to understand how effective a campaign website was at increasing brand perceptions, specifically for Windows 8. By measuring campaign exposures, perceptions and interactions on the site the visitor experience was optimized and the overall perception of Windows 8 was maximized. Hear how a close study of the visitor experience helped the campaign planners and web designers at Microsoft maximize the campaign ROI.
• Roslyn Ku, Vice President, comScore
|4:20 p.m.||Introduction: Emerging Techniques|
Disruptive new ideas and techniques continue to pave the way for the future of online quantitative research. What interesting new methods are being tested now and what areas show promise for near and long-term application?
• Frank Kelly, Global Director, Lightspeed Research
|4:25 p.m.||"Bringing Colour Into Our Digital Lives” |
In 2008 the World Federation of Advertisers published their famous blueprint for consumer-centric holistic measurement: ‘For truly understanding the multi-media behaviour of consumers while respecting their privacy’. A century earlier, Alfred Adler published the basic principles of what later became the social-psychology theory of individual differences. This is a framework for understanding consumers at their ‘deepest’ level and their fundamental motivations that drive purchase decisions. Now what would happen when we combine both? Suppose we have the insights into consumers at the deepest level AND we have all of their online media consumption data? The results of this experiment will be presented.
• Piet Hein van Dam, CEO, Wakoopa
|5:00 p.m.||Perfecting Over-Done Social Media Segmentation|
For much of the past decade, researchers have struggled with the assumption that social media influencers make advertising more effective. The industry is now challenged to validate the numerous segmentations of Twitter and Facebook users, as they all are based on one body of data. This presentation will detail verifiable, stable influencer segmentation that can be validated against online, offline and self-described behavior.
• Margie Strickland, Vice President, Measurement Science, Symphony Advanced Media
|5:35 p.m.||Closing Remarks|
• John Bremer, Toluna
• Bob Fawson, SSI
|Adjournment Day I|
Cocktail Reception with Exhibitors -sponsored by Mindfield Online
Nightlife Party at California Academy of Sciences
Guests experience one of Golden Gate Park’s premier museums, the newly renovated California Academy of Sciences, which brings the whole universe under one roof—an aquarium, a planetarium, a natural history museum and more. Enjoy a night, music, creatures and cocktails (Three-hour open bar). Return to hotel at 10:30 p.m. Additional cost: $95 SOLD OUT
Friday, March 8, 2013
|7:30 a.m.||Breakfast with Exhibitors|
|8:15 a.m.||Opening Remarks|
• John Bremer, Chief Research Officer, Toluna
|8:40 a.m.||Introduction: Respondent Engagement|
Researchers face an increasingly congested competition for respondent attention. While the topic is as broad as it is important, new research shared in this session centers on optimizing surveys for short attention spans and small screens. Such efforts are intended to address how respondents access information - on the go via smartphones and tablets. Learn what techniques are keeping respondents engaged and how they impact survey data.
• Steve Coffey, Chief Research Officer, The NPD Group, Inc.
|8:45 a.m.||"‘Bite-Sized Chunks’—Mobile and CAWI Parallel Proposal” |
The ‘short burst’ style of communication (SMS, Twitter) on mobile devices compels us to explore more ‘bite-sized’ approaches to surveying. As we look to move to mobile, we need to understand how our research methods can be adapted to meet these challenges and how our analyses can fuse small chunks and multiple respondents into a complete study. This study explores how to best use partial completes and seeks to identify appropriate hooks that allow the correct inferences to be made from fused data. In addition to the presenters, the paper was authored by Alex Johnson (Kantar Operations).
• Frank Kelly, Global Director, Lightspeed Research
• Sherri Stevens, Director, Millward Brown
|9:20 a.m.||"Cyborgs vs. Monsters: Assembling Modular Surveys to Create Complete Datasets” |
As respondents continue to show less interest in completing traditional surveys, we need to change our environment to accommodate shorter surveys. Modular surveys breakdown a 20+ minute survey into three 5- to 8-minute modules. However, end users of the information can’t help but have concerns over missing data. This presentation examines two different techniques for combining modules into a complete data set suitable for advanced analytics. These techniques are data imputation (cyborgs) and respondent matching (monsters).
• Edward Paul Johnson, Director of Analytics, SSI
• Lynn Siluk, Vice President, Marketing Sciences, Gongos Research
|9:55 a.m.||"Shorter Isn’t Always Better” |
This experiment addresses the impact of survey length and complexity on respondent fatigue and engagement. The working hypothesis is that a core group of panelists do not fatigue easily or benefit from seemingly more engaging survey enhancements, while the group that does is difficult to engage long term without drastically altering the subject matter. In short, one size does not fit all. Statistical models predicting engagement that leave out interactions between design elements and personality may be underspecified and lead to the wrong conclusions.
• Inna Burdein, Director of Panel Analytics, The NPD Group, Inc.
|10:30 a.m.||Break with Exhibitors|
• Bob Fawson, SSI
|11:05 a.m||"Will You Marry Me? - Exploring the Validity of the Gamification of Research" |
Everybody wants to get engaged. Brands want emotional connections, advertising agencies yearn for creative that engages and inspires consumer participation, and media companies strive to find new ways to seduce consumers into giving them their attention. So why does brand and advertising research continue to ignore what people want? Would engaging respondents with more creative and gamified experiences online lead to more human and nuanced insights that better predict real world brand performance? New research will tell us.
• Terry Sweeney, Vice President, Operations and Client Services, Cross-Tab
• Dan Goldstein, Chief Strategy Officer, DB5
• Steve Becker, Vice President Brand Strategy, DB5
|11:45 a.m.||Conference Prize Winner Announced - |
Win a New Microsoft Surface!
|1:00 p.m.||Mobile Research Presentations and Panel Discussion|
The rush to optimize data collection on mobile devices continues. Issues such as survey content and structure, device variation, geo-location, inclusion in multi-mode studies, and many others continue to prompt discussion and new research.
• Gina Pingitore, Chief Research Officer, J.D. Power and Associates, session moderator
|1:15 p.m.||"Mobile Research Risk: What Happens to Data Quality When Respondents Use a Mobile Device for a Survey Designed for a PC” |
With the rise of smartphone ownership, some respondents find it convenient to take surveys on a mobile device despite researchers’ expectation that survey completion will occur on a computer. This "unintentional” mobile data collection can produce survey rendering problems that make it difficult for respondents to complete surveys. This presentation offers results of an experiment to determine how data quality and other survey measures differ based upon the device used to complete the survey and whether steps were taken to optimize the survey for the data collection mode. Importantly, this research involved random assignment of respondents to different data collection devices, allowing any observed differences to be attributable largely to the data collection method itself, and not to characteristics of respondents choosing a particular data collection device. In addition to the presenter, this paper was authored by Jeff Miller (Burke).
• Jamie Baker-Prewitt, Senior Vice President, Director of Decision Sciences, Burke, Inc.
|1:40 p.m.||"Optimizing Surveys for Smartphones: Maximizing Response Rates While Minimizing Bias” |
Respondents use smartphones to take non-mobile designed online surveys in increasingly greater numbers each year. This raises many questions about survey design, which will be addressed in this presentation. "Designing with mobile in mind” introduces creative elements in addition to traditional research design. This presentation will address maximizing response rates, especially in the arena of customer-supplied sample, as well as minimizing bias between the two modes of mobile and non- mobile. Perspective will be shared on how to optimize "look and feel” to reflect a brand, scales, format adaptation and whether to design for a specific device. In addition to the presenters, this paper was authored by Ted Saunders (Maritz Research).
• Kevin Lattery, Vice President, Marketing Sciences, Maritz Research Inc.
• Gloria Park Bartolone, Senior Vice President, Global Fieldwork Services, Maritz Research Inc.
|2:05 p.m.||"Solving the ‘Unintentional’ Mobile Challenge: Mobile Optimized or Mobile Friendly?” |
We know that "unintentional” smart phone survey takers are different demographically and behaviorally and that their survey-taking experience is quite different from respondents who take surveys on PCs or other larger format devices. But we can’t know the extent to which those observed differences are the result of who chooses to start a survey on a mobile device and to what extent they are driven by the device on which they choose to participate. This presentation addresses several key questions including: How does question presentation or the usability of a survey instrument impact results? Can we meaningfully combine data from mobile and non-mobile responders when they’ve each taken a survey optimized for their respective devices? Will a mobile-friendly design decrease drop-offs and survey lengths for smart phone responders?
• Gregg Peterson, Senior Vice President, Research Operations, Market Strategies International
|2:30 p.m.||Panel Discussion: Mobile Research|
|2:55 p.m.||Closing Remarks|
|3:00 p.m.||Conference Adjournment|
|John Bremer, Chief Research Officer, Toluna; Conference Co-Chair |
|Bob Fawson, Chief Access, Supply, & Engagement Officer, SSI; Conference Co-Chair |
|Steve Coffey, Executive Vice President, Chief Research Officer, The NPD Group |
|Frank Kelly, Global Director, Lightspeed Research |
|Gina Pingitore, Chief Research Officer, J.D. Power and Associates |