Selecting Markets for Focus Groups
By Ron Sellers
Originally published in Alert! (Marketing Research Association), June 2007
There are approximately 55 focus group facilities in Chicago. Around 21 in Dallas, 24 in Atlanta, and 17 in Boston. Even smaller markets such as Hartford, Dayton, and Evansville have multiple facilities from which to choose.
All focus group facilities are not created equal. In fact, it can be dangerous to assume they are all good, or even competent. Even those which are generally excellent might not be the right fit for your qualitative research.
Having managed qualitative projects all over North America for the past 20 years, I’ve encountered more unpleasant surprises than I care to remember (or admit). Each time, I think, “Now I have something else I have to ask the next facility.” While this list is growing longer, I’ve found that a few upfront questions for the facility staff can save a tremendous amount of time and frustration…and even save a project.
Following are some suggestions for what to discuss with facilities before you book a project, along with some unpleasant surprises of my own that hopefully will allow you to avoid the same problems.
How is their recruiting done? This is a critical point. Do they have the staff to handle your recruiting needs comfortably, or will your project stretch them to the limit? Do they recruit on evenings and weekends, or do they expect to find a good cross-section of respondents at home in the late mornings when they call? If the facility is located in a mall, do they actually recruit from a cross-section of local residents, or do they just step into the mall and interview shoppers for a couple of days?
We’ve found facilities that will only recruit from their database, that have no database, that do no recruiting on weekends, that do a big e-mail blast and wait for respondents to call them (and if not enough people call, they claim “the incidence is too low”), and that try to use the same respondents over and over.
Some facilities only use recruiters who make calls from their own homes. While they swear by this method, it may raise concerns about accountability and efficiency for some clients. Mall facilities may only recruit from mall shoppers right outside their doorway. Make sure you are getting recruiting you are comfortable with, rather than finding out halfway through the project that the recruiting approach concerns you.
Unpleasant surprise: one facility with a good reputation was having trouble with the recruit. We found out the hard way that they couldn’t reach people on the customer list because all of their recruiters were single mothers who worked from home and only made calls during weekdays, when their kids were in school.
Do you and the facility define “incidence” the same way? We define incidence as the proportion of people recruiters reach who qualify for the study, whether they are willing to participate or not. Incidence does not take into account factors such as show rate, response rate, ease of reaching people, or list inaccuracies. It is fair to take all of these factors into account when planning a project, but if we tell a facility that the incidence is 50%, we mean 50% of the people they speak with will qualify for the study. How many they can convince to show up is another issue.
Unpleasant surprise: having an extensive argument with a facility director about incidence, even though our RFP clearly defined it. The director insisted that because they attempted to contact 100 people, reached 40, 20 qualified, and five were willing to come to the groups, incidence was 5% and pricing should go way up.
How does the facility handle recruiting that’s not from their database? Often customer or donor lists in any given geography are limited. What ratio of names to recruits does the facility normally require: 10:1? 20:1? 40:1? Know this, and know the size of your list in the facility’s specific recruiting area, before starting a project.
If you know the list is going to be smaller than the facility wants, what is their reaction? We’ve had a facility successfully recruit 20 people from a list of 38 (not that this should be considered the norm – it was a Herculean effort by the facility). But they knew the situation before starting the project, and were willing to roll up their sleeves and get it done.
What if the recruits you need are not the kind of people normally on a facility database, or easily reachable through purchased lists? Finding architects or Sunday school teachers or nursing home workers likely will require some creativity and networking by the facility. Talk with them about it. Do they have a plan for the recruiting, or are they looking to you to tell them exactly how to do it? Do they just plan to pay for a classified ad in the newspaper and hope a bunch of qualified respondents call in?
Unpleasant surprise: giving a facility a relatively small donor database (15:1 ratio) and having them tell us after a week that the recruit couldn’t be accomplished. Their reason? “We’ve called all of these people twice. We’ve even left messages. We can’t help it if they’re not calling back!”
What happens if the project doesn’t go as planned? Sometimes it seems that the only things less accurate than project specs are weather forecasts and economic predictions.
Facilities can only plan according to the project specs you give them. If those specs turn out to be wrong, what happens? Will they have the time and staff to complete the project, or will they run out of resources if the supposed 50% incidence is actually 10%? It’s best to get a sense of how tight their schedule is going to be, and what will happen if something goes awry, before starting the project.
This applies to price, too. If the incidence is bid at 75% and it’s actually 50%, the facility needs to be compensated for the extra work they are doing. But the price shouldn’t double. For the protection of both parties, have a sense up front of what the costs will be if the incidence is significantly higher or lower than the bid.
Unpleasant surprise: finding out that only one recruiter has been assigned to fill the groups. When incidence was lower than expected, the facility had no more recruiters available to complete the project.
How accommodating will the facility be? Unexpected needs are…well, unexpected. But a brief conversation, with a couple of examples, can help you determine whether a facility will bend over backwards to meet your needs, or whether they’ll nickel-and-dime you to death, do what you ask only after kicking and screaming, or both.
Clients, especially, can have unexpected and unusual needs. We want a certain brand of drinks in the back room (not unusual when conducting research for a beverage company). We need vegetarian meals. We need six copies of the DVDs by the time we leave. We need a limo to pick us up at the hotel. We need to be there at 2:00 for a meeting. How will the facility respond to special needs?
Unpleasant surprise: a client insisted that no alcohol be served in the back room, for liability reasons. The facility had a fully stocked bar and a full-size fridge full of beer, complete with a menu and price list (apparently, this was a real money-maker for the facility). When told this would all have to be locked away, they tried to charge a $500 “alcohol removal fee.”
Will the facility be comfortable with your topic? Some recruiters are uncomfortable talking to potential respondents about highly personal topics such as religion, sexuality, or embarrassing medical conditions. Some facilities would rather not deal with certain topics. A devoutly religious facility director may not want to take on groups about tobacco or alcohol, for example.
If your topic or client have any potential issues like this, talk to the facility manager before booking the facility, rather than after you’ve scheduled travel, forwarded the facility information to the client, and given them the screener. It’s much better to know up front that this will be a problem than to discover it midstream.
In fact, even if your groups are completely non-controversial, it’s usually best to share the topic with facilities up front, since you never know when they might have a conflict of interest. It’s probably best not to conduct groups for Chrysler in one room, while in the next room on the same night another moderator is conducting groups for Chevrolet.
Unpleasant surprise: one facility took on a job related to a highly charged social topic. Only a few days later was it discovered that the facility owner was a well-known and highly involved social activist…for the other side of the issue. The groups had to be moved.
Will the facility provide you the information you need? Some facilities run a tight ship. They keep complete records of all the recruiting calls they’ve made, what the exact incidence was, how accurate the customer list was, which numbers were wrong, etc. Others keep virtually no records and can’t back up their assertions that the incidence is different than the bid or the list is full of wrong numbers. What are your record-keeping needs, and is the facility prepared to meet those needs?
Unpleasant surprise: being told the incidence was “like two or three percent,” with no documentation to back it up. When the facility finally gathered all their handwritten notes on the customer list and made an estimate of the incidence, it was actually somewhere around 20% and very recruitable, despite the whining and complaints.
Will the layout and abilities of the facility work for your purposes? Thankfully, most facilities now have a facility layout, including room sizes, readily available on their web site. If not, they should be able to provide one easily. Consider what you need to do in the respondent room. Do you need to spread out lots of visuals? Have multiple easels? Tape or pin visuals to the walls? Project a web site onto the wall? Reconfigure the table to allow for team exercises? Make sure the respondent room is the right size and design to accommodate your needs.
Also, how many viewers will you have? Nothing makes clients crankier than shoving ten of them into a space designed for six, and few want to fly to Portland in order to watch the groups on a 19” monitor in the overflow room. How much they learn from the groups, and what they think of you as a moderator, can both be affected by how comfortable they are in the observation room.
Unpleasant surprise: one facility had no waiting room for respondents. They fed them in the group room at the conference table, meaning any pay-and-sends had to be called out of the room and sent away (which set an uncomfortable tone for the other respondents). The room was also littered with the detritus of the dinner when it came time for the groups.
We have developed a memo that outlines our expectations for the work the facility will do. We carefully discuss the project with the facility manager. And we have a questionnaire each facility must complete that tells us things such as how many observers their room holds and how they do their recruiting. Even so, we still get the occasional unpleasant surprise, like the facility that wanted to charge $100 per day for e-mailing a recruiting report every day rather than their standard every other day. But through knowing what to look for and establishing the ground rules and specs before the project begins, we manage to avoid many problems that can plague qualitative research.
This article is not intended to be critical of qualitative research facilities. The vast majority of our qualitative projects go off without a hitch, thanks to the professionalism and hard work of our qualitative vendors. But things can, and do, go wrong. The point is to anticipate and avoid as many of the problems as possible. A little up-front work dramatically increases the likelihood of a smooth, successful project.
