Quality vs. Quantity: Research Your Market Face-to-face
By Ron Sellers
Originally published in The NonProfit Times, March 15, 1998
"We want to do some focus groups."
Those words have formed the introduction to countless conversations between non-profit executives and marketing research consultants (or departments). The trouble is, they're often coming from people who are one or two steps ahead of themselves in the conversation.
It's not unusual for executives to request research in a form they have used before, or are familiar with: focus groups, a mail survey, a telephone survey, etc. The trouble is that different situations require different methodologies. Choosing the right methodology is a process that must take into account costs, deadlines, the type of people being interviewed, the type of information needed, and numerous other factors.
One of the most basic questions is also one which is often overlooked: do you need qualitative or quantitative information? Failing to think this question through will often lead to marketing research which is less than what you need it to be (and therefore a waste of scarce resources).
Even if you're going to hand the research project over to an expert (whether an internal department or an external consultant), the researcher will still need your input on this key question in order to design the appropriate way to get the information you need. And once the research is done, it is imperative that you understand the methodology used, so you can direct how the resulting information can and cannot be applied.
Qualitative research is an in-depth exploration of what makes people tick on a particular subject: their feelings, perceptions, decision-making processes, etc. The most common form of qualitative research is focus groups, but one-on-one depth interviews are also used. Shalini Subbarao, Director of Marketing Research for the American Cancer Society, understands the reasons for selecting qualitative research. "In some cases it's crystal clear that we need to do focus groups, because we haven't sufficiently understood a target segment," she reports. "In that case we need qualitative research to help determine the right questions to be posed in subsequent research, towards ultimately providing the answers that management needs for decision-making."
Qualitative research will provide a much deeper understanding of how the target market thinks, but it does not provide projectable data. In other words, you can't legitimately walk away from a focus group with the feeling that since eight out of the ten respondents liked your organization, this signals strong public acceptance in the general population.
Quantitative research, on the other hand, is projectable. This methodology should employ a larger sample which is representative of the entire population being researched (donors, prospects, lapsed donors, etc.). Quantitative research is usually done by mail, telephone, or now even through the Internet. Because it's conducted on a larger scale, rather than through a two-hour conversation with a handful of people, it won't provide the depth of information available through qualitative work. However, the information is projectable to the entire population – if 35% of the donors surveyed have concerns about your fundraising expenses, you know that the figure is about the same in your entire donor base.
The most basic question in whether to do qualitative or quantitative research is whether the research needs to produce projectable data. Organizations frequently make the mistake of using focus groups to gather numerical data, such as how many people liked the advertisement, or what proportion of the respondents mentioned that they would like to receive a monthly newsletter.
Focus groups are not completely representative of the total population being researched (the number of respondents is too small to be projectable, the research is usually done in just two or three locations rather than across the country, inarticulate respondents are usually screened out, etc.). Unfortunately, it's not uncommon to finish a focus group project only to hear an executive say, "Did you notice how many donors liked the new newsletter format?" or "I'm concerned that three people said they wouldn't support that new campaign."
The focus in qualitative work should not be on "how many donors" or "three people." The focus should be on how people reached their decisions. For instance, a good researcher will key not on the fact that a lot of donors liked the newsletter format, but on what elements created that positive response, how those positive elements could translate to other communications from the organization, and what were the negatives among people who didn't like the format.
Leave the "how many" to quantitative research – that's its purpose. Mail, phone, and Internet studies are meant to project the results back to the total population. Surveys are the way to provide data such as how many people like the newsletter format, what percentage have concerns about the new campaign, or what is the average age of lapsed donors.
A common approach is to conduct both types of research to investigate a crucial issue fully. Qualitative research will tell you what the major issues are, and a follow-up quantitative study will reveal the relative importance of each issue.
For instance, take the example of a non-profit organization that is considering whether to change its name. Focus groups can be used to explore how the current name is perceived, as well as to test the response to some potential new names. In the groups, it will be of relatively little importance how many people like each of the potential new names. Instead, what should be explored is what image each name gives the organization, what each name communicates, what mental pictures each one brings to people, and what are the potential positive and negative elements of each name.
Once this project is done, the organization is left with a complete picture of what each potential name communicates to people. What's missing is information about what proportion of the target market will have a positive response to each name, plus what type of people react well to each name. For this purpose, quantitative research is employed. A representative sample of current donors, plus a sample of potential donors, is surveyed. The potential names are presented, to determine which has the highest positive response, and what type of people are responding well to each.
From the combination of the qualitative and quantitative work, the organization now knows (for instance) that "Name B" was the second choice overall, but it was number one among their most important demographic group. In addition, they know exactly what image it brings to mind for people, so they can play off this in logo design, communications, advertising, and direct marketing.
Unfortunately, due to constraints of time or finances, this two-step approach can't always be taken. Sometimes you have to choose between the two. In this choice, the trick is to know what kind of information you're seeking. If you need to know how people think, turn to qualitative work. If you need to know what proportion of your target thinks a certain way, that's a problem for quantitative surveys.
Either way, it's important for the decision-makers in your organization to understand the different types of research, even if they're not the ones designing or conducting the project. Once the researcher has turned the information over to the decision-makers is when the information is most likely to get unintentionally used in ways that it wasn't intended to be. The more the executive level knows about the purpose and limitations of each methodology, the more use (and more accurate use) they'll be able to make of the results.
At the American Cancer Society, "that's an area where we're struggling a little bit," says Subbarao. "We present the results to managers, but we really don't have much control over what they go and do with the information." But she has found some ways to minimize the problems in this area. "What has helped us has been making the recommendations and conclusions and action steps as succinct and action-oriented as possible. That makes sure certain aspects of the report are not misconstrued or misinterpreted."
At the beginning of the project, rather than requesting a specific type of research, it will be much more efficient to seek out the research expert you'll be working with and describe the kind of information you need to make better decisions. A good researcher, given this freedom, will take all these factors into consideration and design the right strategic research project for your organization.
This is the approach that has been built over the last few years at the American Cancer Society, according Subbarao. "I used to find three years ago that there were instances where people came and said, ‘I think I need to do focus groups' without really determining specific objectives or working out a research design," she said. She credits the process of educating her internal clients with turning this around. For the researcher, "the key is the skill in educating clients and lending credibility to the whole process. Our executives have gotten to the point that they trust the way we consult them."
