QUALITATIVE DATA ANALYSIS

Qualitative data consist of words, not numbers. As with quantitative data, systematic analysis and interpretation are required to bring order and understanding to the data that has been collected.

Narrative Data

Qualitative data come in many forms. Examples include responses to open-ended questions on a survey administered at the end of a food safety training, the transcript from an interview or focus group session, notes from a log or diary, or field notes. Data might come from many people, a few individuals, or a single case.

Analyzing Qualitative Data

Know the data. Good analysis depends on a thorough understanding of the data set. This means one must read and re-read the text. If tape recordings are available, listen to them several times. Write down thoughts while going through the data. Also, just because one has data does not mean it is quality data. Before beginning any analysis, consider the quality of the data and proceed accordingly. Investing time and effort in analysis may give the impression of greater value than is merited. Always explain the limitations and level of analysis that is appropriate given the data set.

Focus the analysis. Review the purpose of the evaluation. Write down a few key questions that the analysis should answer. These questions might change as one works with the data, but it will help begin the process. Here are two common approaches to focus an analysis.

Focus by question or topic, time period or event. Put all the data from each question together. Look across all respondents and their answers in order to identify consistencies and differences. Apply the same approach to particular topics, or a time period or an event of interest. Later, the connections and relationships between questions can be explored.
Focus by case, individual or group. One might want an overall picture of one case, such as one foodservice establishment; one individual, such as a first-time participant in a workshop; or one group, such as participants in a food safety certification program. Rather than grouping these respondents' answers by question or topic, organize the data from or about the case, individual or group, and analyze it as a whole. Or, combine these approaches and analyze the data both by question and by case, individual, or group.

Categorize information. Some define categorizing information as coding the data. Categorizing does not involve assigning numerical codes as is done in quantitative analysis where one labels data with preset values. Categorizing helps one to bring meaning to the data by identifying themes or patterns — ideas, concepts, behaviors, interactions, incidents, terminology or phrases used – in order to bring meaning to the text.

This can be fairly labor-intensive depending on the amount of data one has. But this is the basis of qualitative analysis. Read and re-read the text to identify coherent categories. Also, assign abbreviated codes of a few letters, words, or symbols and place them next to the themes and ideas that are found. This helps one to organize the data into categories. Provide a descriptive label (name) for each category that is created. Be clear about what to include in the category and what to exclude.

Here are two ways to categorize narrative data that is collected — using preset or emergent categories. With preset categories, start with a list of categories and then search the data for them. Whereas with emergent categories, read through the text and find the themes that recur in the data. These become the categories. This approach allows the categories to emerge from the data. Categories are defined after one has worked with the data or as a result of working with the data. Sometimes, one combines these two approaches — starting with some preset categories and adding others as they become apparent.

The initial list of categories might change as one works with the data. The definition of categories might change or new categories might be identified to accommodate data that do not fit the existing labels. Main categories might be further broken into subcategories. This allows for greater discrimination and differentiation. Continue to build categories until no new themes or subcategories are identified. Add as many categories as needed to reflect the nuances in the data and to interpret data clearly. While one usually wants to develop mutually exclusive and exhaustive categories, sometimes sections of data fit into two or more categories. So some data might need to be cross-indexed.

Identify patterns and connections within and between categories.

As one organizes data into categories — either by question or by case — one will see patterns and connections within and between categories. Assessing the relative importance of different themes might be important to the analysis. Here are some ways to do this.

Within a category description. One might be interested in summarizing the information pertaining to one theme, or capturing the similarities or differences in people's responses within a category. To do this, assemble all the data pertaining to the particular category. What are the key ideas being expressed within the category? What are the similarities and differences in the way people responded, including the subtle variations? Write a summary for each category that describes these points.
Larger categories. One might wish to create larger categories that combine several categories. One can even work up from more specific categories to larger ideas and concepts. Then one can see how the parts relate to the whole.
Relative importance. To show which categories appear more important, count the number of times a particular theme comes up. These counts provide a very rough estimate of relative importance. They are not suited to statistical analysis, but they can reveal general patterns in the data.
Relationships. Two or more themes might occur together consistently in the data. Whenever one is found so is the other. For example, families with limited resources consistently list saving money as the primary benefit of canning foods at home. Some connections might suggest a cause and effect relationship or create a sequence through time. For example, respondents might link improved safe food handling practices to a good training program. From this, one might argue that good training causes improved food handling practices. Such connections are important to look for, because they can help explain why something occurs. Be careful though about cause and effect interpretations. Seldom is human behavior or narrative data so simple. Ask yourself How do things relate? What data support this interpretation? What other factors might be contributing?

Develop a table or matrix to illustrate relationships across two or more categories. Look for examples of responses or events that run counter to the prevailing themes. What do these countervailing responses suggest? Are they important to the interpretation and understanding? Often, one learns a great deal from looking at and trying to understand items that do not fit into the categorization scheme.

Use themes and connections to explain findings. It is often easy to get side tracked by the details and the rich descriptions in the data. But what does it all mean? What is really important? This is called interpreting the data — attaching meaning and significance to the analysis.

Develop a list of key points or important findings discovered as a result of categorizing and sorting the data. What are the major lessons? What has application to other settings, programs, and studies? What will those who use the results of the evaluation want to know?

Develop an outline for presenting results. The length and format will depend on the audience. It is often helpful to include quotes or descriptive examples to illustrate results. A visual display might help communicate the findings. Sometimes a diagram with boxes and arrows can help show how all the pieces fit together. Creating such a model may reveal gaps in the investigation and connections that remain unclear. These might be areas that suggest further study.

Summary

Moving from a mass of words to a final report requires using a systematic method to organize and keep track of the text. This is largely a process of cutting and sorting. Work by hand, either with a hard copy or directly on a computer. How data is managed depends on personal preference and the amount and type of qualitative data. In summary, here are some data management tips:

Check the data. Make sure everything is together. Decide whether the data are of sufficient quality to analyze, and what level of investment is warranted.

Add identification numbers. Add an identification number to each instrument, respondent, group, or site.

Prepare data for analysis. Transcribe taped interviews. How complete to make the transcription depends on the purpose and resources. Sometimes, simply summarizing what people say is adequately for an analysis. Or, certain parts of an interview might be particularly useful and important and just those sections are transcribed. Other times, one needs to have every word of the entire interview transcribed. Transcription is time-consuming so be sure both data quality and use of the data are worth the investment. With small amounts of narrative data, work directly from the original hard copy. Text is usually typed into a computer program.

Decide how to format cases. Willresponses be entered question by question, or whether you want to keep all text concerning one case, individual, group or site together. If data is typed into a word processing program, leave a wide margin on the left so enough space is available to write labels for text and notes. Number each line to help with cutting and sorting later.

Use a special qualitative data analysis program. Using such a program will depend on the size of the data set, resources available, preferences, and level of analysis needed or warranted.

Other considerations when analyzing qualitative data:

When cutting and sorting, keep track of the data source. Keep identifiers attached to all sections of data. Keep enough text together to make sense of the words in their context. Include enough surrounding text so the meaning is not open to misinterpretation.
Make connections. Once the data is sorted, think about how the categories fit together and relate. What seems more important, less important? Are there exceptions or critical cases that do not seem to fit? Consider alternative explanations. Explore conflicting themes, and evidence that seems to challenge or contradict the interpretations.
To trace connections, spread note cards across a table, use sticky notes on walls, or draw diagrams on newsprint showing the categories and relationships. Another approach is to create a two-dimensional or three-dimensional matrix. List the categories along each axis, and fill the cells with corresponding evidence or data.
Count the frequency a topic occurs or how often one theme occurs with another, or to keep track of how many respondents touch on different themes. Such counts might be illuminating and indicate relative importance. But treat them with caution — particularly when responses are not solicited the same way from all respondents, or not all respondents provide a response.

Minimizing Bias

Bias can influence the results. The credibility of the findings can be increased by:

Using several sources of data. Using data from different sources helps check the findings. For example, combine one-on-one interviews with information from focus groups and an analysis of written material on the topic. If the data from these different sources point to the same conclusions, one has more confidence in the results.
Tracking choices. If others understand how the conclusions were drawn, results will be viewed as more credible. Keep notes of all evaluation decisions to help others follow the reasoning. Document reasons for the focus, category labels created, revisions to categories made, and any observations noted concerning the data while reading and re-reading the text.
Document the process used for data analysis. People often see and read only what supports their interest or point of view. Everyone sees data from his or her perspective. It is important to minimize this. Clearly state how data was analyzed so that others can see how decisions were made, how the analysis was completed, and how the interpretations were drawn.
Involving others. Getting feedback and input from others can help with both analysis and interpretation. Involve others in the entire analysis process, or in any one of the steps. Have several people or one other person review the data independently to identify themes and categories. Then compare categories and resolve any discrepancies in meaning.

Pitfalls to Avoid

Do not generalize results. The goal of qualitative work is not to generalize across a population. Rather, a qualitative data collection approach seeks to provide understanding from the respondent's perspective. It tries to answer the Why? Narrative data provide for clarification, understanding and explanation — not for generalizing.
Choose quotes carefully. While using quotes can lend valuable support to data interpretation, often quotes are used that only directly support the argument or illustrate success. This can lead to using people's words out of context or editing quotes to exemplify a point. Think about the purpose for including quotes. Include enough of the text to allow the reader to decide what the respondent is trying to convey.
Respect confidentiality and anonymity when using quotes. Even if the person's identity is not noted, others might be able to tell who made the remark. Therefore, get people's permission to use their words.
Address limitations. Every study has limitations. Presenting the problems or limitations encountered when collecting and analyzing the data helps others better understand the conclusions.

Working with qualitative data is a both a science and an art. It involves critical, analytical thinking and creative, innovative perspectives.

Communicate findings to the respondents who participated in the evaluation. Not only is this courteous, but it helps to ensure their cooperation in future work. Communication methods depend on the audience. Examples include: a written report, short summary statements, videotapes, pictures, photo essays, displays, slide presentations, graphs and visuals, and media releases.

Test Your Knowledge

1. How is qualitative data different from quantitative data?

2. What are two ways to categorize narrative data?

3. Can qualitative data be generalized?

4. Why should the process used to analyze data be documented?

5. What are four ways one assess the relative importance of different themes when analyzing data?

ANSWER KEY