PROGRAM EVALUATION

Program evaluation is the systematic collection, analysis, and reporting of data that can be used to make informed decisions about a program. Program evaluation within the scope of these eight modules is defined as measuring the outcomes of a series of planned educational activities that have been designed to lead to a specific behavior change.

If done right and done well, the results of a program evaluation can provide credible evidence about the effect a program has on participants’ knowledge, attitudes, and behaviors.  Or, it can provide evidence about the impact the program has on identified social, economic, civic, and environmental conditions. A program evaluation that provides defensible evidence:

• Clearly assesses achievement of the goals and objectives of the education program.
• Uses a systematic procedure to collect data. 
• Reviews the data and objectively determines the value of the program.

Five Steps to Conduct a Program Evaluation

1. Planning.  Review the intended goals and objectives of the program to be evaluated.  Identify the planned activities designed to achieve the goals and objectives.  Based on this review, ask What data is needed?  How do I collect the data? How will I analyze data?  How will I summarize the data?
2. Collecting Data.  Determine what types of data are needed to conduct the evaluation.
3. Data Analysis.  Analyze and summarize the data so findings can be presented in a persuasive, credible manner.  Accurate and unbiased interpretation of findings is the key to a successful evaluation.
4. Comparing to Program Goal and Objectives. Clearly state the program goal and objectives before the evaluation begins.  For example, a program objective is that 80% of participants pass a food safety certification exam after completing 16-hours of food safety training. The objectives must be measurable and obtainable.
5. Determining Value. Compare the data against the program goal and objectives.  Findings are usually documented in the form of an evaluation report that is shared with the stakeholders in various formats -- written report or formal presentation.

Evaluation is both an art and a science.  The art of evaluation involves working with others to agree upon the questions to be answered. The science of evaluation involves identifying program objectives, developing a data collection plan, and analyzing and interpreting data in an objective manner. 

Ethical Considerations

The American Evaluation Association has developed standards (Joint Committee on Standards for Educational Evaluation, 1994) to help program evaluators.  Program evaluation must be conducted legally, ethically, and with regard for the dignity and welfare of those participating in the evaluation as well as those affected by its results. The following guidelines were developed to help evaluators meet these criteria.

• Those who are evaluated must be volunteers.  All participants must understand that they are freely participating in the evaluation and that they can withdraw from it at any time without penalties or loss of program services or benefits, now or in the future.
• Potential participants must be given sufficient information about the evaluation, including its purpose, duration, procedures, benefits (if any), and possible risks or discomforts, to enable them to freely choose whether or not to participate.
• No physical, emotional or psychological harm should result as a consequence of participation in the evaluation.
• All sensitive information about participants must be protected. Their privacy is protected by allowing anonymous responses, if possible. Where complete anonymity is not possible (for example, because pretest and post-test scores have to be matched for each subject), protection is provided by separating any personally identifying information from the evaluation data through the use of numeric or other special codes.
• The privacy of participants is further protected by not collecting or reporting any personal information (names, addresses, telephone numbers, or personal descriptors) that would result in evaluation subjects being identified by persons other than the evaluator.
• Participants must be assured that their views and opinions, likes and dislikes, suggestions, and recommendations will be treated confidentially and that only people closely associated with the evaluation will have access to the data.
• Optional -- It is a good practice to accompany each request for information with a consent form that clearly explains the above guidelines and asks subjects to sign and return the request to them. 

All educators who evaluate their programs should use these guidelines in all of their evaluation efforts.

Test Your Knowledge

1.  What is the definition for program evaluation?

2.  What are the five steps of program evaluation?

3.  What is key to a successful evaluation?

4.  What are three ethical considerations that must be addressed when conducting a program evaluation?

5.  Why must potential participants be given sufficient information about the evaluation?

ANSWER KEY

If you have questions, contact Dr. Angela Fraser, Associate Professor/Food Safety Education Specialist, Clemson University, Clemson, SC.

Clemson University is an AA/EEO employer and does not discriminate against any person or group on the basis of age, color, disability, gender, national origin, race, religion, sexual orientation, or veteran status.

 AS 0708