GLOSSARY

Analysis -- the treatment of data and information in order to elicit certain statistical data, assessment results, and evaluative.

Attribute -- a characteristic, capacity, or perceived quality of an individual or of a thing or place, such as the work context or the school. For individuals, attributes include, but are not limited to, attitude, ability, behavior, skill, knowledge, or interest.

Audience -- those individuals who participate in a specific program.

Behavior -- specific, observable actions of an individual in response to internal and external stimuli.

Bias -- a systematic tendency toward a lack of objectivity, fairness, or impartiality on the part of the assessor or evaluator, often based on personal preferences and inclinations or systematic error in an assessment instrument and/or procedures or in the interpretation and evaluation process.

Case study -- an intensive, detailed description and analysis of a single project, program, or instructional material in the context of its environment.

Certification -- official recognition of advanced status, outstanding performance or a high level of expertise in a profession.

Code -- a symbol, either numeric or alphabetic, used to represent attributes or words (such as FSE = Foodservice Establish; H = Hygiene; C-C = Cross-contamination).

Coding -- the process of transforming data, evidence, information, judgments, notes, and responses to numeric and/or alphabetic codes.

Competency -- a skill, knowledge, or experience that is suitable or sufficient for a specified purpose.

Conclusions (of an evaluation) -- the final judgments and recommendations resulting from the assessment information collected about a program.

Confidentiality -- the protection of data and information from persons other than those authorized to have access

Consent --the granting of permission concerning the collection, use, retention, or access to assessment data and information

Consistency -- (1) implementation of procedures in an identical or near identical manner across individuals or over time. (2) Obtaining the same or similar results across multiple administrations or scoring of an assessment. (3) A type of rater effect in which an assessor or an evaluator tends to rate or to interpret different data and information in a similar way.

Control group -- a group as closely as possible equivalent to an experimental group (one that is exposed to a program, project, or instructional material), and exposed to all the conditions of the investigation except the program, project, or instructional material being studied.

Correlation -- the degree to which two or more sets of measurements vary together; e.g., a positive correlation exists when high values on one scale are associated with high values on another; a negative correlation exists when high values on one scale are associated with low values on another.

Cost effectiveness -- the extent to which one program, project, or instructional material produces equal or better results than competitors that cost about the same amount of time, effort, and resources; or the extent to which an object produces the same results as competitors but is less costly.

Curriculum -- the knowledge, skills, attitudes, behaviors, and values participants are expected to learn from a program, course, or training; includes statement of expected learner outcomes, descriptions of material and activities, and the planned sequence that will be used to help students acquire the expected outcomes.

Data -- evidence, in either numerical or narrative form, gathered during the course of an evaluation and that serves as the basis for information, discussion, and inference.

Data analysis -- the process of organizing, summarizing, and interpreting numerical, narrative, or artifact data, so that results can be validly interpreted and used to guide future development of students.

Data collection procedures -- the persons, documents, products, activities, events, and records from which data are obtained.

Defensible -- an action, conclusion, or statement that is explainable or justifiable, based on a solid foundation and policy, explicit evaluation procedures, valid assessments, and sound evaluation and decision-making practices.

Delphi technique -- a method for obtaining group consensus involving the use of a series of mailed questionnaires and controlled feedback to respondents which continues until consensus is reached.

Design -- a representation of the set of decisions that determine how a student evaluation is to be conducted; e.g., identifying purposes and use of the information, developing or selecting of assessment methods, collecting assessment information, judging and scoring student performance, summarizing and interpreting results, reporting evaluation findings, and followingup evaluation results.

Desired outcomes -- the results or products that a training program, process, instructional unit, or learning activity strives to achieve, as defined in measurable terms.

Discrimination index -- an index that indicates how well an item distinguishes between the leaners who understand the content being assessed and those who do not. Positive discrimination indicates that the item or task is discriminating in the same way as the assessment method of which it is a part.

Documentation -- the collection or compilation of all tangible materials, records, and forms used in the assessment of a program

Educational objective -- a statement describing the knowledge, skill, attitude, or behavior a learner is expected to master or perform and the content on which it will be performed as a result of instruction

Effectiveness -- an attribute of those schools, teachers, programs, and approaches that meet their stakeholders' needs.

Error -- the extent to which a score, assessment, or calculation is incorrect or inaccurate.

Estimate -- an approximation of a true score or value

Ethical -- performing the evaluation or behaving in accordance with a moral code of conduct that addresses such as the innate rights of individuals

Evaluation -- systematic investigation of the worth or merit of a program in relation to a set of learner expectations or standards of performance

Evidence -- documentation and verbal statements from credible sources to assess a program. Evidence is used both to generate and to justify judgments about a program

Experimental Design -- the plan of an experiment, including selection of subjects, order of administration of the experimental treatment, the kind of treatment, the procedures by which it is administered, and the recording of the data (with special reference to the particular statistical analyses to be performed)

Experimental Group -- a group of subjects assigned to receive a treatment (independent variable), the effects of which are measured (dependent variable). Comparison are often made between these effects and those observed for a comparison or control (nontreatment) group.

Fairness -- impartiality. Fairness refers to such aspects of the assessment program and evaluation system as equal opportunity to acquire the knowledge and skills to be assessed, use of developmentally appropriate assessments, sound procedures, appropriate use of evaluation results, and reasonable demands on the teachers being evaluated in terms of such factors as time, costs, and personal resources required.

Feasibility -- the extent to which an evaluation is appropriate and practical for implementation.

Field Test -- the study of a program, project, or instructional material in settings like those where it is to be used. Field tests may range from preliminary primitive investigations to full-scale summative studies.

Focus Group -- a group selected for its relevance to an evaluation that is engaged by a trained facilitator in a series of discussions designed for sharing insights, ideas, and observations on a topic of concern.

Follow-up -- actions taken to maintain the strengths and address the weaknesses that were identified in an evaluation.

Forced-choice Response -- a format for a test, assessment, rating, or survey item where the respondent is given a limited number of options from which to select an answer. In cases where there is a correct or best answer, the other options are referred to as distractors. Examples of forced-choice items are multiple-choice questions, true/false items, checklists , and a five-point rating scale.

Format -- the structure of assessment instruments, evaluation forms, and materials. Format includes mode of delivery (such as paper versus computer, in-person versus telephone, audio tape versus video tape), and general design or layout.

Formative Evaluation -- evaluation conducted while a program is in process, designed and used to promote growth and improvement in a learner's performance or in a program's development.

Generalizability -- the extent to which information about a program, project, or instructional material collected in one setting can be used to reach a valid judgment about how it will perform in other settings.

Goal -- a statement of intent or an end that a person or a group strives to attain. A goal tends to be more general than an objective.

Information -- the knowledge about the attributes and performance of leaners and/or a program, based on assessments, documentation, and data sources used in the assessment and evaluation processes.

Informed Consent -- prior to the collection of information and/or its release in evaluation reports, an agreement by program participants that their names and/or the confidential information supplied by them may be used in specified ways, for stated purposes, and in light of possible consequences.

Instrument -- an assessment device adopted, adapted, or constructed for the purposes of the evaluation.

Interview -- a series of orally-delivered questions designed to elicit responses concerning attitudes, information, interests, knowledge, and opinions. Interviews may be conducted in person or by telephone, and with one individual or a group. The three major types of interviews are: (1) structured, where all questions to be asked by the interviewer are specified in advance; (2) semi-structured, where the interviewer can asked other questions and prompts in addition to the specified questions; and (3) unstructured, where the interviewer has a list of topics, but no or few specified questions.

Item -- a single question, problem, or task used to assess a student.

Item Analysis -- a technique used to analyze responses to objective test items. The technique is used both to improve the quality of items and to enhance interpretation of results. This technique shows the difficulty of the items and the extent to which each item properly discriminates between high achieving and low achieving learners.

Knowledge -- the sum of the information and experience an individual has acquired or learned and is able to recall or use.

Legally Defensible -- an action, conclusion, or statement that can be upheld under current legislation, governmental mandates, and court decisions.

Mean -- a measure of central tendency calculated by dividing the sum of all the values by the number of the values.

Measure (noun) -- an instrument or device that provides data on the quantity or quality of that aspect of a program being evaluated.

Measure (verb) -- to classify or estimate, in relation to a scale, rubric, or standard, the degree of quality or quantity of that aspect of teaching being evaluated.

Method of Assessment -- the techniques or instruments used to measure attributes and/or behaviors. Examples include rating scales, observation checklists, structured interviews, and portfolios.

Method of Data Collection -- the specific means used to get information. Essentially, this includes the data forms and procedures necessary to define the specifics of the evaluation model or system. The five methods commonly used are test scores, observation, reflection, ratings, and portfolios.

Model -- an example of a coherent method, approach, procedure, or strategy of program evaluation, as defined by its key or unique assumptions, propositions, attributes, supportive theory, research, practical precedent, or foundation, and which implicitly defines accomplished or good teaching.

Monitoring -- the checking on a process or a person to verify that progress is being made, required activities are occurring, assessment and evaluation procedures are being implemented, suggested teaching practices are being tried, prior information is still applicable, earlier decisions can still be justified, and/or standards are being met.

Objectives -- the pre-specified intended outcomes of a program, process, or policy. In the case of education, these are usually in the form of learning and behavioral objectives for students. Professional development objectives may also be part of the teacher evaluation process. Objectives tend to be more specific than goals.

Observation -- one of several methods used to collect data about an individual or group. It may cover individual behavior, the teaching context, and the learning environment. Observing should include the recording of evidence and notes while watching.

Open-ended Response -- a format of a test, assessment, or survey item that calls for the answer to be supplied by the respondent rather than selecting from a list of options. Examples include essay questions, short-answer questions, drawings, and fill-in-the-blank items

Parallel Forms -- two or more forms of a test constructed to be as comparable and interchangeable as possible in their content, difficulty, length, and administration procedures and in the scores and test properties (e.g., means, variance, and standard error of measurement).

Pedagogy -- the art and science of teaching. Some pedagogical skills apply across teaching situations whereas others apply only to specific subject areas (pedagogical content knowledge).

Pilot Test -- a brief, simplified preliminary trial study designed to learn whether a proposed evaluation seems likely to yield valuable results.

Pitfall -- a not easily recognized difficulty believed to be associated with a particular standard. These are mistakes that would be made out of ignorance of the import and intent of a standard.

Population -- all the persons in a particular group.

Post-test -- an instrument that assesses attitudes, knowledge, and/or behaviors after the administration of a program, project, or instructional material.

Pretest -- an instrument that assesses attitudes, knowledge, and/or behaviors prior to the administration of a program, project, or instructional material.

Procedures -- the directions for implementing an evaluation process. Procedures specify how the evaluation is to be conducted, designated timelines, persons responsible, forms to be used, documentation to be provided, the analysis plan, and the steps to be followed.

Profile -- a representation of an individual, group, or program based on a number of attributes, measures, or dimensions that use the same scale.

Protocol -- (1) the rules and formalities that guide the administration and scoring of an assessment and the implementation of an evaluation. (2) A record or document of evidence and information relating to an assessment or evaluation.

Purpose -- the primary reason or intended use that provides direction for the design, interpretation, and use of an assessment and evaluation system.

Qualitative Data -- information presented and/or summarized in narrative form, for example, written expressions descriptive of a behavior or product.

Quantitative Data --information presented and/or summarized in numerical form; for example, scores on a paper-and-pencil test or on a five-point analytical scale.

Questionnaire -- an instrument consisting of a series of queries and statements that is used to collect data and information from an individual.

Random Sampling -- drawing a number of individuals from a larger group or population, so that all individuals in the population have the same chance of being selected.

Reliability -- ameasure of how consistent the results are.

Responses -- the answers to test, interview, or questionnaire items.

Sample -- a part of a population.

Scale -- a series of numerical or descriptive ratings on a continuum used to assess or judge specific levels of performance.

Score -- a specific value in a range of possible values describing the performance of a student.

Self-report Instrument -- a device in which persons make and report judgments about the functioning of their project, program, or instructional material.

Skill -- the ability to use knowledge in a practical manner.

Stakeholder -- any person legitimately involved in or affected by a program, such as local industry, parents, county commissioners.

Standard -- a description of the expected level of performance that describes minimum competence in relation to a critical score or other measure of student performance.

Standard Deviation -- a calculated number that describes the extent to which scores are spread out from the mean.

Statistic -- a summary number typically used to describe a characteristic of a sample and from which inferences about the population represented by the sample are made.

Stratified Random Sample -- a grouping achieved by dividing the population to be surveyed into a number of nonoverlapping classes or categories which together include all cases, followed by taking cases at random from within the categories.

Summative Evaluation -- an evaluation designed to present conclusions about the merit or worth of a student's performance.

Test (noun) -- an assessment instrument consisting of a sample of items or tasks from a particular domain and that can provide an estimate of performance in that domain

Test-retest Reliability -- the extent to which two administrations of the same test to the same group of individuals yield consistent results.

Validation -- the process of determining the appropriateness, meaningfulness, and usefulness of a measure, an instrument, or an assessment process, and of the inferences made from the results of it.

Validity -- related to the purposes of the evaluation, the degree to which inferences drawn about an individual's knowledge, skills, attitudes, and behaviors from the results of assessment methods used are correct, trustworthy, and appropriate for making decisions about students.

Value -- an estimation or a measure of the merit and/or worth of a teacher in terms of the intrinsic qualities of the individual teacher (merit) and of the teacher's potential benefit to the school and its students (worth).

Variable -- a characteristic of students that can take on different values: for example, achievement, skill, attitudes, and behavior.

Verification -- the process of checking the accuracy of data and information about the teacher's experience, training, performance, and other attributes.