Section 3: Evaluating, Implementing & Managing Instructional Programs and Projects

CIPP vs. Kirkpatrick's 4 levels

I enjoyed reading about these 2 models of evaluation. Each of the models seems to have their own advantages.  Kirkpatrick’s theory of evaluation extends goal-based evaluation models and "translates into four levels of evaluation...known as reaction, learning, behavior, and results" (Eseryel, 2002, p. 2). Kirkpatrick's goal-based, four-level model of evaluation of training is in contrast to modern systems approaches to evaluation of training that include the Context, Input, Process, Product (CIPP).

Each of these models can be used effectively to create different teaching methods and design. I think it's clear that Kirkpatrick's model is better utilized after a project is complete. I think it should be used as a reflection for how to better prepare and create future models. The advantage of using Kirkpatrick’s model after the data is collected would be to create a better environment based on the models information the next time this model is updated. The disadvantage is that it doesn't allow for adjustments. This is where the CIPP model is more effective, especially in the process and product phases. From my own experience using tech in the classroom can be effective but, sometimes not enough training has been done for students to use certain technologies. It is just assumed they will know what to do. Therefore, an adjustment is often times necessary throughout the process to correct this. While working in a middle school classroom I have seen students who were very technology literate and also some who did not have a clue how to preform basic task using certain technology.

Tyler’s Evaluation Model

1. Begin with the behavioral objectives that have been previously determined. Those objectives should specify both the content of learning and the student behavior expected.

2. Identify the situations that will give the student the opportunity to express the behavior embodied in the objective and that encourage this behavior.

3. Select, modify, or construct suitable evaluation instruments, and check the instruments for objectivity, reliability, and validity.

4. Use the instruments to obtain summarized or appraised results.

5. Compare the results obtained from several instruments before and after given periods in order to estimate the amount of change taking place.

6. Analyze the results in order to determine strengths and weaknesses of the curriculum and to identify possible explanations about the reason for this particular pattern of strengths and weaknesses.

7. Use the results to make the necessary modifications in the curriculum.

     The Tyler’s Objective-Centered Model, as stated in the name this model takes an objective approach to education evaluation. The model emphasis is on consistency, for the learning experience, the educational objective and the overall outcome. The Tyler model has several advantages: It is relatively easy to understand and apply. It is rational and systematic. It focuses attention on curricular strengths and weaknesses, rather than being concerned solely with the performance of individual students. There are four main principles of teaching defined by Tyler; the first is to define appropriate learning objectives. Second is to establish useful learning experiences. Tyler indicates his belief that children learn through exploration of their environment and he states” Children should become actively engaged in what the world is like.” The third principal is to organize the learning experiences so they have the best cumulative effect on the students learning. What Tyler places as central to his model is effectively organizing the learning activities and keeping them consistent and connected. There are three major criteria in building organized learning experiences: Continuity, Sequence and Integration. (Prideaux,2003) .

How I would use Tyler’s method to evaluate my instruction?

      I would look at the objectives that were already in place and talk with the students to determine if those were effective, and then gather input from faculty, administration and parents to determine if there were other options available now, that were not available during the last evaluation that would increase learning. With the fast changing computer technology education evaluation has a difficult time staying current. The development of new software and technology has teachers using computers in the classroom in totally different ways than they have in the past. I would repeat the evaluation annually using newly constructed or modified instruments to measure, compare and contrast results. One of the criticisms of this model is the amount of time that will be needed to construct the behavior objectives; this model has a narrow interpretation of objectives and the curriculum is restricted to the range of student’s skills and knowledge.

                                                    Stake’s Responsive Model

 This model recommends an interactive and recursive evaluation process that embodies these steps:

••The evaluator meets with clients, staff, and audiences to gain a sense of their perspectives on and intentions regarding the evaluation.

••The evaluator draws on such discussions and the analysis of any documents to determine the scope of the evaluation project.

••The evaluator observes the program closely to get a sense of its operation and to note any unintended deviations from announced intents

••The evaluator discovers the stated and real purposes of the project and the concerns that various audiences have about it and the evaluation.

••The evaluator identifies the issues and problems with which the evaluation should be concerned. For each issue and problem, the evaluator develops an evaluation design, specifying the kinds of data needed.

••The evaluator selects the means needed to acquire the data desired. Most often, the means will be human observers or judges.

••The evaluator implements the data-collection procedures.

••The evaluator organizes the information into themes and prepares “portrayals” that communicate in natural ways the thematic reports. The portrayals may involve videotapes, artifacts, case studies, or other “faithful representations.”

••By again being sensitive to the concerns of the stakeholders, the evaluator decides which audiences require which reports and chooses formats most appropriate for given audiences. ( Glatthorn, 1987)

         A responsive evaluation study is a search and documentation of program quality. (Mathison,2005) one of the advantages of the responsive model is the  high value it places on personal experience.  It looks at the disciplines and the way people perceive their quality and worth. This evaluation also looks at program activity, determining how unique this program of instruction and drawing attention to the unanticipated side effects of the program efforts. The responsive evaluation considers the culture of the people. It seeks to identify their concerns and is sensitive to their values, also involving them in different phases throughout the evaluation. The responsive model has a great value in its flexibility, once client concerns have been identified. Its weakness would seem to be the amount of attention it gives to the concerns of the staff and the judgment of the students. The worth of the program is determined by how people subjectively perceive what is going on, and manipulation of the outcome for a certain agenda could easily be accomplished using this method.

How would I use a responsive evaluation of my instruction

I would first make a plan of observation, and then I would arrange for various people to observe my instruction paying special attention to the activities more than the intent.

Meet with administration and staff to get their perspective on the success and short comings of the curriculum.

Document all responses and note any differences.

Prepare a narrative of the findings and share with the observers and the administration allowing them to comment on the various findings.Prioritize the findings in order to make the most effective changes, and work with administration to make the recommended adjustments.

                                                                             References 

                                 Glatthorn, A. A. (1987). Curriculum leadership.

                                  New York: HarperCollins. 

                               

                                 Mathison,Sandra (2005). Encyclopedia of evaluation.

                                 Thousand Oaks, California:Sage Publications,Inc.

                                   

                                   Prideaux,D. (2003). Curriculum design:ABC of learning and teaching in                                  medicine.British Medical Journal,326(7383), 268-270.