Content of the Evaluation Component

The ability to fully understand both the big picture of your program and the individual pieces that make up that big picture is a must. Evaluation design requires dedicated thinking. First, you need to consider your organization’s definition of success—the “so what?” factor. Then you must determine the relationship between the expected outcomes and the activities described in the proposal. Finally, you need to identify the most important aspects of the program, then identify why it is important to evaluate them.
Organizations conduct evaluations primarily to accomplish six purposes:
1. Find out whether or not the hypothesis was correct: Did what the organization originally proposed actually do what the organization expected that it would?
2. Determine if the methods that were specified were indeed used and the objectives met.
3. Determine if an impact was made on the need identified.
4. Obtain feedback from the clients served and other members of the community.
5. Maintain some control over the project.
6. Make midcourse corrections along the way to increase the program’s chances of success.
 
When preparing the evaluation section of the proposal, answering the following questions will help you frame what you will say:
1. What is the purpose of your organization’s evaluation?
2. How will the findings be used?
3. What will you know after the evaluation that you do not know now?
4. What will you do after the evaluation that you cannot do now because of lack of information?
5. How will the lives of the people or community you serve be better?
 
The focus of this workbook, as in previous editions, does not allow for detailed information on program evaluation methods. A free resource you should consider reviewing as an accompaniment to this workbook is the W. K. Kellogg Foundation Evaluation Handbook, which can be found on the foundation’s website (www.wkkf.org).
That said, here is a broad overview that can provide some assistance as you determine the parameters most appropriate for your project. Generally, there are two approaches to data collection: quantitative methods and qualitative methods.
Quantitative methods are, as their name implies, methods to quantify (measure or count) data. They directly answer the question, “How much did we do?” Using this method, you collect data that you can analyze statistically, using averages, means, percentiles, and the like. These analyses allow you to make statements about cause-and-effect relationships. Employ quantitative methods for questions focused on
• Understanding the quantities or frequency of particular aspects of a program (such as number of enrollees or number of dropouts)
• Determining whether a cause-and-effect relationship is present
• Comparing two different methods seeking to achieve the same outcomes
• Establishing numerical baselines (through such means as pretests, posttests, and quarterly or yearly follow-ups)
 
Qualitative methods, in contrast, are based on direct contact with the people involved with a program. These methods consist of interviews (group or individual), observation (direct or field), and review of selected documents. According to the Nonprofit Good Practice Guide Glossary (www.npgoodpractice.org/Glossary), this approach “implies an emphasis on processes and meanings that are rigorously examined, but not measured in terms of quantity, amount, or frequency.” Employ qualitative methods for questions focused on
• Understanding feelings or opinions about a program among participants, staff, or community members
• Gaining insight into how patterns of relationships in the program unfold
• Gathering multiple perspectives to understand the whole picture
• Identifying approximate indicators that clients are moving in the “right” direction
 
In other words, pretests and posttests are not the only measures of success. By taking the time to think clearly and strategically up front, an organization can come up with a creative and valuable evaluation design that incorporates both quantitative and qualitative methods. For example, Some City Senior Center staff might use a pretest and posttest (quantitative method) to measure each new Spanish-speaking member’s individual success in grasping the new diabetes management protocols being taught in the new diabetes management classes, and they might also observe the classroom and the facility in which the curriculum is taught to better understand each member’s experience (qualitative method).
The evaluation component should highlight the data collection methods you plan to employ for the program. Like every other component of the proposal, the evaluation should connect directly with both the objectives and the methods. If your objectives and methods were crafted as recommended—meaning they are measurable and time-specific—that will make the task of preparing a good data collection plan and proposal evaluation much easier.
Take a look at the Sample Evaluation Component prepared for the Senior Latino Community Outreach Pilot Project.
Sample Evaluation Component
Our formal referral system and feedback mechanism will serve a dual purpose. First, we can track all referrals of new Latino seniors, which services we are able to provide, and where the referrals are coming from. Second, our feedback loop will allow us to gather information about member satisfaction with our services, which we will review every other month. This is of particular importance since one of our objectives with the program is to grow and institutionalize our collective cultural competence. The satisfaction reports will allow us to fine-tune our outreach and newly developed Spanish language program offerings.
Specific to our diabetes self-management class, we will employ a combination of evaluation tools. We will use pretests and posttests with every new monolingual Spanish-speaking senior who participates in the classes. We will also take the blood sugar level of each new senior at the beginning of the classes, and monitor each senior’s blood sugar level once weekly for three months upon completion of the classes. This monitoring will be done by the nurse, who will use a glucometer to measure each participant’s levels. The medical readings will be documented in the client’s file every week for three months to determine whether participation in the self-management classes has an effect on the seniors’ ability to successfully manage their blood sugar levels and therefore effectively manage their diabetes.
 
 
Answer the questions in Worksheet 6.1A to begin planning your evaluation section. The sample answers in Worksheet 6.1B will help you get started. Be sure to refer to the Sample Evaluation Component before writing the organization’s evaluation section. When you complete your evaluation section, review it with the Evaluation Review Questions.
WORKSHEET 6.1A: Evaluation Planning Questionnaire
Winning Grants Step by Step,Third Edition. Copyright © 2008 by JohnWiley & Sons, Inc. All rights reserved.
017
Winning Grants Step by Step,Third Edition. Copyright © 2008 by JohnWiley & Sons, Inc. All rights reserved.
018
WORKSHEET 6.1B: Evaluation Planning Questionnaire Example
Winning Grants Step by Step,Third Edition. Copyright © 2008 by JohnWiley & Sons, Inc. All rights reserved.
019
Winning Grants Step by Step,Third Edition. Copyright © 2008 by JohnWiley & Sons, Inc. All rights reserved.
020
Evaluation Review Questions
1. Does the evaluation section focus on assessing the project results?
2. Does it describe how the evaluation will assess the efficiency of program methods?
3. Does it describe who will be evaluated and what will be measured?
4. Does it state what information will be collected in the evaluation process?
5. Does it state who will be responsible for making the assessments?
6. Does it discuss how the information and conclusions will be used to improve the program?
7. Does it provide the organization’s definition of success?
If the program is successful and you are able to document that success through evaluation, you are probably going to want that program to continue. Step Seven addresses the need to plan now for program sustainability beyond the initial funding.
..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset