Appendix A

Rating Guides for Courseware and Facilitation

The four levels of evaluation assess a course delivery from the standpoints of reaction, learning, transfer, and impact or results. They do not assess the overall course delivery or the quality of facilitation.

Rating guides for delivery and facilitation take a much broader view by assessing the entirety of a course and the facilitation. Rating guides for delivery are generally scaled 0-5, with a score of 0 meaning missing or absent and 5 meaning high or excellent. Rating guides for facilitator assessment are usually scaled from 0-4, with 0 being not at all and 4 being to a very great extent.

Figure A-1 is a rating guide for classroom delivery. Such instruments review the quality and completeness of the facilitator guide, design issues, the evaluation plan, learning strategies, participants’ guides, media, and the train-the-trainer process.

Figure A-2 is a rating guide for online delivery. This instrument reviews the quality and completeness of the more general aspects of the course, design, the evaluation plan, learning strategies, technical support, and any enhanced capabilities.

Figure A-3 is a rating guide for a self-study. It covers the quality and completeness from a general aspect and then addresses design issues, learning strategies, and the evaluation plan.

Figure A-4 is a rating guide for facilitation. It is a weighted instrument that allows you to distribute 100 points across the six major areas of assessment. Then, the evaluator rates the facilitator according to the areas of assessment. By multiplying the weight by the rating and summing the data, you arrive at a score between 0 and 400. Your HRD organization then sets the criterion score for acceptable performance.

For assistance in using these instruments, you may contact the author at [email protected].

Figure A-1. Rating Guide for Talent Development Review—Classroom Delivery

A six-point rating scale (0-5) is used for the talent development quality review:

0. Missing/Absent—no evidence exists. The training material or learning experience is incomplete without this factor or quality being present.

1. Poor/Inconsistent—evidence for a factor or quality exists, but the quality is poor or inconsistent. The training material or learning experience may be less effective without this factor or quality being improved or consistently applied.

2. Better than Level 1 but not at Level 3.

3. Good/Consistent—evidence for a factor or quality exists, and the material is of good or consistent quality. The training material or learning experience may be effective due to the presence of this factor or quality in the course.

4. Better than Level 3 but not at Level 5.

5. High/Excellent—evidence for a factor or quality exists, and the material is of excellent or highly consistent quality. The training material or learning experience may be very effective due to the excellent or outstanding qualities of the course.

Adapted from Performance Advantage Group, 2016.

Figure A-2. Rating Guide for Talent Development Review—Online and Virtual Classroom

A six-point rating scale (0-5) is used for the talent development quality review:

0. Missing/Absent—no evidence exists. The training material or learning experience is incomplete without this factor or quality being present.

1. Poor/Inconsistent—evidence for a factor or quality exists, but the quality is poor or inconsistent. The training material or learning experience may be less effective without this factor or quality being improved or consistently applied.

2. Better than Level 1 but not at Level 3.

3. Good/Consistent—evidence for a factor or quality exists, and the material is of good or consistent quality. The training material or learning experience may be effective due to the presence of this factor or quality in the course.

4. Better than Level 3 but not at Level 5.

5. High/Excellent—evidence for a factor or quality exists, and the material is of excellent or highly consistent quality. The training material or learning experience may be very effective due to the excellent or outstanding qualities of the course.

Adapted from Performance Advantage Group, 2016, and Wright (2011).

Figure A-3. Rating Guide for Talent Development Review—Self-Studies

A six-point rating scale (0-5) is used for the talent development quality review:

0. Missing/Absent—no evidence exists. The training material or learning experience is incomplete without this factor or quality being present.

1. Poor/Inconsistent—evidence for a factor or quality exists, but the quality is poor or inconsistent. The training material or learning experience may be less effective without this factor or quality being improved or consistently applied.

2. Better than Level 1 but not at Level 3.

3. Good/Consistent—evidence for a factor or quality exists, and the material is of good or consistent quality. The training material or learning experience may be effective due to the presence of this factor or quality in the course.

4. Better than Level 3 but not at Level 5.

5. High/Excellent—evidence for a factor or quality exists, and the material is of excellent or highly consistent quality. The training material or learning experience may be very effective due to the excellent or outstanding qualities of the course.

Adapted from Performance Advantage Group, 2016.

Figure A-4. Rating Guide for Facilitation

Below is a list of behaviors describing the demonstration of facilitation and presentation skills. As an assessor, weight each behavior with the total being 100. Then use the following scale to indicate the extent to which the individual demonstrates the listed behaviors.

0 = not at all, 1 = to very little extent, 2 = to a moderate extent, 3 = to a great extent, 4 = to a very great extent

Upon completion, multiply the weight of each behavior by the rating and add up the total column. The total will be between 0 and 400. This is the participant’s assessment score. The criterion reference score for passing is _______.

Adapted from Performance Advantage Group, 2016.