review of student feedback objectives and the forms in which such feedback was collected;

 review of student feedback collected;

 observation of learning and teaching activities in class;

 interviews with selected students;

 interviews with teachers (full-time, part-time and visiting lecturers and/or other tutors);

 interviews with programme and course coordinators and other related administrative staff.

 rating of the course arrangements (e.g. learning and teaching environment, support from the general office);

 rating of the course contents (including assignments and assessments);

 rating of teacher performance (including tutors);

 other comments related to the course and the teacher(s) (including tutors).

Advantages

The Stage I findings showed that the student feedback process conducted only at the end of the course was mainly summative in nature (for further discussion of summative evaluation and assessment, see Gioka, 2008; Glickman et al., 2009; Morrow, 2005; Patton, 2011; West, 1975). In general, summative student feedback has several key objectives and advantages.

First, collecting one-off, summative student feedback at the end of a course is easy and convenient in terms of planning and implementation, particularly when gauged by time required and human and other resources employed (McDowell, 2008; Morrow, 2005; Siu, 1998, 2000). It is easy and consistent to carry out collection of such a feedback for different programmes and courses. This is also why this form of student feedback has been widely adopted by many universities. Furthermore, consistency is now an important requirement and concern in course evaluation. People responsible for evaluating courses generally prefer to eliminate all non-standard factors and items, including non-standard tools for collecting feedback and evaluation, collected data, and methods of analysis which are relatively more difficult to manage (McDowell, 2008; Rudney and Guillaume, 2003; Siu, 2007).

One advantage of adopting a standardised summative student feedback format is that it enables comparison between different programmes, courses and teachers (Davidson-Shivers, 2006; Kimball, 2001; King, 2003; McDowell, 2008; Rudney and Guillaume, 2003; Siu, 2000; West, 1975). Although a number of researchers have argued that it is inadequate to evaluate and compare different programmes and courses using the same means of evaluation (such as standardised, inflexible methods and tools), many programme planners, coordinators and lecturers still expect to use more standardised means of assessment (O’Donoghue, 2010; McDowell, 2008; Rudney and Guillaume, 2003; Siu, 1998, 2000). To eliminate discrepancies due to differences in tools and a lack of quantitative elements, summative student feedback conducted at the end of a course is generally considered the most reasonable, rational and widely applicable approach (for standards of evaluation and assessment, see Aylett and Gregory, 1996; Cascallar and Cascallar, 2003; Ellis, 1993; McDowell, 2008; Thackwray, 1997).

Another advantage of obtaining summative student feedback at the end of a course relates to time and post-evaluation management issues. First, there is no pressure to take prompt or immediate action (i.e., data analysis, feedback and action for change and improvement) after feedback is collected (Siu, 2007). For example, this study highlighted the intensive nature of today’s programme structures and demonstrated that administrative and academic staff bear a heavy workload. According to a programme coordinator interviewed for this study, collecting student feedback at the end of a course implies that it is not so urgent to do so, or at least that academic and administrative staff are not under pressure to take immediate action to analyse data and make suggestions for improvement. In many Asian universities, a department may now run a large number of concurrent programmes (covering more than a hundred courses), particularly due to the rapid increase in self-financed programmes. Further, as pointed out by programme coordinators and teachers interviewed in this study, analysis and follow-up action immediately after the student feedback has been collected is nearly impossible.

Common problems and limitations

Summative student feedback entails a number of problems and limitations that critically affect the quality of learning and teaching. In practice, some of its advantages and benefits can also give rise to problems and limitations. While summative student feedback can be considered a type of ‘outcome evaluation’ (Gioka, 2008), its key limitation is that this type of feedback cannot provide prompt and timely guidance on action leading to immediate improvements (Siu, 2007).

Although consistency in collecting and analysing student feedback is important, collecting student feedback in a rigid and consistent fashion may sometimes have its disadvantages. For example, it may result in student feedback collection becoming a routine administrative requirement, rather than a constructive form of action taken to improve the programme and course evaluation process and inform learning and teaching activities. This is contrary to the core objective of collecting student feedback. In addition, different programmes and courses require different learning and teaching environments, or different learning and teaching objectives that may be ‘flattened’ by applying a universal evaluation approach. It is neither effective nor constructive to utilise a standardised evaluation approach regardless of the nature of the programme or course (Davidson-Shivers, 2006; McDowell, 2008).

As was pointed out by a teacher who was interviewed, student feedback provides ratings in different areas related to programme planning, course arrangements and contents, learning and teaching activities, and teacher performance. Summative student feedback generates useful and constructive comments, opinions, and suggestions. It still fulfills the important purpose of guiding ‘future’ improvements (O’Donoghue, 2010; Siu, 2000). However, the major limitation of one-off summative feedback provided at the end of the course is that it cannot inform constructive action that might be taken to improve the teaching of the course before the course is finished. First, the function of student feedback is restricted, or it cannot be maximised. Moreover, as discussed above, collecting summative student feedback fails to provide course leaders and teachers with immediate feedback on how to make immediate improvements to the programme (Gioka, 2008). Further, although students who give feedback have an opportunity to raise their concerns, they do not benefit from their own advice. Finally, teachers cannot benefit from students’ suggestions (i.e. make any improvements) before the course has been completed.

In recent years, more researchers and educators have emphasised that learning and teaching is a continuous process. In particular, learning objectives are now expected to centre more on the learning process, rather than solely on the final output (Coles, 1997; Dew and Nearing, 2004; Fisher, 2003). Thus, it is important to obtain feedback on different stages of a course; this goal cannot be attained through summative student feedback (Gioka, 2008).

A key finding of this study is that the motivation for students to give feedback was low. Several of the students who were interviewed indicated that they did not take the feedback questionnaires seriously. This was also reflected in the student feedback questionnaires, where students rarely provided written comments in the ‘other comments’ section of the questionnaire. One common perception among students was that this type of summative evaluation did not provide any practical benefit for their learning of the course they had just completed. Some of the students also commented that they did not believe the programme and course coordinators and teachers would take their feedback seriously. Thus, students placed a low value on feedback questionnaires.

In the same way, the low level of student motivation to give feedback also implies that the feedback provided via students’ comments, opinions and suggestions was low in quality. A ripple effect then occurred, in that the programme and course coordinators, and especially the teachers, did not take student feedback seriously. Course teachers were most concerned about whether they had reached the rating thresholds applied by the school and university. Many of the teachers also admitted that they seldom read other written comments provided by students unless they saw that the rating of their teaching performance was low or sensed that the student had raised a complaint.

In summary, it would be unfair to say that summative student feedback serves no useful purpose. One teacher interviewed for this study indicated that he paid attention to negative comments made by students to improve his future teaching. Nevertheless, summative feedback cannot bring about any change to the course just completed or directly benefit those students who supply comments.

Advantages

The major advantage of formative student feedback is that it provides an opportunity for improvements to be made during the learning and teaching process (Stockley, 2006; Tessmer, 1993). It can be considered a process where both learning and teaching activities and the evaluation and improvement process can be run in parallel (Thackwray, 1997). In Stages II and III, due to the feedback collected during the course (i.e. through the mid-stage questionnaire and small group comments), teachers had an opportunity to make improvements to course contents and their teaching performance. This was reflected in the final student feedback, where students commented that their experience of the course improved after they had provided mid-stage feedback.

Formative student feedback also has the advantage that learning and teaching activities can be improved according to students’ changing needs (George and Cowan, 1999; Gioka, 2008; Holmes and Brown, 2000; Nilson, 2010; Tessmer, 1993). Because coordinators and teachers may not be familiar with their students’ backgrounds and learning needs (e.g. age, study experience or expectations of the course) before the course begins, mid-stage feedback may assist them in addressing this gap in their knowledge (Holmes and Brown, 2000; Siu, 1998). Thus, the formative student feedback collected during the course in this study provided teachers with the flexibility and opportunity they needed to adjust their teaching content and methods. For example, in Stage II, the teacher originally assigned students a design project utilising a particular software application that had received highly positive ratings from students in previous academic years (i.e. reflected in Stage I feedback). However, in the mid-stage informal small group meetings, some of the Stage II students informed the teacher that they would prefer to learn how to use a new software application. The teacher therefore modified the project requirements and allowed the students to select their preferred software application in the design implementation phase. As was also pointed out by Andrade and Cizek (2010), Behrendt (2001) and Mann (2006), this type of motivation generated through formative evaluation is important for effective learning (Siu, 1998; Stockley, 2006; Van Evera, 2004). Flexibility and opportunity for change are also major objectives of formative student feedback.

Moreover, students indicated their appreciation of the opportunity to provide informal feedback in small group meetings, and most students were more willing to give their comments on the course. The major reason for this was that students realised that the teachers were taking their feedback seriously and wanted to improve the quality of the course, rather than treating the feedback process as a routine exercise required by the school and university. This became more obvious in the mid-stage student feedback and small group comments.

This increased student motivation was reflected in students providing more (in quantity) and more detailed (in quality) comments in the mid-stage feedback exercise, and teaching practice also improved as a result of the student feedback (compared to Stage I). The qualitative change in teaching practice was highlighted by several of the students who were interviewed. The students perceived that teachers prepared much more thoroughly for the mid-stage feedback exercise, and that they took student feedback more seriously. The arrangements teachers made were different both from those made in the past and from those implemented for other courses. Thus, instead of giving only a marginally positive rating for each item as was the general practice in the past, the students showed they were more serious in providing feedback on each individual item. This was particularly clear because in Hong Kong and many Asian countries, students seldom take this type of student feedback exercise seriously. In treating it as a routine course evaluation exercise, many students seldom give negative and critical comments about a teacher unless the teacher’s performance or attitude is very bad.

As discussed above, this higher level of motivation to provide feedback can result in more and more detailed comments being obtained from students (Nilson, 2010). The questionnaire review conducted in this study showed that students were more willing to provide comments on how to improve the course and teaching performance. As indicated by the teachers, more serious comments gave clearer and detailed guidance to course coordinators and teachers on how to improve the course content and enhance learning and teaching activities.

The previous discussion demonstrates that the advantages and value of formative student feedback lie not only in the objective and content of feedback on how to enhance the course, but also in the creation of a culture of providing continuous feedback for improvement.

Difficulties

Collecting formative student feedback in Stages II and III was more complicated than collecting summative feedback in Stage I. The coordinators and teachers who were interviewed, noted that the process consumed time and resources and that the teachers had to take on additional responsibilities in terms of preparation, implementation, data analysis and follow-up.

In comparison with Stage I, teachers could not rely on a ready-made and standardised means of collecting student feedback: the nature of collecting formative student feedback required them to make adjustments to the ways in which feedback was collected. Despite their significant benefits, the formative student feedback exercises conducted in Stages II and III created significantly increased the teachers’ workloads.¹ As noted by a teacher involved in Stage III teaching, the extra workload became even more obvious when improvements were required and student feedback was translated into action. The teacher pointed out that improvements were required not only to the course contents and learning and teaching activities, but it also required at least fine-tuning of the evaluation methods and contents for the next stage. The same teacher also indicated that this increased workload would have to be maintained if formative evaluation was to remain meaningful.

Data analysis is another major part of the formative student feedback process that presents difficulties for education professionals (Andrade and Cizek, 2010; Davies, 2003). Its challenge lies in the greater potential of formative student feedback to include qualitative components in the matters evaluated (Mann, 2006; Siu, 1998). This then implies a need for analysis of qualitative data, creating a potential increase in workload. Teachers interviewed for this study indicated that this type of data, encompassing diverse and sometimes contradictory comments, was not easy to analyse and distill into suggestions for improvement. Further, the data had to be analysed and suggestions promptly acted on. As indicated in recent research, this difficulty is also the reason why many programme and course coordinators and teachers refuse to utilise formative evaluation and assessment despite its proven advantages (Andrade and Cizek, 2010; Cascallar and Cascallar, 2003; Ellis, 1993; Rudney and Guillaume, 2003; Siu, 2007; Van Evera, 2004). This reluctance is also due to the fact that academic programmes in Asia are progressively intensifying the pressure on academics (Siu, 1998; Burden and Byrd, 2010; Holmes, 2009; Partin, 2009).

Possibilities

Although there are a range of difficulties involved in implementing formative student feedback, enhancing the quality of learning and teaching by applying a continuous and action-based evaluation programme cannot be overlooked as an option (Aylett and Gregory, 1996; Ellis, 1993; Donaldson and Scriven, 2003; Massy et al., 2007; Patton, 2011; Stockley, 2006). Particularly in recent years, educational changes and social expectations on the role of education have led more educators and members of the public to recognise the importance of formative student feedback and consider how it can be effectively implemented (Siu, 2007). This type of recognition and consideration is critically important as a driving force for the government, related education sectors, and the public to agree to put more resources and effort into promoting and utilising formative student feedback and to take action to achieve this end.

Education reforms differing in nature and scale have been implemented worldwide since the mid-1990s, particularly in the Asian region and in developing countries where policymakers have recognised the importance of good quality education. Among all the aspects of such reforms, quality assurance in programme planning, implementation, and evaluation has been highlighted as a key priority (Mok and Chan, 2002; Neave, 2000; Postiglione, 2002; Salmi, 2002). As discussed above, formative student feedback is valuable in enhancing the quality of learning and teaching activities. Thus, education reforms may become a force that pushes more academics and others to give serious consideration to the need for formative student feedback.

The recent emphasis on student-centred learning and outcome-based learning has also made it possible to promote and implement formative student feedback programmes (Nilson, 2010). This is because student feedback has become one of the key factors in education quality assurance. On the programme and course levels, it is important that learning and teaching are continuously evaluated. An action approach to collecting student feedback and making improvements is considered appropriate and effective (Stockley, 2006).

Changes in, and pressures on, higher education systems around the world have intensified through the internationalisation and globalisation of education programmes, with the greater number of students enrolling from different countries and regions making the situation even more complicated (Jiang, 2008). To follow these ‘market’ trends and needs, updated and accurate student feedback is crucial to the success of such programmes and courses. As discussed above, formative student feedback has the advantage of facilitating both evaluation and improvements in a prompt and effective manner. The continuous and action-based nature of formative student feedback can assist in adjusting to the changing and varied needs and preferences of diverse student cohorts. By utilising formative student feedback, problems and deficiencies in learning and teaching matters can be resolved at an early stage to prevent their accumulation (Massy et al., 2007; Tessmer, 1993).

A majority of the programme coordinators and teachers interviewed for this study believed that a greater proportion of academics were currently willing to take an active role in improving the quality of their teaching. In other words, more coordinators and teachers recognised the importance and practical value of obtaining student feedback, and thus of putting more effort into collecting and utilising such feedback.

In recent years, more Asian students have started raising their concerns, including suggestions for improvement. This has meant a change from the past, when many Asian students were reluctant to voice their views and comment on learning and teaching matters. Lin (2004) has also identified this as the reason Asian students found it difficult to adapt to foreign (particularly Western) learning cultures when they studied abroad. This change in learning styles among Asian students in recent years has enabled coordinators and teachers to introduce formative student feedback. Students’ greater willingness for, and more active participation in, providing their feedback can also provide a greater motivation for academics to improve their teaching.

Another practical and positive shift in promoting and implementing formative student feedback is the increase in general support for collecting student feedback (Siu, 2007). Although many administrators and teachers have continuously complained about resource cuts or shortages in university education, the funding provided to support quality assurance has actually increased in recent years (Brown, 2004; Gokulsing, 2008; Higher Education Quality Council, 1995a). Taking Hong Kong as an example, not only has the amount spent on education programme quality assurance increased, but so too has funding for audits conducted by government councils, committees and international departments of universities. Student feedback was collected in a piecemeal manner in the past, whereas more universities in Hong Kong have now set up central quality assurance units. General and teaching support staff now assist with the student feedback collection process. All of these changes made in recent years have helped to eliminate some of the difficulties encountered in coordinating the student feedback collection process (Higher Education Quality Council, 1995b; Latchem and Jung, 2010; Williams, 2008).

Andrade, H.L., Cizek, G.J. Handbook of Formative Assessment, New York: Routledge, 2010.

Aylett R., Gregory K., eds. Evaluating Teacher Quality in Higher Education. London: Falmer Press, 1996.

Behrendt H., ed. Research in Science Education: Past, Present, and Future. Boston, MA: Kluwer Academic, 2001.

Brown, R. Quality Assurance in Higher Education: The UK Experience Since 1992. London: RoutledgeFalmer; 2004.

Burden, P.R., Byrd, D.M. Methods for Effective Teaching: Meeting the Needs of All Students, 5th edn. Boston, MA: Allyn and Bacon; 2010.

Cascallar, A., Cascallar, E. Setting standards in the assessment of complex performances: the optimized extended-response standard setting method. In: Segers M., Dochy F., Cascallar E., eds. Optimising New Modes of Assessment: In Search of Qualities and Standards. Boston, MA: Kluwer Academic; 2003:247–266.

Coles, M. Curriculum evaluation as review and development: the curriculum leader’s role in creating a community of enquiry. In: Preedy M., Glatter R., Levačić R., eds. Education Management: Strategy, Quality, and Resources. Buckingham, UK: Open University Press; 1997:113–126.

Cox, R. Formative Evaluation: Interpretation and Participation. London: University Teaching Methods Unit, University of London Institute of Education; 1974.

Davidson-Shivers, G.V. Web-based Learning: Design, Implementation, and Evaluation. Upper Saddle River, NJ: Pearson/Merril Prentice Hall; 2006.

Davies, L.M. Monitoring and evaluating adult education programs in the District of Columbia. Unpublished thesis. Amberst, MA: University of Massachusetts Amherst; 2003.

Dew, J.R., Nearing, M.M. Continuous Quality Improvement in Higher Education. Westport, CT: Praeger; 2004.

Donaldson, S.I., Scriven, M. Evaluating Social Programs and Problems: Visions for the New Millennium. Mahwah, NJ: Lawrence Erlbaum; 2003.

Ellis, R. Quality Assurance for University Teaching. Buckingham, UK: Society for Research into Higher Education and Open University Press; 1993.

Fisher, M.M. Designing Courses and Teaching on the Web: A ‘How to’ Guide to Proven, Innovative Strategies. Lanham, MD: Scarecrow Press; 2003.

George, J.W., Cowan, J. A Handbook of Techniques for Formative Evaluation: Mapping the Student’s Learning Experience. London: Kogan Page; 1999.

Gioka, O. Teacher or assessor? Balancing the tensions between formative and summative assessment in science teaching. In: Havnes A., McDowell L., eds. Balancing Dilemmas in Assessment and Learning in Contemporary Education. New York, NY: Routledge; 2008:145–156.

Glickman, C.D., Gordon, S.P., Ross-Gordon, J.M. Supervision and Instruction Leadership: A Developmental Approach. Boston, MA: Allyn and Bacon; 2009.

Gokulsing, K.M. The New Shape of University Education in England: Interdisciplinary Essays. Lewiston, NY: Edwin Mellen Press; 2008.

Higher Education Quality Council. Managing for Quality: Stories and Strategies: A Case Study Resource for Academic Leaders and Managers. London: Higher Education Quality Council; 1995.

Higher Education Quality Council. A Quality Assurance Framework for Guidance and Learner Support in Higher Education: the Guidelines. London: Higher Education Quality Council; 1995.

Holmes, A., Brown, S. Internal Audit in Higher Education. London: Kogan Page; 2000.

Holmes, E. The Newly Qualified Teacher’s Handbook, 2nd edn. London: Routledge; 2009.

Jiang, X.P. Why Inter culturalisation? A Response to the Internationalisation of Higher Education in the Global Knowledge Economy. Rotterdam, The Netherlands: Sense Publishers; 2008.

Kimball, S.M. Innovations in teacher evaluation: case studies of two school districts with teacher evaluation systems based on the framework for teaching. Unpublished thesis. Madison, WI: University of Wisconsin-Madison; 2001.

King, K.P. Keeping Pace with Technology: Educational Technology that Transforms. Creskill, NJ: Hampton Press; 2003.

Latchem, C., Jung, I. Distance and Blended Learning in Asia. New York: Routledge; 2010.

Lewin, K. Action research and minority problems. Journal of Social Sciences. 1946; 2(4):34–46.

Lin, C.Y., Taiwanese students in a United States university: expectations, beliefs, values, and attitudes about learning and teaching. Unpublished thesis. Pennsylvania State University, University Park, PA, 2004.

Mann, B.L. Selected Styles in Web-based Educational Research. Hersey, PA: Information Science Publishers; 2006.

Massy, W.F., Graham, S.W., Short, P.M. Academic Quality Work: A Handbook for Improvement. Bolton, MA: Anker Publishers; 2007.

McDowell, L. Students’ experiences of feedback on academic assignments in higher education: Implications for practice. In: Havnes A., McDowell L., eds. Balancing Dilemmas in Assessment and Learning in Contemporary Education. New York, NY: Routledge; 2008:237–250.

Mok, K.H.J., Chan, D.K.K. Globalization and Education: the Quest for Quality Education in Hong Kong. Hong Kong: Hong Kong University Press; 2002.

Morrow, J.R. Measurement and Evaluation in Human Performance, 3rd edn. Champaign, IL: Human Kinetics; 2005.

Neave, G. The Universities’ Responsibilities to Society: International Perspectives. Amsterdam, The Netherlands: Pergamon; 2000.

Nilson, L.B. Teaching at its Best: a Research-based Resource for College Instructors. San Francisco, CA: Jossey-Bass; 2010.

O’Donoghue, J. Technology-supported Environments for Personalized Learning: Methods and Case Studies. Hersey, PA: Information Science Reference; 2010.

Partin, R.L. The Classroom Teacher’s Survival Guide: Practical Strategies, Management Techniques, and Reproducibles for New and Experienced Teachers, 3rd edn. San Francisco, CA: Jossey-Bass; 2009.

Patton, M.Q. Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use. New York: Guilford Press; 2011.

Postiglione, G. Chinese higher education for the twenty-first century: Expansion, consolidation, and globalization. In: Chapman D.W., Austin A.E., eds. Higher Education in the Developing World: Changing Contexts and Institutional Responses. Westport, CT: Greenwood Press; 2002:149–166.

Rudney, G.L., Guillaume, A.M. Maximum Mentoring: an Action Guide for Teacher Trainers and Cooperating Teachers. Thousand Oaks, CA: Corwin Press; 2003.

Salmi, J. Higher education at a turning point. In: Chapman D.W., Austin A.E., eds. Higher Education in the Developing World: Changing Contexts and Institutional Responses. Westport, CT: Greenwood Press; 2002:23–44.

Siu, K.W.M. The evaluation of higher education in a compressed spatial and temporal world. Education Today. 1998; 48(1):9–13.

Siu, K.W.M. Re-construction of learning space for design education. Design and Education. 2000; 8(1):20–28.

Siu, K.W.M. Balance in research and practice: critical reform of research studies in industrial and product design. Global Journal of Engineering Education. 2007; 11(1):15–27.

Somekh, B. Action Research: a Methodology for Change and Development. Maidenhead, UK: Open University Press; 2006.

Stockley, D. Strategies to collect and use online student feedback: Improving teaching through formative evaluation. In: Mann B.L., ed. Selected Styles in Web-Based Educational Research. Hersey, PA: Information Science Publishers; 2006:246–259.

Tessmer, M. Planning and Conducting Formative Evaluations: Improving the Quality of Education and Training. London: Kogan Page; 1993.

Thackwray, B. Effective Evaluation of Training and Development in Higher Education. London: Kogan Page; 1997.

Van Evera, W.C. Achievement and motivation in the middle school science classroom: the effects of formative assessment feedback. Unpublished thesis. Fairfax, VA: George Mason University; 2004.

West, R.W. The Summative Evaluation of Curriculum Innovations. Brighton, UK: University of Sussex Education Area; 1975.

Williams, B.C. Preparing Effective Teachers of Reading: Putting Research Findings to Work for Student Learning. New York: Peter Lang; 2008.