Interpretation of Student Feedback Data and Responding to it

The Productive Pedagogies Framework

The productive pedagogies framework has been developed as a research tool and metalanguage to support teachers to critically reflect on their teaching practices (Mills et al, 2009). Since the primary focus of the new SU student feedback questionnaire is to support lecturers in their professional development, this framework was deemed appropriate. The productive pedagogies framework offers lecturers and Heads of Departments opportunities for discussions on aspects such as:

 

Intellectual Quality
The intellectual quality dimension of the productive pedagogies model stresses the importance of all students, regardless of background and perceived academic ability, being presented with intellectually challenging work (Darling-Hammond, 1997; Newmann & Associates, 1996; Sizer, 1996; Boaler, 2002; Sarra, 2006; Perry, Steele, & Hilliard III, 2003).

 

Connectedness
Connectedness refers to a focus on developing students’ deep understanding in worthwhile and meaningful contexts and that this will require students to use higher order thinking that goes beyond simple recall, recognition, and reproduction to analysis, synthesis, evaluation, and production of ideas and performances.

 

Supportive Classroom Environment
In arguing for the creation of a supportive classroom, the productive pedagogies framework suggests that students be given a voice in the classroom in order to have some say over the direction that activities take within various units of work, that explicit criteria be provided to students so that expectations are clear, and that a classroom environment is created where students are prepared to take risks with their learning. While care is central to good teachers’ work (Lingard et al, 2001), Hargreaves (2003:47) has stated that, “Care must become more than charity or control: it must become a relationship in which those who are cared for (pupils or parents) have agency, dignity and a voice”.

 

The productive pedagogies framework structured the questionnaires which students complete.  Thus student feedback at Stellenbosch University is focused on the elements of the evaluation that matter most: Intellectual quality, connectedness, a supportive classroom environment, and the overall effectiveness and quality of the learning experience, e.g. whether the module outcomes were achieved and whether students were clear on assessment expectations.  See the next section on how to apply them when you look at student feedback.

Making Sense of Student Feedback Data

There is a diverse body of opinion and scholarship available regarding the validity and usefulness of student feedback (Walsh 2014). Following is only a few suggestions for interpreting your student feedback data:

  • Read your evaluations with a thick skin.
    It is difficult to absorb criticism if you feel defensive or insulted. If you want the evaluations to be of use, prepare for the worst before you start reading them because you never know when you are going to get an insensitive, rude, or even vulgar comment. If you receive such an inappropriate comment, or a particularly harsh score, try not to let it colour the overall message of the evaluations as a whole.
  • Look for general trends and overlook the outliers.
    While you may learn something through a close reading of each individual evaluation, the best approach is to look for the general trends. What are the recurring observations made by students? What do the numbers and the comments in these trends reveal about your teaching strengths and weaknesses? As for the outliers, bear in mind that even the greatest teacher in the most highly successful class may get one or two evaluations that contain comments or ratings that don’t match up with those of the other students. Although in some cases these outliers may reveal something to you about your teaching, in most cases you can safely ignore these and look for broad trends indicated by the class as a whole. Note that this applies to both unusually positive and unusually negative outliers.
  • Seek explanations, but don’t explain away all the data.
    If you want to use evaluations to improve your teaching, it doesn’t help to read them in a defensive posture. While you may want to consider whether there may be reasons, independent of the quality of your teaching, that might be affecting your scores, you don’t want to explain away all feedback so that you can gain nothing from it. Even if some factors really do bias the ratings somewhat, the existence of some bias does not delegitimize all the data. Rather, try to mentally correct for the bias, and then look at what the numbers and comments tell you, all possible bias aside. Also, note that the research has shown that many common conceptions about bias (such as lecturer appearance, gender, rank; size of class; perceived easiness) are inaccurate. That is, many factors that you might think bias students for or against you probably don’t.
  • Focus on the elements of the evaluation that matter most.
    If there’s something you’re working on in your teaching, pay particular attention to that portion of the form to gauge how it’s going.
    The most important categories according to the SU questionnaire are:
    Intellectual Quality (e.g. do students acquire skills to analyse and solve problems; are they encouraged to take responsibility for own learning; did contact sessions help them master the learning material)
    Connectedness (e.g. resources were used effectively to master learning material; learning technologies were used in meaningful ways; learning technologies enriched students’ learning experience; feedback on assessments helped students to improve their weak points; lecturer helped students improve their understanding of the learning material)
    Supportive Classroom Environment: (e.g. an atmosphere of mutual respect and support for everyone; lecturer was approachable; sufficient opportunity to contribute meaningfully to activities)
    Overall Effectiveness and Quality (e.g. module outcomes were achieved; relevance of the module to qualification was clear; it was clear before each assessment what would be expected of students).
  • Use several sets of student feedback data
    Review student feedback data from different modules over an extended period of time to see if you can identify trends in terms of your teaching that cut across all your modules.

 

For module improvement, multiple feedbacks of a single module over time and even with different lecturers can be useful.

  • Prepare to change one or two things about how you teach.
    Student feedback is meant to help you identify your teaching strengths and weaknesses and to help you decide where to focus your improvement efforts. You may have reasons to feel skepticism about these evaluations or frustration about the role they play in your professional life, but if you are genuinely interested in improving your teaching, it will help to set those other concerns aside and to read your teaching evaluations with a determination to find the most important things that you can do to become a better teacher.
  • Choose one or two areas and commit to improving yourself in those areas.
    When you have identified the general trends of the evaluation, corrected for possible biases, and persuaded yourself that the trends accurately reveal something about your teaching, focus in on one or two substantial things to work on. You can’t change everything about your teaching overnight and trying to do so may do more harm than good. Meet with a peer, a chair/HOD, a colleague, or the teaching & learning advisor allocated to your faculty to discuss ways to make these improvements.  Find the teaching and learning advisors here.
  • Supplement the end-of-semester evaluation form.
    The common form used at end-of-semester evaluations, shouldn’t be the only way you are getting feedback from your students. Design smaller versions of this evaluation form and give it out to students periodically during the semester. You may want to focus on something you are working on, perhaps some new assignment, method, or technology. At a minimum, use a mid-semester feedback form, asking general questions about what students like about the class, what they dislike, and what suggestions they have for you to help them learn more. Note that if you solicit this feedback, be prepared to act on it and tell students how you have done so.