Thanks, Gosia. I'm interested to know: do you think the 10/80/10 model would work for you & your clients, e.g. in light of things like client expectation and the limitations of the tech?
Yes, it seems like a very good framework for the the world of language training I operate in. I have been using it with clients since Aula days and it definitely drives engagement. More difficult to get evidence of mastery as I often do not have access to summative data. My bugbear though is with the last part - feedback! I have been grappling with this topic, as there are lots of issues around effective 'qualitative' feedback, especially if you are really forced to give a 'score' or have limited resources (time and budget) for giving qualitative appraisal to individuals. In addition, language for assessment/feedback is a true Pandora's box, especially with peer feedback:) There is a lot of useful literature about it and models e.g. Martin & White's (2005) appraisal framework but I would be very interested if you could expand of the last 10% - feedback part - how to handle it best across the range of options, with your amazing practical tips!
Yeah, I agree - scaling feedback & assessment is a wicked problem! I’m doing some interesting work with machine learning to explore whether it’s possible to use NLP (basically, a “TA” trained by the SME) to provide meaningful qualitative feedback on text, images, audio and other outputs @ scale.
In the meantime: I’d recommend exploring peer to peer feedback - there’s research to show that with a solid rubric it’s as and in some cases more effective than tutor feedback.
Other strategies:
- provide a walkthrough of an exemplar and have students self assess
- combine a qualitative task with a closing knowledge check to enable application + a score of some sort
- consider the value of the score generated by MCQs etc. Would it be more valuable to provide summary verbal feedback recorded in advance covering common errors etc? The evidence suggests it would.
Love the sctivity toolkit / so practical!
Thanks, Gosia. I'm interested to know: do you think the 10/80/10 model would work for you & your clients, e.g. in light of things like client expectation and the limitations of the tech?
Yes, it seems like a very good framework for the the world of language training I operate in. I have been using it with clients since Aula days and it definitely drives engagement. More difficult to get evidence of mastery as I often do not have access to summative data. My bugbear though is with the last part - feedback! I have been grappling with this topic, as there are lots of issues around effective 'qualitative' feedback, especially if you are really forced to give a 'score' or have limited resources (time and budget) for giving qualitative appraisal to individuals. In addition, language for assessment/feedback is a true Pandora's box, especially with peer feedback:) There is a lot of useful literature about it and models e.g. Martin & White's (2005) appraisal framework but I would be very interested if you could expand of the last 10% - feedback part - how to handle it best across the range of options, with your amazing practical tips!
Yeah, I agree - scaling feedback & assessment is a wicked problem! I’m doing some interesting work with machine learning to explore whether it’s possible to use NLP (basically, a “TA” trained by the SME) to provide meaningful qualitative feedback on text, images, audio and other outputs @ scale.
In the meantime: I’d recommend exploring peer to peer feedback - there’s research to show that with a solid rubric it’s as and in some cases more effective than tutor feedback.
Other strategies:
- provide a walkthrough of an exemplar and have students self assess
- combine a qualitative task with a closing knowledge check to enable application + a score of some sort
- consider the value of the score generated by MCQs etc. Would it be more valuable to provide summary verbal feedback recorded in advance covering common errors etc? The evidence suggests it would.
Let me know what you think!
Thanks Phil. Intrigued by the NLP idea for sure!