Assessing criticality in student research reports: Preliminary results from a new educational card sorting activity

Abbey, H, Esteves, J E, Vogel, S and Tyreman, S J (2014) Assessing criticality in student research reports: Preliminary results from a new educational card sorting activity. International Journal of Osteopathic Medicine, 17. pp. 12-21.

Full text not available from this repository.

Abstract

Background: Many osteopathic educational institutions (OEIs) require students to complete research projects, which normally involve external assurance of academic standards. Different academic factors can lead to tension between teaching approaches and attitudes to criticality as a necessary competency for effective practice in evidence-informed healthcare. Lack of clarity about different purposes of student research can lead to varying interpretations of assessment criteria and inconsistent marking. Objective: A new card sorting was designed to enable analysis of opinions about appropriate standards of criticality in student research reports. Methods: Data was obtained from a convenience sample (n=50) of participants attending four conference workshops. Participants read an abstract from a hypothetical student project and sorted cards containing project extracts into 'unacceptable', 'acceptable' or 'good' examples of criticality and recorded scores on marking grids. Results: Scores demonstrated poor inter-rater agreement (kappa<0.20), especially for cards expected to show 'acceptable' levels of criticality, although participants in one workshop achieved 'fair' levels of agreement (κ=0.22-0.39). Conclusions: The workshops promoted discussion about the challenges of encouraging students to question underlying osteopathic principles but there was poor inter-rater agreement about appropriate levels of criticality. Heterogeneous workshop groups and anonymised data meant that differences between OEIs and confounding factors such as linguistic variables and levels of experience could not be assessed. Further studies should explore different pedagogical approaches and assessment values to address inequalities in assessments, develop agreed standards for academic practice, enhance research education outcomes and support the long term development of a credible evidence base for osteopathic practice. © 2013 Elsevier Ltd.

Item Type:	Article
Keywords:	Agreement,Assessment,Criteria,Criticality,Education,Research,Standards,article,feasibility study,human,information processing,normal human,osteopathic medicine,research,student research
Depositing User:	Dr Hilary Abbey
Date Deposited:	21 Dec 2020 13:01
Last Modified:	16 Mar 2021 13:27
URI:	https://hsu.repository.guildhe.ac.uk/id/eprint/135

Actions (login required)

Edit Item