Three in five University students who engage in contract cheating will be caught by markers using machine learning software, new academic research has found.
The research conducted by Associate Professors Phillip Dawson and Wendy Sutherland-Smith from Deakin University, in collaboration with Principal Product Manager Mark Ricksen from academic integrity solutions provider Turnitin, is the first of its kind into the potential promise of machine learning to address the problem of contract cheating.
Contract cheating occurs when students outsource their assessed work to a third-party and submit it under the pretence of being their own work. Contract cheating is problematic and prevalent in Australia, with 6% of University students admitting to having obtained an assignment through a third-party.
The research presents the first quantitative empirical study into the use of either authorship analysis technology or machine learning to improve detection rates of contract cheating. Using an early alpha version of Turnitin’s new Authorship Investigate tool, which evaluates whether an assignment was written by the same student or not, 24 experienced markers were asked to spot contract cheating in bundles of 20 student assignments, which included 14 legitimate assignments and six purchased from cheating sites.
When markers were paired with a machine learning system and provided with authorship report for an assignment, their accuracy of detecting contract cheating increased significantly, from 48 per cent to 59 per cent, without a significant increase in false positives (incorrectly flagging contract cheating); in other words, markers were able to accurately identify three out of five cases of contract cheating.
The authorship report generated for markers provides an analysis of the student assignments’ linguistic attributes, such as sentence complexity, sentence length, and other stylometrics, as well as document information such as date created and last modified. The report does not specify whether contract cheating has occurred, but rather provides a recommendation for investigation based on statistical measures.
Phillip Dawson, Associate Director, Centre for Research in Assessment and Digital Learning, at Deakin University said the research demonstrated how machine learning can be an effective component of institutional strategies to address contract cheating.
“When markers were provided with a copy of the authorship report to review evidence and corroborate information, they were able to more accurately determine whether assignments had been authored by the student or if contract cheating had occurred. In addition to potentially improving detection rates, authorship analysis approaches using machine learning also offer benefits in terms of raising awareness about contract cheating and efficiently providing evidence if they wish to take their suspicions further.”
Following this alpha version, Turnitin is in development of a machine learning prediction model to offer a more robust recommendation system that prioritises papers across a corpus of documents or students in a cohort by level of suspicion of inconsistent behaviour. Turnitin believes this will replicate the “gut feeling” a marker experiences when suspicion of authorship of a student’s work occurs.
“Whilst Authorship Investigate was in early stages of development when this study was conducted, we’re pleased to see the value of the tool in the detection process, in bringing together all submissions made by a student and allowing rapid scanning of key points of evidence,” said Mark Ricksen, Principal Product Manager at Turnitin.
“Collaboration with higher education institutions and industry enables us to constantly test and iterate our tool so it can be used in a better, faster and more impactful way by markers to address contract cheating. There’s also potential for the software to speed up the investigation process by highlighting submissions of concern by a student and potentially determining the direction and focus of any investigation.”
Flinders University’s New Venture Institute (NVI) has been tapped as one of the leading university business accelerators in the latest UBI World Benchmark study. Read More
Australians are feeling insecure about how robots will effect their livelihoods, for good reason, any number of jobs from stock trading to manufacturing have been heavily impacted by automation. Read More
Hour of Code, the global movement introducing computer science to over 100 million children worldwide, is running from 9–15 December 2019 during Computer Science Education Week. Read More
We used to believe that you’re either creative or you’re good at maths, as it turns out to be really good at maths you have to be very creative too as intuitive non-cognitive thought processes are vital. Read More