Formative feedback within plagiarism education:

Is there a role for text-matching software?

International Mary Davis

Journal for Oxford Brookes University, UK
Educational [email protected]
Integrity

Jude Carroll
Oxford Brookes University, UK
[email protected]

Keywords: plagiarism education, formative feedback, Turnitin

Abstract

The need for effective education to enhance students’ understanding and

development of academic integrity has been well established, particularly for
international students new to tertiary study in English-speaking countries. Most
research recommends the use of methods such as instruction and warnings, as part
of a plagiarism education programme. Few studies have looked at the role of
formative feedback through tutorial intervention in the process of academic writing, in
which use of the text-matching tool Turnitin is made in ways which support learning,
rather than guide assessment. This study addresses that gap using data gathered
over three years from cohorts of international students in the United Kingdom (UK)
with regard to four identified areas of development: avoidance of plagiarism, decrease
in over-reliance on some sources, correct use of citation and appropriate
paraphrasing. Individual tutorials were held to give formative feedback on students’
own writing, with particular regard to their use of sources. A supportive environment
was created in which questions about references could be discussed, by using the
Turnitin originality reports directly. The tutorial feedback appeared to have a positive
effect on students’ understanding of academic integrity reflected in improved drafts.
This implies that tutorial feedback using Turnitin could be a key factor in plagiarism
education. Recommendations for future use are given at the end of the study.

Introduction

This paper explores ways of providing plagiarism education through individual

feedback on students’ own work during a formative stage of students’ writing practice.
It also examines the place of text-matching software in supporting and evaluating the
impact of tutor interventions designed to teach students academic citation skills. The
tool used in this study, marketed under the trade name of Turnitin, is referred to with a
variety of claims (“the standard in online plagiarism prevention”, iParadigms, 2007;
“anti-plagiarism software”, Sutherland-Smith, 2008), but in general, has the purpose
of highlighting students’ unoriginal work. In brief, the tool functions in the following
way: once a text is uploaded to the tool’s software, it provides what the makers call an
“originality report”. The report suggests an overall percentage of the student’s text
which matches sources on its database, and indicates the level of match with a
coloured icon (0=blue, 1-24=green, 25-49=yellow, 50-74=orange 75-100=red). The
software can also indicate the percentage of matching text for each named source
and presents these as a ranked list in order of quantity. With each student’s text, a

The International Journal for Educational Integrity is available online at:

http://www.ojs.unisa.edu.au/journals/index.php/IJEI/

58 © International Journal for Educational Integrity Vol. 5 No. 2 December, 2009 pp. 58–70 ISSN 1833-2595
 reviewer can click on a matched section and then see the suggested ‘original’ text
side-by-side for an instant comparison between the two texts.

The authors present a three-year case study in the UK where they investigate the
effect of feedback on students’ attention to important aspects of academic writing, in
particular to avoidance of plagiarism. The approach also examined three features of
writing that may be connected with plagiarism: over-reliance on some sources,
incorrect or incomplete citation, and insufficient paraphrasing. The underlying
approach supports the argument for shifting the focus away from a ‘catch-and-punish’
approach to student plagiarism (Carroll, 2005), and towards one designed to support
learning, through formative feedback to students on the use of academic conventions.

Debates continue in higher education as to an appropriate role for text-matching

software. Levin (2006) probably speaks for many when he describes those who use
Turnitin as the “plagiarism police” since in his view, it polarises student work into two
categories: original or potentially plagiarised. In contrast, advocates of using Turnitin,
such as the authors, believe it to be a useful support to other traditional teaching
methods when used formatively. It is likely that the dichotomy of views between those
in favour and against using Turnitin with students reflects much wider beliefs about
student learning, and the role of teachers. This study offers evidence of positive
impacts from Turnitin on students’ learning and a key role for teachers.

Literature review

The literature on teaching appropriate writing techniques and deterring student

plagiarism is extensive and continues to grow. A cursory search of the Internet will
reveal good practice guides for deterring plagiarism (for example, Carroll, 2007), well-
developed websites (for example, the Acknowledgements site based at Monash
University), and sites where writing and deterrence are closely combined (for
example, the statement by the US Council of Writing Programme Administrators). All
such sources offer guidance to teachers on how to help students avoid plagiarism.
Key elements include: students must draw from a range of sources, adopt acceptable
paraphrasing practices, and correctly attribute others’ texts when including them in
student work, using an agreed referencing system. Whilst these aspects of writing can
seem to be straightforward matters to academics experienced in academic writing, the
growing literature on academic writing demonstrates that students find them
problematic (such as Chanock’s 2008 study of why students may plagiarise). Pecorari
(2003) confirms “most students will use sources inappropriately before they learn how
to use them appropriately” (p. 342). Advice on addressing their difficulties emphasises
the importance of practice (Emerson, Rees & MacKay, 2005), detailed feedback
(Barrett & Malcolm, 2006) and using examples (Biggs, 1999). The importance of
addressing students’ difficulties with writing can not be over-estimated, as where
students fail to comply with the requirements, their work is judged to be plagiarised.

In many countries, including the UK, where the case study in this paper was
undertaken, policies on plagiarism have been revised to include statements regarding
institutional responsibilities to ensure students are able to comply with requirements to
‘do your own work’ before awarding punishments for plagiarism should they not do so
(Macdonald & Carroll, 2006). Therefore, the recognition that institutions must ensure
students have the required skills to avoid plagiarism has lessened the tendency to
assume students arrive with them already in place and/or acquire the necessary
academic skills informally. Explicit teaching aimed specifically at plagiarism avoidance
is increasingly important since students enter higher education from ever more
diverse backgrounds, particularly as international students, in English-language
contexts such as the UK. Many authors have discussed the difficulties students
experience when moving to a new academic culture in general terms (for example,
Carroll & Ryan, 2005) and an increasing number of studies focus on international
students’ needs when learning new writing conventions and trying to avoid plagiarism

59 ©©International
InternationalJournal
Journalfor
forEducational
EducationalIntegrity
IntegrityVol.
Vol.52No.
No.22December,
December 2009
2006 pp.
pp. 58–70
xx-xx ISSN 1833-2595
 (for example, Handa & Power, 2005). International students are frequently described
as lacking confidence in writing and many researchers link this to greater reliance on
textual borrowing (Ryan, 2000; Pecorari, 2003). As one lecturer in Sutherland-Smith’s
2008 study exclaims: “students who are coming to us from overseas just don’t have
the grounding before they start…Thirteen weeks is not enough time to really
understand plagiarism and be able to overcome it. Some students have a life-time of
just copying textbooks to get over!” (p. 12). All students, but arguably, some
international students in particular, may find it especially difficult to engage with the
learning outcomes and may need particular help to learn to do so in acceptable ways,
rather than unacceptable ones such as plagiarism.

In understanding how to avoid plagiarism, the concept of authorship is also discussed

in the literature. A recent study involving five universities in the UK (Pittam, Elander,
Lusher, Fox & Payne, 2009) analysed student views which confirmed students’
problems with understanding the concept of ‘authorial identity’. The students in this
study tended to feel they were not authors since they saw authorship as something
done by professional writers showing original thinking, whereas assignments were
viewed as tasks which required them to keep to established literature. Thus they saw
their own writing as more akin to ‘editing’ than authorship. Pittam et al. (2009)
conclude that problems with establishing authorial identity may be key to
understanding why some students plagiarise. Similarly, Sutherland-Smith (2008)
found the concept of authorship to be blurred for many students going through a
written process involving starting and stopping and absorbing many kinds of other
influences before a finished ‘product’ eventually emerged.

Studies such as those cited show that within plagiarism education, students are
learning new skills and adapting previous beliefs. One way to encourage students’
learning is by offering clear, helpful formative feedback, in line with the reminder to UK
academics of the Quality Assurance Agencies’ principles (General Principle 12) that
“Tutors in higher education today are encouraged to ensure that appropriate feedback
is provided to students on assessed work in a way that promotes learning and
facilitates improvement” (cited in Orsmond, Merry & Reiling, 2002). Tutors may
recognise the importance of ‘appropriate feedback’, yet in relation to the serious issue
of plagiarism, Hyland (2001) found that written feedback sometimes is seen by
students as vague and ambiguous, and that students may be confused as to what
they have to do to improve their drafts. Meanwhile, spoken feedback given on a one-
to-one basis may be one of the most useful means to discuss issues such as
plagiarism, since by its very communicative nature, involving both student and tutor, it
can have a clear formative purpose (Gardner, 2004). This was confirmed by
McGarrell and Verbeem (2007) who also stressed the importance of formative
feedback in the process of acquiring academic literacy to enable more competent
drafting. Orsmond et al. (2002) also advocated feedback clinics as a way to discuss
the wider aspects of tutors’ comments, thus bringing greater benefit to students.

As well as dealing with plagiarism by increasing students’ skills and knowledge, there
has also been widespread discussion in the relevant literature of the use of tools to
detect text copying which could be judged to be plagiarism. Many such tools have
been created and comparisons made as to their utility (see, for example, Bull,
Coughlin, Collins & Sharp, 2001) as well as specific investigations of particular tools
such as the one used in this study, Turnitin, which analyse the systems’ benefits and
flaws (for example Pearson, 2002; Goddard & Rudzki, 2005). Makers’ claims such as
the statement which describes Turnitin’s ‘originality report’ as “allow[ing] for easy and
unambiguous interpretation of document source analysis results” (iThenticate, 2007)
are challenged by studies such as that by Peacock, Sharp and Anderson (2006),
which found that many academics have not found the interpretation so easy and
stress the need for clear guidelines and training to ensure tutors use the reports
correctly. This level of ongoing interest confirms the need to note the limitations of any

60 ©©International
InternationalJournal
Journalfor
forEducational
EducationalIntegrity
IntegrityVol.
Vol.52No.
No.22December,
December 2009
2006 pp.
pp. 58–70
xx-xx ISSN 1833-2595
 text-matching tool, rather than assuming its abilities to identify and deal with
plagiarism.

The impact of Turnitin on students’ work and tutors’ practice has also been explored.
Turnitin’s capacity to deter students from plagiarism has been asserted by lecturers
(Bennett, 2005; Sutherland-Smith & Carr, 2005), by those promoting use of the tool
(Murray, 2006) and by students themselves (Davis, 2007). For example, Murray
(2006) claimed that 76% of students responding to a survey “felt that Turnitin could
discourage plagiarism”. Contrasting views on its use as a deterrent are often shared
via informal networks (for example, the UK government – funded Plagiarism Advisory
Service mailbase discussions). Concerns include the danger that introducing students
to Turnitin may simply teach them better strategies for avoiding detection, or what one
mailbase contributor called “making the colour go away” (removing the colour-coded
matches). There is a concern among academics that it could take away responsibility
from the tutors to support student learning (e.g. Sutherland-Smith & Carr, 2005), but
many institutes (such as the University of Maryland University College) actively
promote Turnitin as an approach to improve students’ awareness of plagiarism.
However, there are few studies which investigate how best to use Turnitin to
encourage learning, and furthermore, even fewer which investigate the central role of
formative feedback in this process. This study addresses this apparent gap.

Case study: Oxford Brookes University, UK

The case study was made at Oxford Brookes University, UK, of international students
on a year-long pre-Master’s diploma for language and study skills, known as a
programme of English for Academic Purposes (EAP). A total of 66 students
participated in the research over three years (2007:19, 2008:23, 2009:24). The
investigation followed the intervention of a pre-assessment tutorial feedback session
using Turnitin on a second semester academic writing module, entitled the Extended
Writing Project, towards the end of the programme. The module required students to
produce a 3,000 word assignment, in preparation for their Master’s level dissertation,
on a topic related to their future study. Students were preparing for postgraduate
study in a range of disciplines but a core majority were planning to take Business. The
students were approximately 60% East Asian (Chinese and Japanese), while the
remaining 40% were from the Middle East, North Africa and Eastern Europe; all had a
non-UK accredited undergraduate degree. Students were admitted via an English
language competency exam (the International English Language Testing System)
with a band of 5.5 or above.

Plagiarism education formed a central theme of the module. At least six weeks of
instruction was devoted to information-giving, examples and awareness-raising
activities, supplemented by online discussion facilities and institutional warnings in the
student handbook. To be consistent with the approach across the university, the
Harvard system was taught, practised and discussed with particular emphasis on
issues related to accurate use of sources. Following one trial of Turnitin in 2006, its
use was made an integral part of the educational process during the module each
year. Students submitted an electronic version of their first drafts to their tutor, who
uploaded them to Turnitin and studied the resulting originality reports, before using
the screen-based information as part of 15 minute individual tutorial feedback
interviews. Tutors asked questions during the tutorial to promote student thinking
about source use.

Data on the impact of the above teaching strategy was gathered by analysing issues
that arose from perceived changes in student work from first draft to final draft in the
tables below using similar measures of severe, moderate, minor and none, for
consistency. The Turnitin originality reports (excluding the matches to bibliographies)
were analysed for changes between the drafts. At the end of the module, all students
in the research also completed a questionnaire to evaluate the usefulness of Turnitin

61 ©©International
InternationalJournal
Journalfor
forEducational
EducationalIntegrity
IntegrityVol.
Vol.52No.
No.22December,
December 2009
2006 pp.
pp. 58–70
xx-xx ISSN 1833-2595
 for different areas of learning (for which the mean percentages have been calculated
below), and a number of students (16) contributed comments in a focus group in
2007. Four key areas emerged from the data:
 reduction in the amount of plagiarism
 reduction in over-reliance on one source
 reduction in citation errors
 reduction in insufficient paraphrasing.

Avoiding plagiarism
For the purpose of this study, levels of plagiarism were categorised as severe, (10%
or more of unattributed text in the draft), moderate (5-9.9%), minor (0.1-4.9%) and no
plagiarism (complete absence of unattributed text). To produce a representative
result, particularly with the varying lengths of first drafts compared with second drafts,
the number of unattributed words from a matched source was calculated as a
percentage of the word count (excluding bibliography).

Table 1.
Percentages of plagiarism in first and final drafts 2007–2009

First draft Final draft

Year 2007 2008 2009 Total & 2007 2008 2009 Total &
(n=19) (n=23) (n=24) mean (n=19) (n=23) (n=24) mean
(n=66) (n=66)

Severe 3 0 0 3 0 1 0 1
(10% +) (16%) (4.5%) (4%) (1.5%)

Moder- 1 3 3 7 1 0 1 2
ate (5%) (13%) (13%) (11%) (5%) (4%) (3%)
(5-9.9%)
Minor 6 8 8 22 12 10 11 33
(0.1- (32%) (35%) (33%) (33%) (63%) (44%) (46%) (50%)
4.99)
None 9 12 13 34 6 12 12 30
(0) (47%) (52%) (54%) (51.5%) (32%) (52%) (50%) (45.5%)

Total final drafts with reduced plagiarism 12 8 10 30

(63%) (35%) (42%) (45.5%)

The table indicates a reduction in levels of severe and moderate plagiarism to the
majority ending in the minor or no plagiarism categories. The results were affected by
one student in 2008 submitting a shorter first draft with little source use, and a final
draft with long sections of copied and pasted unattributed text; as an example of
severe plagiarism at the final draft stage, this is clearly a case where the tutorial
feedback was unable to help. Nevertheless, most identified plagiarism was at a very
minor level of 1-2% of text, and a mean of almost half (45.5%) reduced plagiarism
between drafts over the period.

Overall, from the questionnaire results, a mean of 73% stated that Turnitin was useful
for understanding how to avoid plagiarism and talked about the important learning
opportunity they had experienced:

The software shows your mistakes. Then we can correct them. At the end,
when we submit the final draft we don’t want to fail (Business student).

62 ©©International
InternationalJournal
Journalfor
forEducational
EducationalIntegrity
IntegrityVol.
Vol.52No.
No.22December,
December 2009
2006 pp.
pp. 58–70
xx-xx ISSN 1833-2595
 I would like to use it again because sometimes we do plagiarism without
noticing it (Hospitality student).

The software is to find plagiarism, so it does not teach how to avoid it. The
teacher teaches us that (Technology student).

As perceived by the students, Turnitin offers support to help them avoid ‘mistakes’ or
unintentional plagiarism, but importantly, they also recognise its limitations and the
importance of the tutor’s role.

Over-reliance on sources
From the list of matched sources Turnitin supplies, it was possible to examine how
much students were relying on one source or a number of sources. Similar
descriptors are used for levels of the highest rank source use (severe, moderate,
minor, none), though the percentages are scaled lower to suit this category.

Table 2.
Percentages of use of highest ranked source in first and final drafts 2007–2009

First draft Final draft

Year 2007 2008 2009 Total & 2007 2008 2009 Total &
(n=19) (n=23) (n=24) mean (n=19) (n=23) (n=24) mean
(n=66) (n=66)
Severe 7 3 2 12 2 1 0 3
(5% +) (37%) (13%) (8.5%) (18%) (10.5%) (4%) (4.5%)
Moder- 7 11 13 31 13 9 6 30
ate (37%) (48%) (54%) (47%) (68.5%) (39%) (25%) (45.5%)
(2-4.9%)
Minor 4 8 3 15 4 13 18 33
(0.1- (21%) (35%) (12.5%) (23%) (21%) (57%) (75%) (50%)
1.9%)
None 1 1 6 8 0 0 0 0
(0)* (5%) (4%) (25%) (12%)
Total final drafts with reduced use of top source 10 7 13 30
(53%) (30%) (54%) (45.5%)

During the three year period, approximately half (mean: 45.5%) reduced the
percentage of the top ranked source in their final drafts. In first drafts, the highest
percentage from one source was 31% in 2007, 6% in 2008 and 11% in 2009, while
the highest result for second drafts was over 5% in only three cases over the period,
two of which were presentations of government data. In fact, the majority (70% over
the period) of top-ranked sources in final drafts were 1-2%, which is generally
regarded as an insignificant amount of matching (Davis, 2007). Those who increased
the top source did so by a small percentage (66% of those who increased the top
source only did so by 1%), and this especially occurred following the submission of a
much shorter first draft (in 60% of cases of an increase in the percentage of a top
source, the first draft was under 1,000 words). In addition, questionnaires revealed
that a high percentage of 82% believed Turnitin was useful for avoiding over-reliance
on sources; they seemed to find the visual representation clear and memorable, as
shown by the comments:

Turnitin is useful for avoiding over-reliance…because there is a sort of

pressure, from the software to make aware of use of sources (Business
student).

63 ©©International
InternationalJournal
Journalfor
forEducational
EducationalIntegrity
IntegrityVol.
Vol.52No.
No.22December,
December 2009
2006 pp.
pp. 58–70
xx-xx ISSN 1833-2595
 It helps my work lead to the right track. I would like to use it for to find out over-
relianced sources because I tend to cite many points from one source
(Business student).

Correct and complete citation

Attempts at using citation within the text (in particular, the use of author’s surname,
year and where appropriate, page number) were recorded as problems when
students’ efforts were incorrect or incomplete (note: for this study, errors in the
bibliography were not analysed). Differences between the instances of incorrect or
incomplete citation in the first and final drafts were evident, especially as Turnitin
highlights different references in colour. Numbers of errors were separated into four
levels with similar descriptors: severe as ten or more errors, moderate at five to nine
errors, minor as one to four errors and no errors.

Table 3.
Percentages of levels of citation errors in first and final drafts 2007–2009

First draft Final draft

Year 2007 2008 2009 Total & 2007 2008 2009 Total
(n=19) (n=23) (n=24) mean (n=19) (n=23) (n=24) mean
(n=66) (n=66)

Severe 10+ 0 2 0 2 0 0 0 0
errors (9%) (3%)

Moderate 4 2 1 7 1 1 0 2
5-9 errors (21%) (9%) (4%) (11%) (5%) (4.5%) (3%)

Minor 9 15 16 40 8 10 9 27
1-4 errors (47%) (65%) (67%) (60%) (42%) (43.5%) (37.5%) (41%)

None 6 4 7 17 10 12 15 37
(32%) (17%) (29%) (26%) (53%) (52%) (62.5%) (56%)

Total final drafts with reduced citation errors 10 15 16 41

(53%) (65%) (67%) (62%)

The table indicates a reduction of higher levels of citation errors, with none at a
severe level in final drafts, and a great reduction of those at moderate level, thus a
much greater number at a lower level, with more than half of final drafts showing no
citation errors. The mean reduction in the number of errors was 62%, showing
evidence of greater understanding of how to use citation accurately in their final
drafts. This shows they have learned and used their learning, not just ‘made the
colour go away’ from the original instances. Furthermore, the type of error changed in
ways that indicate students began attending to significant aspects of the referencing
process. Common errors in first drafts were use of first names rather than surnames,
use of authors’ surnames with an initial for first names (copying a bibliographical
style), absence of page numbers for direct quotations, problems with punctuation and
spacing. In contrast, the small number of errors that persisted in final drafts tended to
be relatively minor inaccuracies of punctuation and spacing. Errors of format, such as
using a bibliographical style reference for in-text citation, were eliminated.

Questionnaire responses revealed a mean of 69% of students found Turnitin reports

useful for learning to write citations accurately. Students themselves described their
errors as arising from both the process of writing and from incomplete knowledge. It

64 ©©International
InternationalJournal
Journalfor
forEducational
EducationalIntegrity
IntegrityVol.
Vol.52No.
No.22December,
December 2009
2006 pp.
pp. 58–70
xx-xx ISSN 1833-2595
 was very notable that there were no cases in any year of the number of citation errors
increasing, even where first drafts were very short (under 1,000 words) compared to
the required 3,000 word final draft; this also indicates that students seem to have
more knowledge of accurate citation use by the final draft:

The software helped me to be more cautious about citation (Business student).

Sometimes we make a mistake and don’t use citation or not correctly… this
way gives us a second chance (Business student).

For me, I will never ever forget about efficient citations according to my first
experience of Turnitin (Business student).

Therefore, students seem to recognise that the tutorial using Turnitin reports could be
an important and memorable learning opportunity to take more care with citation.

Paraphrasing
Unlike generic search engines such as Google, Turnitin is able to identify continued
textual borrowing that includes gaps. This makes it possible to study attempts at
paraphrasing in detail and to consider whether the alterations made by the student
were appropriate, for example where they are adopting a synonym substitution
strategy (e.g. changing one in every four words) for large parts of a text. This study
paid particular attention to instances where text highlighted by Turnitin showed an
extract of ten words or more, with one to four words in grey text, which signals places
where students have altered the original. These instances have been calculated as
insufficient paraphrasing, using the same levels as for plagiarism above, for ease of
understanding.

Table 4.
Percentages of insufficient paraphrasing in first and final drafts 2007-2009

First draft Final draft

Year 2007 2008 2009 Total & 2007 2008 2009 Total &
(n=19) (n=23) (n=24) mean (n=19) (n=23) (n=24) mean
(n=66) (n=66)

Severe 0 0 0 0 0 1 0 1
(10% +) (4%) (1.5%)
Moderate 2 1 1 4 0 0 0 0
(5-9.9%) (11%) (4%) (4%) (6%)

Minor 4 18 9 31 9 18 9 36
(0.1-4.99) (21%) (78%) (37.5%) (47%) (47%) (78%) (37.5%) (54.5%)

None 13 4 14 31 10 4 15 29
(0) (68%) (18%) (58.5%) (47%) (53%) (18%) (62.5%) (44%)

Total final drafts with a reduced level of insuffi- 6 9 10 25

cient paraphrasing (31.5%) (39%) (42%) (38%)

It is striking that the severe and moderate levels of insufficient paraphrasing largely
disappeared by the final draft (despite the one example in 2008, the same outlying
example as in the plagiarism table above). Most drafts with examples of insufficient
paraphrasing were at a minor level, with less than 5%. The overall number of final
drafts with a reduced level of insufficient paraphrasing was not as high as the result
above for plagiarism, and in some cases the percentage rose. One explanation could

65 ©©International
InternationalJournal
Journalfor
forEducational
EducationalIntegrity
IntegrityVol.
Vol.52No.
No.22December,
December 2009
2006 pp.
pp. 58–70
xx-xx ISSN 1833-2595
 be the number of shorter drafts (59% producing first drafts of less than 2,000 words)
because longer final drafts meant more attempts at paraphrasing. It is also recognised
that paraphrasing is a complex skill needing practice; if few attempts are made at the
first draft stage, students could not benefit from tutor feedback.

Nevertheless, a high proportion of students overall (76%) stated that Turnitin was useful
for thinking about appropriate paraphrasing. Students made comments such as:

International students are not good at paraphrasing. So Turnitin is useful for

students thinking about it (Business student).

It highlights some part of bad paraphrasing, then we can correct it (Social

Sciences student).

Turnitin was useful for thinking about paraphrasing… well that is what the
software is all about (Business student).

It seems that students were therefore connecting the software with learning about
paraphrasing and found Turnitin helpful for finding ‘bad’ paraphrasing. However, unlike
the more ‘technical’ issues with applying citation conventions, the more complex writing
skills needed for paraphrasing clearly take practice and repeated feedback, presumably
beyond the limits of this case study.

Discussion

Unlike a similar study by Barrett and Malcolm (2006) in which heavily plagiarised first
drafts could only achieve a bare pass in final results, in this study no penalties were
imposed at the formative stage, even if there was evidence of serious plagiarism. This
was to encourage open discussion and to reduce the possible threat students might feel
about assessment at the drafting stage. The feedback session included questions about
the writing process, such as how it would be possible to use other author’s words in a
different way and whether a certain percentage of use of others’ words represented
appropriate or too extensive support to arguments. The students could use the
feedback to amend their use of sources, re-draft their text and submit a final version for
assessment four weeks later. The intervention through guided tutorial feedback using
Turnitin reports seems to have encouraged an improvement in the areas under review.

Avoiding plagiarism
Much of the learning about academic writing is evidently focused on acquiring good
practices and avoiding unacceptable ones such as plagiarism. However, research has
shown that students often do not internalise instruction about plagiarism until it refers to
their own work (Barrett & Malcolm, 2006), thus the discussion of sources in the
feedback is an opportunity to achieve this, in a clear and direct way (Hyland, 2001).
Many students seemed to have a kind of ‘eureka’ moment, when faced with the on-
screen evidence of how they had used sources, where they understood more fully
about issues related to academic integrity, as they connected to their own work. These
realisations were clearly useful learning opportunities, which bears out the high number
of students believing Turnitin helped them understand plagiarism found by Murray
(2006).

Over-reliance on sources
The visual display seems to have been a factor in students’ awareness of how much
they were using sources, and whether they were, at times, relying too heavily on some
sources. This seemed a useful and unexpected finding of the study, especially as there
are few studies of the appropriate amount of source use in the literature. Students who
need help to use sources well are often not specifically advised to use a wide range of

66 ©©International
InternationalJournal
Journalfor
forEducational
EducationalIntegrity
IntegrityVol.
Vol.52No.
No.22December,
December 2009
2006 pp.
pp. 58–70
xx-xx ISSN 1833-2595
 texts and sources. This relates to Pecorari’s (2003) study, where some confusion was
reported on the amount of source use, and student respondents who were concerned
that too many quotations may be seen as “something like plagiarism” (p. 337).
Howard (2000) has made a very important distinction here, by identifying the practice
of putting together sections of text from other authors to form a new text without a
clear new voice as “patch writing”. This may be seen as an expected and important
stage of learning (Pecorari, 2008). The highlighting by Turnitin of these patches
helped to show students where there was greater need for original voice in academic
writing, sometimes an unfamiliar skill for students from different academic cultures
(Pennycook, 1996; Hayes & Introna, 2005). This also confirms the findings of Pittam
et al. (2009) regarding the difficulty for students to establish authorial identity.

Correct and complete citation

Similarly, international students are frequently new to the convention of in-text citation.
It can be difficult to know when, how and how much to cite, as Shi’s (2008) study of
undergraduate writers shows. As Turnitin highlights matched citations and quotations,
it makes it easy to see errors or lack of appropriate citation. Students seem to have
noted its usefulness in understanding good practice with citation. Furthermore, as
Chanock (2008) found, students may copy sections of text and put a citation but not
acknowledge them as a quotation; Turnitin reports can help to show evidence of this,
and formative feedback may encourage students to rectify an area which they may
have misjudged to be acceptable, thus avoiding any serious consequences should
this be linked to academic malpractice.

Paraphrasing
In contrast to the other areas, there seemed to be less reduction in the cases of
inappropriate paraphrasing in final drafts. The result is perhaps surprising, given the
students’ perception that Turnitin was helping them effectively with paraphrasing, as
shown in their comments above. The increase in errors may be a consequence of the
methodology used for collecting data, linked to the length of students’ draft
submissions. Shorter drafts, particularly those of less than 1,000 words, permitted few
attempts at paraphrasing to be analysed and given feedback on. Subsequent 3,000-
word final drafts included more attempts and therefore, more opportunities for error.
However, the finding does indicate a need for more instruction and practice of
paraphrasing in class, as found by Davis (2007). Furthermore, since the skill of
paraphrasing is a complex one, which takes repeated practice and extensive reading
(as opposed to the more technical and straightforward issues of applying citation
conventions to punctuation), it seems unlikely that Turnitin could do more than
stimulating discussion about paraphrasing.

Conclusions

This study indicates that formative feedback on a one-to-one basis using Turnitin
originality reports seemed to have a positive impact on students’ final submissions.
Explanations for this effect include offering time, encouraging students to feel less
distanced from their work and/or more engaged, all of which have been shown to be
relevant. As suggested by Hyland (2001), giving direct advice on plagiarism in a non-
threatening, formative way in spoken feedback seems an effective approach.

The results show that the tutorial feedback had some specific effects on drafts
including a noticeable reduction in plagiarism, mainly to a very minor level each year.
Furthermore, the learning opportunities could be seen in other areas related to
plagiarism education. After the intervention, there were reductions in the numbers of
citation errors, a reduced amount of over-reliance on sources and some, though
limited, effects on insufficient paraphrasing.

67 ©©International
InternationalJournal
Journalfor
forEducational
EducationalIntegrity
IntegrityVol.
Vol.52No.
No.22December,
December 2009
2006 pp.
pp. 58–70
xx-xx ISSN 1833-2595
 Thus the potential for using Turnitin as an educational tool can be seen, in common
with the study by Peacock et al. (2006), who found that “when used in non-policing
mode, Turnitin acted as a form of goal-oriented learning for good referencing
practice”. It is this learning opportunity for good practice that Turnitin may support
which is the most important result. It is acknowledged that numbers are small for each
year (19-24), but as the same data was collected over three years involving a total of
66 students, the results may be considered more convincing. Overall, it does seem
that the tutorial feedback may have helped students to improve their academic
integrity, although we can only claim that these results are one of several means to
determine whether levels of plagiarism have decreased.

Recommendations

Following the study, a major recommendation is to include an individual tutorial

feedback session at the formative stage using Turnitin for important learning
opportunities about avoiding plagiarism. Where this is impossible (or is seen to be
impractical) due to large class size, small group tutorials, or a large group session
providing examples of reports and typical errors may be worthwhile ways to offer
some of the benefits found in this study. A student’s overall programme can be
created to offer this one-to-one opportunity at a strategic point, or points in their
learning.

A limitation of this study was that the students’ continuing performance has not been
monitored in their further studies. A follow-up investigation is necessary to assess
how much students are able to transfer this knowledge to a new learning context.
Further research could also be carried out by collecting tutors’ views on the
usefulness of tutorial feedback using Turnitin, which would help to evaluate the
benefits and issues still to be addressed.

About the authors

Mary Davis is a Senior Lecturer at Oxford Brookes International, Oxford Brookes
University where she manages a pre-Master’s course for international students in
English language and study skills. Her research interests are tutor responses to
plagiarism and the use of new technology in learning and teaching, especially related
to academic literacy. She is currently undertaking a PhD on the development of
source use in academic writing,

Jude Carroll is a Principal Lecturer at the Oxford Centre for Staff and Learning
Development (OCSLD) at Oxford Brookes University where she works on a range of
issues such as deterring plagiarism, work-based learning, and qualification training for
newly appointed HE teachers. She also serves as co-director with Dr Janette Ryan in
TALIS, a national centre focussed on teaching of international students and holds a
visiting teaching position at the Royal Technical University in Stockholm, Sweden.

References

Barrett, R., & Malcolm, J. (2006). Embedding plagiarism education in the assessment
process. International Journal for Educational Integrity, 2(2), 38–45.
Bennett, R. (2005). Factors associated with student plagiarism in a post-1992
university. Assessment and Evaluation in Higher Education, 30(2), 137–162.
Biggs, J. (1999). Teaching for quality learning at university. Buckingham, Society for
Research in Higher Education/Open University Press.
Bull, J., Coughlin, E., Collins, C., & Sharp, D. (2001).Technical review of plagiarism
detection software report. Joint Information Systems Committee, 1–36.
Retrieved January 9, 2009, from http://www.jisc.ac.uk/uploaded_documents/
luton.pdf

68 ©©International
InternationalJournal
Journalfor
forEducational
EducationalIntegrity
IntegrityVol.
Vol.52No.
No.22December,
December 2009
2006 pp.
pp. 58–70
xx-xx ISSN 1833-2595
 Carroll, J. (2007). A handbook for deterring plagiarism in higher education (2nd ed.).
Oxford, Oxford Centre for Staff and Learning Development: Oxford Brookes
University.
Carroll, J. (2005). Deterring, detecting and dealing with plagiarism, a brief paper for
Brookes staff for Academic Integrity week. Retrieved January 4, 2007, from
http://www.brookes.ac.uk/services/ocsd/5_research/jude.html
Carroll, J., & Ryan, J. (2005). Teaching international students: Improving learning for
all, London: Routledge.
Chanock, K. (2008). When students reference plagiarised materials – what can we
learn (and what can we do) about their understanding of attribution?
International Journal of Educational Integrity, 4(1), 3–16.
Davis, M. (2007). The role of Turnitin in the formative process of academic writing: A
tool for learning and unlearning? Brookes e-Journal of Learning and Teaching,
2(1). Available at: http://bejlt.brookes.ac.uk/article/
the_role_of_turnitin_within_the_formative_process_of_academic_writing/
Emerson, L., Rees, M., & MacKay, B. (2005). Scaffolding academic integrity:
Creating a learning context for teaching referencing skills. Journal of University
Learning and Teaching Practice, 2(3a), 12–24.
Gardner, S. (2004). Knock-on effects of mode change on academic discourse.
Journal of English for Academic Purposes, 3(1), 23–38.
Goddard, R., & Rudzki, R. (2005). Using an electronic text-matching tool (Turnitin) to
detect plagiarism in a New Zealand University. Journal of University Teaching
and Learning Practice, 2(3b), 58–63.
Handa, N., & Power, C. (2005). Land and discover! A case study investigating the
cultural context of plagiarism. Journal of University Teaching and Learning
Practice, 2(3b), 64–84.
Hayes, N., & Introna, L. (2005). Plagiarism, alienation and fairness: Towards an
inclusive educational practice. Ethics and Behaviour, 15(3), 213–231.
Howard, R. M. (2000). Sexuality, textuality: The cultural work of plagiarism. College
English, 62(4), 473–491.
Hyland, F. (2001). Dealing with plagiarism when giving feedback. ELT Journal, 55(4)
October, 375–381.
iParadigms (2007). iParadigms: Digital solutions for a new era in information.
Retrieved October 15, 2007, from http://www.iparadigms.com
iThenticate (2007). Retrieved January 4, 2009, from http://www.ithenticate.com/static/
features.html
JISC Plagiarism Advisory Service (2007). Mailbase discussions. Retrieved January 4,
2009, from http://www.jiscpas.ac.uk/
Levin, P. (2006). Why the writing is on the wall for the plagiarism police. Retrieved
January 20, 2009, from http://www.student-friendly-guides.com/plagiarism/
index.htm
Macdonald, R., & Carroll, J. (2006) Plagiarism: A complex issue requiring a holistic
approach. Assessment and Evaluation in Higher Education, 31(2), 233–245.
McGarrell, H., & Verbeem, J. (2007). Motivating revision of drafts through formative
feedback. ELT Journal. 61(3), 228–236.
Monash University. Accessed January 24, 2009, from http://calt.monash.edu.au/staff-
teaching/plagiarism/acknowledgement/module8/index.html
Murray, W. (2006). JISC Plagiarism Advisory Service Frequently Asked Questions.
Available at http://www.jiscpas.ac.uk/documents/FAQs.pdf
Orsmond, P., Merry, S., & Reiling, K. (2002). The student use of tutor formative
feedback in their learning. Paper presented at the Learning Communities and
Assessment Cultures Conference organised by the EARLI Special Interest
Group on Assessment and Evaluation, University of Northumbria, 28-30 August
2002. Retrieved January 11, 2009, from http://www.leeds.ac.uk/educol/
documents/00002233.htm

69 ©©International
InternationalJournal
Journalfor
forEducational
EducationalIntegrity
IntegrityVol.
Vol.52No.
No.22December,
December 2009
2006 pp.
pp. 58–70
xx-xx ISSN 1833-2595
 Peacock, S., Sharp, J., & Anderson, S. (2006). Pickup a plagiarism practice? A
holistic approach to deploying TurnitinUK as a sustainable driver in altering
institutional plagiarism practice. Centre for Academic Practice, Queen Margaret
University College, Edinburgh. Paper presented at JISC International
Plagiarism Conference, Newcastle, 19-21 June 2006.
Pearson, G. (2002). Electronic Plagiarism Seminar, Le Moyne College, Syracuse, NY.
Retrieved February 10, 2009, from http://web.lemoyne.edu/~pearson/
plagiarism/
Pecorari, D. (2003). Good and original: Plagiarism and patch writing in academic
second-language writing. Journal of Second Language Writing, 12, 317–345.
Pecorari, D. (2008). Academic writing and plagiarism: A linguistic analysis. London:
Continuum.
Pennycook, A. (1996). Borrowing other’s words: Text, ownership, memory and
plagiarism. TESOL Quarterly, 30(2), 201–230.
Pittam, G., Elander, J., Lusher, J., Fox, P., & Payne, N. (2009). Student beliefs and
attitudes about authorial identity in academic writing. Studies in Higher
Education, 34(2), 153–170.
Ryan, J. (2000). A guide to teaching international students. Oxford: Oxford Centre for
Staff and learning development, Oxford Brookes University.
Shi, L. (2008). Textual appropriation and citing behaviours of university
undergraduates. Applied Linguistics. December amn045, 1–24.
Sutherland-Smith, W., & Carr, R. (2005). Turnitin.com: Teacher’s perspectives of anti-
plagiarism software in raising issues of educational integrity. Journal of
University Teaching and Learning Practice, 2(3b), 94–101.
Sutherland-Smith, W. (2008). Plagiarism, the internet and student learning: Improving
academic integrity. New York: Routledge
University of Maryland University College (2007). Virtual Academic Integrity
Laboratory Tutor. Academic Integrity. Retrieved January 4, 2007, from http://
www-apps.umuc.edu/vailtutor/index.html
US Council of Writing Programmes. Administrators’ statement. Retrieved February 2,
2009, from http://www.wpacouncil.org/node/9

70 ©©International
InternationalJournal
Journalfor
forEducational
EducationalIntegrity
IntegrityVol.
Vol.52No.
No.22December,
December 2009
2006 pp.
pp. 58–70
xx-xx ISSN 1833-2595