Get Your Head into

the Clouds:
Using Word Clouds for
Analyzing Qualitative
Assessment Data
By Concetta A. DePaolo and Kelly Wilkinson, Indiana State University

Abstract concrete information to provide a roadmap

for course improvement. Is there a method for
Word clouds (or tag clouds) are popular, educators to summarize large amounts of data
fun ways to display text data in graphical form; in a meaningful and efficient way? We propose
however, we contend that they can also be using word clouds.
useful tools in assessment. Using word clouds, Word clouds are more than just an attractive
instructors can quickly and easily produce visual representation of words; they can have
graphical depictions of text representing student important applications to a variety of different
knowledge. By investigating the patterns of types of assessments. A word cloud (or a tag
words or phrases, or lack thereof, in textual cloud) is a . . . “visual presentations of a set of
student responses, instructors can evaluate if words, typically a set of tags, in which attributes
students, as a whole, have grasped or missed of the text such as size, weight or color can be
key concepts or have made common mistakes. used to represent features (e.g., frequency) of
Word clouds can be used to provide formative the associated terms” (Havley and Keane, 2007).
assessment feedback to students, and to identify Typically, a word cloud takes the most frequently
common themes in student reflection papers. used words and displays them in an appealing
They can also be useful as a starting point or visual representation that identifies key words in
screening tool for large amounts of text data, different sizes and colors based on the frequencies.
whether related to assessment or not. By examining frequencies in a word cloud, an
Keywords: Analyzing Text, Assessment, instructor can look for specific patterns of words
Qualitative Data, Tag Clouds, Word Clouds and phrases, or the lack thereof, in text data.
The effectiveness of the word cloud is

T he saying, “A picture is worth a thousand

words,” continues to hold true even in the
assessment world. The ability to organize
theoretically grounded in the learning model
of graphical organizers. “A graphic organizer
is a visual communication tool that uses
assessment data, or any data for that matter, in visual symbols to express ideas and concepts,
a graphical representation makes the data easy- to convey meaning” (Instructional Strategies
to-read and comprehend. Course management Online, n.d.). A graphical organizer can provide
systems, online surveys and other technology an assessment picture of individual concepts
tools give instructors the ability to gather large along with a “big picture” assessment of the
amounts of data from students. However, it may interrelationships of the individual concepts.
be difficult for the instructor to quickly leverage It may also allow the viewer to discover his or
these data to assess students’ learning and give her own patterns of relationships have meaning

38 TechTrends • May/June 2014 Volume 58, Number 3

for the viewer. Graphic organizers portray to show differences;
knowledge in a meaningful way, which helps • Course evaluations or midterm feedback
bring clarity to ideas as connections are made. from students to teachers;
Fountas and Pinnell (2001) assert that when • Graphical representations of student
content is illustrated with pictures or figures, the knowledge or learning;
information can be maintained in the mind of • Assessing experiential learning by analyzing
the viewer over a period of time. student reflection papers;
According to Joyce, Weil and Calhoun (2000), • Programmatic assessment using longitudinal
Ausubel stated in his work that using organizers data; and
is way of presenting information in a more active • Formative assessment feedback to students.
format that allows the viewer to draw his/her own
conclusions. Although he was an educational Assessing the Use of Key Words
theorist, Ausubel saw the importance of the in Student Responses
presentation of data to draw conclusions and
construct relationships. His theory of meaningful One possible way in which word clouds can
learning can be adapted for evaluating assessment be used in assessment is to gauge the use of key
data, particularly the issues concerning: (1) how words in student responses to essay or short answer
knowledge is organized; (2) how the mind works questions. For example, in a statistics course, an
to process new information; and (3) how the instructor wished to assess if students understand
assessor applies the information in meaningful some basic concepts of linear regression. She
ways (Joyce, Weil, & Calhoun, 2000). asked students to submit electronic answers to
We propose that a word cloud may also the questions “What is regression analysis used
be considered an “infographic,” defined as for?” and “What is the underlying principle of
“visual representations of information, data or calculating the regression line?”
knowledge that present complex information
quickly and clearly” (Smiciklas, 2012), and/or
as a “dashboards,” defined as “a graphical user
interface that organizes and presents information
in an easy-to-read format” (Techtarget.com,
n.d.). Regardless of what one chooses to classify
the word cloud as, its purpose in assessment is
similar. It condenses large amounts of assessment
data based on frequencies, and presents a
holistic, themed picture that is easy for the user
to understand and comprehend, and therefore to
use in assessment decisions. Given the power of
graphic organizers, infographics or dashboards
to be impactful in assessment activities, we
propose the use of word clouds as quick, easy and
meaningful graphic dashboards in assessment.
We present some ways in which word clouds Figure 1: Student responses to the question “What is regression analysis used for?”
can be used in assessment and show examples of
actual applications to real data. We also provide By analyzing the resulting word clouds (see
some practical advice for implementing word Figures 1 and 2), the instructor was able to quickly
clouds as assessment tools and provide resources determine that students correctly used key words
for users who wish to begin using these tools. such as “relationship”, “variables”, “independent”,
“dependent” and “predict” when answering the
Using Word Clouds question “What is regression analysis used for?”
She was also able to determine that students, as
Word clouds are useful in analyzing any a whole, appeared to understand the underlying
kind of text data, including essays, short answers principle of calculating the regression line by
or written responses to survey or opinion observing the high frequency of the word “best”
questions. In this section, several examples of and common uses of the words “minimizes”,
uses of word clouds for assessment purposes are “squared” and “distance.” While this analysis
shown, including: does not substitute for a formal assessment of
• Assessing the use of key word in student short the number and percentage of students giving
answer responses; acceptable answers, it does provide a quick look
• Comparison of pre- and post-test responses for formative purposes.

Volume 58, Number 3 TechTrends • May/June 2014 39

 As figure 3 indicates, students beginning
the course have scattered and unfocused ideas
about what the study of statistics is about. They
most commonly used words are “business”,
“data” and “numbers.” On the other hand, at
the end of the course, students, as a group, have
focused in on some key statistical concepts such
as “population” and “sample” and have better
articulated that statistics is a tool for converting
data to information and making decisions.

Course Evaluations or Midterm Feedback

from Students to Teachers
Another way in which word clouds can
be used is for feedback from students to
instructors, either for midterm or final course
Figure 2: Student responses to the question “What is the underlying principle of cal-
evaluations. For example, a business statistics
culating the regression line?” instructor who recently implemented a
technology-rich approach to an introductory
Comparison of Pre- and Post-test Responses course was interested in students’ opi nions
Another way in which word clouds can be about the course. In an online survey, she
used is to compare student responses to the asked students what they liked best and least
same question before and after a course, activity, about the course. The resulting word clouds are
or intervention to look for differences. For shown in Figures 5 and 6.
example, a business statistics instructor asked By creating a word cloud from the text, the
students on the first day of an introductory instructor quickly determined that the newly
course “What is statistics?” On the final exam, implemented approach using laptops and
the instructor asked the same question of the applets was liked by students. On the other
same students and compared the word clouds hand, what students liked least was the online
(see Figures 3 and 4). homework and apparently the time it took
to complete the assignments. The instructor
was then able to follow up with students
to investigate what about the homework
was troublesome to students and to make
adjustments accordingly.

Graphical Representations of Student

Knowledge or Learning
Word clouds can also be used as a graphical
representation of student learning. For example,
in a junior-level business writing class, students
were asked to review their graded papers and
Figure 3: Student responses to “What is statistics?” on the first day of an introductory create a journal with their plan to improve
course their writing based on the comments in the
graded paper. Students were required to have
at least three journal entries based on three
graded papers. The instructor took the students’
journals entries and created a word cloud. The
resulting graphic (see Figure 7) was used by
the instructor to determine the weaknesses
these students identified by the comments of
the own papers. The word cloud identifies the
words that are most used in the journal such as
“paragraphs”, “past tense” and “I”. The instructor
can determine the areas that may need to be
addressed and/or remediated in class based on
the size of the words in the world cloud.
Figure 4: Student responses to “What is statistics?” at the end of an introductory
course

40 TechTrends • May/June 2014 Volume 58, Number 3

Assessing Experiential Learning by Analyzing
Student Reflection Papers
One way in which word clouds may be
useful to instructors is in helping to understand
what students are learning or getting out of
experiential learning. For example, an instructor
of an experiential capstone course in business
strategy and consulting requires students to
write a reflection paper at the end of the course
discussing what they have learned from their
experiences. The resulting word cloud (see
Figure 8 on the following page) shows that
students placed emphasis on working in teams
and helping the community, organization or
business.
The instructor can use this word cloud as
Figure 5: Student feedback about what they liked most about the class
a starting point for more closely analyzing the
reflection papers. He can note the most common in a business writing course, one assignment
key words and infer from them which phrases or required students to write about their own
concepts are most likely to be cited by students. progress toward graduation. One paper used the
This approach could save time when coding or first person, passive tense, and slang extensively
gathering information from a large amount of (see Figure 11).
text for assessment purposes.

Programmatic Assessment Using

Longitudinal Data
When faculty wish to compare longitudinal
data to track progress of students over the
course of a program, word clouds can provide
a quick and simple visual representation of
student development. For instance, art faculty
wished to compare responses of first-year art
majors with those of upperclassmen when both
groups of students were asked to compare the
same two portraits. Faculty were interested in
how student responses changed over the course
of the program.
In Figure 9 (see following page), the re-
sponses of freshmen art majors are displayed. Figure 6: Student feedback about what they liked least about the class
Note that the most commonly used words, with
the exception of color, are simplistic and not
focused on artistic words or concepts. On the
other hand, when junior and senior art majors
compared the same two portraits, their analy-
ses were more sophisticated, focusing on words
such as “strokes”, “brush”, “viewer” and “model”
(see Figure 10). Art faculty may analyze these
clouds by, for example, comparing them to text
or key concepts generated by faculty.

Formative Assessment Feedback to Students

Another way in which word clouds can be
used is to provide feedback to students on their
writing. For example, if a student submits a paper
or other writing assignment that the instructor
wants to critique, the instructor could provide a
word cloud of the text to help the student visually
identify issues with their writing. For example, Figure 7: Students’ weaknesses as identified in their writing improvement plans

Volume 58, Number 3 TechTrends • May/June 2014 41

 which word clouds could be
utilized. For example, if data
are gathered from a focus group
for any kind of assessment,
a word cloud could be an
initial step in honing in on the
important concepts identified
by the group. This step could
save a great deal of time in
coding qualitative data, since
the researcher would already
have an idea of which terms
or ideas are most common
Figure 8: What students learned from experiential capstone course and would therefore warrant
in business consulting closer investigation. In this
way, the word cloud acts as a map for further
analysis. This approach would also be helpful
By providing word cloud feedback to the when analyzing text data from exit interviews,
student, the instructor was able to quickly show open-ended survey questions, student-teaching
the student the common occurrences of “I” and or internship supervisor evaluations, or any
“I’m”, which they were instructed not to overuse, qualitative research or assessment data.
as well as the passive verbs “been” and “being.” Though we have, in the past, used word clouds
In addition, the instructor was able to illustrate as initial screening tools for our own qualitative
the extensive occurrence of the word “lot” as research data, we do not claim that it replaces a
well as the use of an informal term “bumps” careful and detailed analysis of text. On the other
(as in “bumps in the road). This feedback, hand, in some situations and depending on the
accompanied by written text, can help students particular user and the particular application, it is
process and correct their mistakes in order to plausible that a word cloud could provide all the
improve their writing. information that is needed.
Using Word Clouds in Qualitative Research Practical Advice for Implementing Word
The uses of word cloud given here are just a Clouds as an Evaluation Tool
few ways in which these graphical displays can There are several websites that create word
be used in assessment. There are other ways in clouds, for example,
• www.worditout.com
• www.wordle.net and
• www.tagcrowd.com.
The advantages of these sites include that
they are free, easy to use, fast and platform
independent. Each of these sites has slightly
different options and capabilities. Depending
on the site, there are options for color, text and
orientation settings. Some sites allow you to
download the resulting cloud as a .png or .pdf
file, while others require you to do a screen
capture to preserve the cloud you created.
Besides the visual appearance of the cloud,
the main concern of users is probably the
issue of which words to exclude and which to
combine. For example, if the text being analyzed
is in response to the question “What did you
learn from …?” then you will probably want to
exclude the word “learned” because it is part of
the prompt. Similarly, users may wish to combine
different forms of verbs, for example, “analyze”,
“analyzed” and “analyzing.” Another possibility
is keeping words from commonly used phrases,
Figure 9: First-year art students comparison of two portraits

42 TechTrends • May/June 2014 Volume 58, Number 3

Figure 10: Third- or fourth-year art students comparison of two portraits

Figure 11: Writing feedback to student

Volume 58, Number 3 TechTrends • May/June 2014 43

 perhaps terminology or concepts from the perceive and act on this feedback, and if this
subject, together. Some sites allow you to easily is an effective way to help students learn and
exclude words you don’t want to show in your improve their work.
word cloud, while others require you to clean In closing, a picture is worth a thousand
the data before entering it. This cleaning process words. Once users are able to overcome the
involves using editing features of Word to find perception that word clouds are pretty and
and replace words. It is likely that a user will end fun toys, they will realize that these graphical
up creating a few clouds before settling on one representations can be impactful and useful as
that it meaningful, since it is sometimes difficult research and assessment tools.
to know which words should be combined or
Concetta A. DePaolo is a Professor of Operations &
excluded until one explores an initial cloud. Supply Chain Management in the Scott College of Business
There are some obvious limitations to at Indiana State University. She earned her master’s and
creating word clouds for evaluation. Besides Ph.D. in Operations Research from Rutgers University, and
changing the visual appearance and excluding an undergraduate degree in Mathematical Sciences from
some words, creating a word cloud is a one- Worcester Polytechnic Institute. Her research interests include
trick pony. There is not a lot one can do to vary optimization, statistical methods, and statistics education,
the tool. In addition, a word cloud does not and teaching and learning with technology. She has taught
replace a careful qualitative analysis of the data. statistics, management science and decision modeling in the
It is useful as a quick look, but does not provide Scott College since 2000.
details about the text or how the words were Kelly Wilkinson is a Professor in the Management,
used in context. Information Systems, and Business Education department
in the Scott College of Business at Indiana State University.
Conclusions She has a bachelor’s and master’s degree from Arkansas State
University and a Ph.D from the University of Missouri-
A word cloud provides a graphical represen- Columbia. Her research interests are pragmatic use of
tation of knowledge that allows a viewer to form technology in learning and assessment.
a quick, intuitive sense of a text. This tool is an
easy way to share high-level data without infor- Correspondence in regard to this paper should
mation overload for the user. be addressed to: Concetta A. DePaolo, Department of
Marketing & Operations, Scott College of Business, Indiana
Many people may be familiar with word
State University, Terre Haute, Indiana 47809 (Phone) 812-
clouds for amusement or entertainment, 237-2283 (Email) [email protected]
drawing on the visually appealing, artistic
presentation and the “wow” factor; however, References
when one looks beyond the surface, the
uses for assessment and evaluation become Fountas, I.C., & Pinnell, G.S. (2001). Guiding readers and
apparent. Clouds provide instant graphics to writers: Grades 3–6. Portsmouth, NH: Heinemann.
represent student knowledge for formative and Havley, M. & Keane, M.T. (May 2007). An assessment of tag
summative assessment purposes and can be presentation techniques, poster session presented World
Wide Web Conference, Banff, Alberta, Canada.
used in activity, course or program assessment.
Instructional Strategies Online. (2004-2009). What are
While preserving the anonymity of the subjects, graphic organizers? Retrieved October 27, 2012 from
they show immediately what common themes Instructional Strategies Online: http://olc.spsd.sk.ca/
and phrases appear in the text, providing an de/pd/instr/strats/graphicorganizers/index.html.
excellent starting point for analysis and coding Joyce, B., Weil, M., & Calhoun, E. (2000). Models of
of qualitative data. They can be used in gap teaching, Boston: Allyn and Bacon.
analyses, providing a quick snapshot of what is Smiciklas, M. (2012). The Power of Infographics: Using
missing and what is expected in the text. Pictures to Communicate and Connect with Your
We believe there could be more investigation Audience. Pearson/Que Publishing: Indianapolis, IN.
into word cloud feedback to students as part of Techtarget.com. (n.d.). Definition: dashboard, Search CIO.
formative assessment of writing or submissions. Retrieved October 27, 2012 from What is a dashboard?:
http://searchcio.techtarget.com/definition/dashboard.
Further research might explore how students

44 TechTrends • May/June 2014 Volume 58, Number 3