Lily Award Assessment

Fall 2022
Lily Award Assessment Project Fall 2022 2

Contents
Words From Our Team............................................................................................................3
Overview of Lily Community Engagement Award....................................................................4
Program Outcomes/Strategic Planning...................................................................................5
Assessment Methods...............................................................................................................6
Recommendations...................................................................................................................6
References.............................................................................................................................10
Lily Award Assessment Project Fall 2022 3

Words From Our Team

To the Western Carolina University community,

We partnered with the Center for Community Engagement & Service Learning (CCESL) to
share the story of the Lily Award. After analyzing current existing data, we provided
recommendations to ensure program sustainability. This includes recommendations for the
program’s assessment process going forward and the program’s structure.

This report includes a brief description of the use of assessment practices within higher
education, an overview of the Lily Award Program, limitations to our research, and program
recommendations. Through this process and report, we hope to enact efficiency in CCESL's
department and enhance the student experience. We hope our work will assist the Center for
Community Engagement & Service Learning’s efforts in impacting the student experience while
simultaneously serving the community.

Sincerely,
Casey, Emma, Gwyn, Isaiah, and Zach
Lily Award Assessment Project Fall 2022 4

Overview of Lily Community Engagement Award

The Lily Community Engagement Award (LCEA) is awarded to students through the Center for
Community Engagement and Service Learning (CCESL) at Western Carolina University.
CCESL’s mission “focuses on the development, promotion, and measurement of programs and
initiatives that engage all partners in the mutually beneficial process of community development,
with the intention of fostering a sustainable campus culture and the personal habit of community
engagement in our students” (Western Carolina University, n.d.). The award was first approved
by WCU’s Student Government Association (SGA) and the Commencement Committee in
March 2013, with the first LCEA awarded in May 2013. In 2015, the LCEA with distinction was
created and in 2016, the first LCEA with distinction was awarded.

As of August 2022, the LCEA was “developed to encourage and reward students who participate
in community engagement opportunities at WCU, the Lily Award is open to all students.
Engagement activities are divided into four categories” (Western Carolina University). To
qualify for the Lily Community Engagement Award, students must maintain a 3.25 GPA,
complete 15 direct service opportunities, and complete two of the three options:

1) Pass two designated service-learning courses.

2) Complete five Awareness & Advocacy service opportunities.
3) Complete five Donation & Funding service opportunities.
Lily Award Assessment Project Fall 2022 5

Program Outcomes/Strategic Planning

The Center for Community Engagement and Service Learning has two Strategic Directions that
the Lily Community Engagement Award falls under:

 Strategic Direction #1: ENGAGE, DEVELOP, AND SUPPORT STUDENTS IN THE

SERVICE AND COMMUNITY-FOCUSED SIDE OF ACADEMICS
o Goal 1.1: Intentionally engage students with structured community-based projects
in curricular and co-curricular capacities that respect an order of increasing
intensity of experiences.
 Initiative 1.1.1: Develop and sustain the Lily Community Engagement
Award Program as a system for incentivizing sustained student
engagement with co-curricular community projects.
 Strategic Direction #4: RECOGNIZE, SUPPORT, AND REWARD EXEMPLARY
COMMUNITY ENGAGEMENT
o Goal 4.2: Develop and where appropriate align internal recognition systems to
inspire and incentivize exemplary external engagement amongst students.
 Initiative 4.2.1: Develop a systematic process (e.g., Lily Community
Engagement Award) to track, assess, evaluate, and provide reflection
opportunities for students who engage in co-curricular activities organized
by the CCESL and other organizations on campus that recognizes and
rewards students’ sustained involvement with service work during their
time at WCU.

The LCEA meets both strategic directions of supporting students in service and recognizing
exemplary community engagement. As it stands now, and prior to meetings with the new
Director of CCESL, Dr. Amanda Hall, the LCEA does not have stated or written learning
outcomes. Given the existing strategic initiatives by CCESL, our group has three proposed
learning outcomes for the LCEA. Although we have offered three learning outcomes, this is not
an exhaustive list. These learning outcomes can be expanded or added to the existing strategic
directions and any new contextual development within CCESL. The three learning outcomes we
have curated are:

1) Students will identify service-learning sites on and off campus.

2) Students will demonstrate civic engagement on campus and in local communities.
3) Students will promote issues of diversity and democracy through awareness and
advocacy service opportunities.
Lily Award Assessment Project Fall 2022 6

Assessment in Higher Education

Assessment’s role in higher education is essential to the continual improvement and

accountability of programs, functional areas, departments, and institutions at large (Schuh et al.,
2016). Student affairs professionals participate in assessment when they gather data on their
programs and use this data to inform decisions on improving programs’ effectiveness at
achieving the programs’ learning outcomes. Upcraft and Schuh (1996) provided a succinct
definition of assessment as, “any effort to gather, analyze, and interpret evidence which describes
institutional, departmental, divisional, or agency effectiveness” (p. 18). To ensure that the
assessment cycle gathers data that clearly shows a program’s effectiveness, each part of the
process must be clearly defined to keep the process consistent while providing accountability for
how program’s effectiveness is represented at the end of an assessment cycle (Schuh et al.,
2016).

Our Process

At our initial consultation meeting with CCESL, we discussed the vision for this project with the
previous Interim Director of CCESL, Dr. Jennifer Cooper. Our team was asked to explore what
barriers exist that prevent students from completing the Lily Award, as well as the motivations of
students who complete it. During the early stages of this project, we received data sets and
promotional materials about the Lily Award. After reviewing all the materials CCESL was able
to provide, the team came to the consensus that we would be unable to achieve the original goal
for the project. Due to current data structure and assessment practices within CCESL, we were
unable to locate or generate the needed and desired data sets to identify barriers and motivations
of Lily Award students.

At that point in the project experience, our team set out to make a series of recommendations that
fell into two categories: ways in which to run the Lily Award program and creating an
assessment plan. All recommendations and tools built by our team were made to improve how
the Lily Award is run, as well as create a date rich environment within the CCESL office. By
implementing our recommendations, the CCESL office will have the right data sets and
assessment practices to answer the questions of what barriers exist that prevent students from
completing the Lily Award and why students choose to complete the award.

Example of Service-Learning Award Program from another institution

To help inform our recommendations, we looked at what other institutions are doing for their
Service-Learning Award programs. At Michigan State, the Spartan Volunteer Service Award is
awarded to students who serve a minimum of 100 hours in a variety of Service and Civic
Engagement focused events. The different areas where one can accrue these hours include:

 Advocacy
 Community Conversations
 Community and Economic Development
 Democratic Engagement
Lily Award Assessment Project Fall 2022 7

 Philanthropy
 Volunteering

This award utilizes an app to help students input and track hours. Students input their student ID
number, and then begin inputting their experiences. Students must reflect on their experiences
when they submit each of their individual hours. In this way, students still take part in a
reflection experience. The ease of using a phone application should help encourage student
participation and make it easier for staff to track data. These ideas around ease of accessibility
for students and ease of staff assessment are why we chose this program as an example program
that the Lily Award can model itself after or draw inspiration from. Of course, Michigan State
University is a larger institution than Western Carolina University. However, utilizing the work
of other institutions to inform our own creates dynamic and up to date programs that are current
with trends within Higher Education.

Recommendations

The Lily Award has a well-built foundation for incentivizing and rewarding students who
make service learning a part of their college experience. Looking towards the future, this
assessment team has a few recommendations worth considering.

Integrate the Lily Award into Service Learning Courses

One of the roles of the Center for Community Engagement and Service Learning at WCU
is mediating how service can be combined with academic course work. Because of this, many
students on WCU’s campus are introduced to service learning through academic coursework.
Best practices for introducing students to service in an academic setting call for reflection as a
meaning making tool; therefore reflection assignments are required in service-learning
designated classes. Students in these classes are in a frame of mind to value their service work,
and the Lily Award can be used as a tool to incentivize students to continue service past the
completion of their specific service-learning course. Our recommendation is making enrollment
in the Lily Award program a strong recommendation of students in service-learning courses. To
do this, professors who seek to fulfill the requirements of making a course service-learning
designation, must be properly trained on instructing students on how to navigate the Lily Award.

Require Students to Complete Intake Survey

As it is currently run, students who complete steps towards earning the Lily Award
record these acts of service through Engage, WCU’s event planning and signup software.
Students who submit an act of service on Engage that counts as a Lily Award service opportunity
become part of the tracking process for the award. Because of this, many students start the
award, but do not finish it. To better understand the students who are taking part in the Lily
Lily Award Assessment Project Fall 2022 8

Award process, and to better help facilitate the future assessment of the program, we suggest an
intake survey be mandatory for students who intend on starting the program. This survey will
provide Lily Award facilitators demographic data, reasons why students are taking part in the
program, attitudes and prior experiences with service learning, and intended areas of service
(Appendix D).
As well as providing onboarding data for future assessment, the intake survey will help
narrow down the number of students in the program, helping the Center for Community
Engagement and Service Learning staff work more closely with students who are more likely to
commit to finishing the award. As mentioned previously, most students who begin the Lily
Award process do not finish it. This intake form will help us assess why, but also assure that
students understand what they are getting into. Students who are not serious about the Lily
Award will be more likely to not submit this intake form, meaning there will be less students in
the Lily Award system overall to oversee.

Utilize Canvas to Track Students

Canvas is a software that allows academic administrators and professors to create

groups and webpages for classes. On these pages, assignments can be submitted, syllabus and
other course resources can be accessed, and collaborative forums can be created. Some non-
academic programs have begun using Canvas to organize groups around a common objective.
We suggest the Center for Community Engagement and Service Learning utilize Canvas to more
effectively organize the Lily Award. This means having a central space where program
facilitators can communicate with students, and reflections from students can be submitted. Just
as important is a space where students can track their progress and communicate with organizers
to make sure they are meeting the requirements of the Lily Award.
Students must opt-in to receive access to the Canvas course. Once students opt-in, they
will have access to the Canvas course (Appendix E). As students complete their 15 service
opportunities, they will complete a Qualtrics form for each service opportunity to be approved by
CCESL staff (Appendix F), then upload one reflection on the Canvas course after the completion
of the 15 opportunities. A document will be available to students to download to assist in
tracking their 15 service opportunities (Appendix G). When students progress to completing the
5 service activities for either Awareness & Advocacy or Donation & Fundraising, they will
download and submit the Lily Award Assignment (Appendix H) for each service activity. They
will then complete one reflection for their 5 service opportunities. Additionally, a visual tracker
at the bottom of each section will assist students monitor their progress.

Learning Management Systems (LMS) have been adopted by higher education

institutions to offer course flexibility, provide students with resources, and organize course
content. Canvas offers a variety of tools that can be useful for CCESL staff and students who
pursue Lily Award completion. Canvas offers a centralized location for staff and students to
communicate, submit documents, and share resources. An organized Canvas page can offer
students a visual map for successfully achieving the award. Tseng (2020) stresses the importance
of creating an “effective and user-friendly environment” when using LMS in students’ learning
(p. 14). After analyzing the impacts of LMS on student learning, Tseng (2020) reports students
identified LMS as a tool that “creates an effective learning environment” and strengthens the
communication and evaluation processes that involve students (p. 14).
Lily Award Assessment Project Fall 2022 9

Tracking of student progress is a key element to the Lily Award and can be done via
Canvas. As students submit their reflections, staff may evaluate and offer feedback on their
submitted work. When students submit work that meets the standards communicated by the
CCESL staff, they advance in the award process. As students will be at different levels of award
completion, Canvas can track student progress, which will be useful for staff and students. An
additional visual tool on users’ Canvas homepage to measure award progress can prove
beneficial and motivational. In researching LMS tools, Joo et al. (2016) found “perceived ease of
use” impacted students’ perceptions of LMS usefulness (p. 625). The authors continue by stating
that “perceived usefulness” increased student satisfaction levels, which generated greater rates of
continued LMS usage.

Streamline Excel Spreadsheet Data Management

Excel as a data management tool is important for tracking student completion rate
towards fulfilling the Lily Award requirement. Current data that is pulled from Engage does not
provide clear insight into which requirements students have completed, along with some students
representing multiple entries in the data without data rules to explain this phenomenon.
Streamlining the information that is collected and tracked in conjunction with the Canvas page’s
reflection data will allow for CCESL staff to know which students are completing requirements,
also if they are achieving the previously discussed learning outcomes. Excel spreadsheets can be
limited to two main data tracking documents, one will be for students that have gone through the
intake process and their completed events. This will be on a semester-by-semester basis so that
the spreadsheet does not become over encumbered with data, while also providing a clear
snapshot of where each participating student is in the process. The other will track the SLC
designated courses, awareness and advocacy events, and donation and fundraising events with
the expressed purpose of keeping records of Lily Award designated courses and events, as well
as the students that attend them. This will allow for CCESL staff to analyze which courses and
events students are engaging with, which provides guidance for what programming to offer from
semester-to-semester. It will also provide staff with a picture of what programming needs to be
reimagined or revised to meet student learning goals and expectations, possibly even
discontinued to focus on new programming.

Create a Culture of Assessment

The Lily Award program impacts student’s experiences at WCU. To gather more
information about how much or how little the program impacts students, a culture of assessment
must be created. We suggest utilizing pre and post program surveys to better understand
student’s intents and outcomes for the program. Focus groups should also be used to gather
qualitative data about student experiences. Because so few students who start the award finish it,
more data would help identify what makes students successful in the program. Of course, the
spreadsheets and intake/exit surveys mentioned above will help facilitate this process, but formal
assessment is needed to help make sense of the data.
Assessment helps tell the story of students in the Lily Award and utilizing annual or bi-
annual assessment will provide Center for Community Engagement and Service-Learning staff
Lily Award Assessment Project Fall 2022 10

to make decisions about the future of the Lily Award. We suggest annual focus groups, an exit
survey and post-graduate interviews because they do an excellent job at providing building
blocks for the story of the Lily Award. Working assessment as a consistent part of the program
will lead to a more cohesive goals and departmental understanding about how the program is
performing. Since the outcomes and performance of the Lily Award was not being tracked in a
consistent manner, little data could be derived about the program. To combat that, leadership
focused on consistency in the assessment of the Lily Award should be considered in the goal of
creating a culture of assessment.

In order to measure the degree to which the proposed Lily Award outcomes are met, the team has
designed three assessment tools. Each tool serves to measure the program outcomes and
contribute to a cycle of assessment.

To generate data from students actively participating in the award, we created questions for an
annual focus group (Appendix A). Focus groups "help the researcher capture shared lived
experiences, accessing elements that other methods may not be able to reach” (Liamputtong,
2011). By conducting a 6-8 person focus group of active Lily Award participants, the office will
be able to identify common trends and address issues as they occur.

To generate data from students who have completed the Lily Award, we created an exit survey
for each Award recipient to complete before receiving their award and regalia (Appendix B).
This exit survey serves to gain holistic insight from students who saw the Award through to
completion.

In order to understand the long term impact of completing the Lily award, we have created a post
graduate survey for alumni who earned the Lily Award to complete one and two years after
graduation (Appendix C).

Conclusion

The Lily Engagement Award represents the mission of CCESL by improving student and overall
community engagement through advocacy, fundraising, and service-learning experiences. This
assessment project looked to provide an assessment plan for the efficiency and efficacy of the
program. This was done through creating student learning outcomes that are aligned with
CCESL’s strategic directions 1 and 4, along with recommending a restructuring of the award that
models institutions such as Michigan State University. Recommendations continue to streamline
the process of bringing students into the program, housing student reflection data in a
centralized, designated Canvas page, and Excel data management documents that will track
students’ program completion on a semester-by-semester basis. These recommendations are not
an exhaustive list of options for improving the Lily award going into the future, and we
acknowledge that the award may be taken in a different direction going forward. This assessment
plan hopes to establish a culture of assessment in CCESL which will cultivate improving the
program, among others, going forward.
Lily Award Assessment Project Fall 2022 11
Lily Award Assessment Project Fall 2022 12

Appendices

APPENDIX A: Focus Group Questions

1. Why did you begin the Lily Award process?

2. How invested did you feel throughout the process of completing the lily award?
3. What barriers have kept you from completing this award, if any?
4. How did you navigate barriers you encountered while participating in the Lily Award?
5. What support did you receive that helped you complete the Lily Award, if any?
6. How did this award experience meet your expectations, or fail to meet them?
7. What did you think we were going to talk about today that we didn’t ask you?

APPENDIX B: Exit Survey Questions

1. Self Reported Demographic Information

a. major, age, class (freshman, etc), gender, race/ethnicity, self-reported GPA, any
extracurriculars and/or job
2. Duration of award completion?
3. Post Graduation employment? If yes, what?
4. Length of Degree Completion (optional)
5. Why did you complete the Lily Award process?
6. How invested did you feel throughout the process of completing the lily award?
7. What barriers did you encounter while completing this award, if any?
8. How did you navigate barriers you encountered while completing the Lily Award?
9. What support did you receive that helped you complete the Lily Award if any?
10. How did completing this award impact you personally and/or professionally?

APPENDIX C: Post Graduation Survey Questions

1. Are you employed? If yes, what is your job?

2. Has the lily award impacted or affected your professional performance?
3. Has the Lily Award impacted or affected your personal development?
4. Have you gained new experiences in the past year? (graduation to year one, year one to
year two)

APPENDIX D: Student Intake Form Questions

1. What is your preferred name?
2. What is your 920 #?
3. What is your gender identity?
4. What is your racial/ethnic identity?
5. What is your undergraduate major?
6. What is your age at the time of entering the award program?
7. What class are you in? (ex. freshmen, sophomore, junior, senior)
8. What is your current GPA?
9. Do you participate in any extracurriculars? (ex. clubs, student organizations, etc.)
10. Are you currently working? if yes, are you working part-time or full-time?
Lily Award Assessment Project Fall 2022 13
Lily Award Assessment Project Fall 2022 14

APPENDIX E: Canvas Outline

Lily Award Canvas Page
1. Lily Award Overview
o Introduction
 What the award is and why students should participate. (Benefits of
participation)
 What students receive upon completion
 Graduation cord, certificate, recognition at annual awards
ceremony
 Previous award recipients’ testimonials
 Written samples/quotations
 Stretch Goal: Video of WCU alum who also completed Lily
Award- How the experience impacted their professional
development (Partner with WCU Marketing team)

o Lily Award Steps

 Requirements (GPA and 15 service opportunities with one reflection)
 Select 2 of 3
 2 Service-Learning Courses
 Awareness & Advocacy service opportunities- 5 and a reflection
 Donation & Fundraising service opportunities- 5 and a reflection
 Describe the visual tracking system which measures students’ progress

2. Direct Service
o Direct service introduction
 Description of what qualifies as a direct service opportunity and that it
must be outside of Western Carolina University.
 Examples of direct service opportunities.
 List of events/upcoming projects students can register for- include
date/time/location of event

o Direct service tracking

 Complete and submit this Qualtrics form for approval for each of the
service opportunities completed. (15 total)
 Provide Qualtrics link
 Personal tracking document (for students’ own records)
 Direct Service Reflection
 Students submit as assignment
Lily Award Assessment Project Fall 2022 15

 Include visual tracker at bottom of page (measures student’s progress)

3. Awareness and Advocacy

o Intro page
 Overview of process- Complete 5 opportunities (submit 1 assignment for
each) and 1 overall reflection (Provide a list of appropriate service
opportunity examples for this category)
 Provide list of upcoming events that students can register for to meet
requirement
o Assignments
 Students download and submit the Lily Award assignment for each service
opportunity (5 total)
 Staff review each uploaded assignment for approval
o Reflection
 Only 1 reflection (Use existing Lily Award reflection template)
o Visual tracker at bottom of page measuring students’ progress

4. Donation and Fundraising

o Intro page
 Overview of process- Complete 5 opportunities (submit 1 assignment for
each) and 1 overall reflection (Provide a list of appropriate service
opportunity examples for this category)
 Provide list of upcoming events that students can register for to meet
requirement
o Assignments
 Students download and submit the Lily Award assignment for each service
opportunity (5 total)
 Staff review each uploaded assignment for approval
o Reflection
 Only 1 reflection (Use existing Lily Award reflection template)
o Visual tracker at bottom of page measuring students’ progress

5. Service-Learning Designated Courses

o Intro page
 Overview of process- Successfully pass 2 designated SLC courses and
complete an overall reflection
o Course information
 List of WCU courses offered that are SLC designated courses
 Course name, number, and upcoming offerings
Lily Award Assessment Project Fall 2022 16

o Reflection
 Only 1 reflection (Use existing Lily Award reflection template)
o Visual tracker at bottom of page measuring students’ progress
Lily Award Assessment Project Fall 2022 17

APPENDIX F: Qualtrics
Question 1: Where did the service occur? (Include title of event if applicable)
Question 2: Please indicate the service date.
Question 3: Provide the event/organization’s point of contact information. (Name, phone and/or email)
Question 4: Briefly describe the service opportunity. Who did you serve and how? What did you do? (4-5
sentences)

APPENDIX G: Direct Service Tracker

Direct Service Tracker

You can use this document to track your 15 direct service opportunities. This is for your records
and does not need to be uploaded to Canvas as you will complete a Qualtrics form for each
service opportunity.

Service Event Date of Contact Name Contact

Service Phone/Email
1.
2.
3.
4.
5.
6.
7.
8.
9.
10
.
11
.
12
.
13
.
14
.
15
.

Thank you for serving your community and representing WCU!

Lily Award Assessment Project Fall 2022 18
Lily Award Assessment Project Fall 2022 19

APPENDIX H: Lily Award Assignment

Service Event Information

Date of Service Contact Name Contact Phone Contact Email

1. Please briefly describe your service opportunity. Include the name of the organization
where you volunteered and your service responsibilities. (4-5 sentences)

2. Was this opportunity affiliated with a club/organization/office at Western Carolina

University? If so, please identify.

Thank you for serving your community and representing WCU!

References

Liamputtong, P. (2011). Focus group methodology: Introduction and history. SAGE

Publications Ltd, https://dx.doi.org/10.4135/9781473957657

Tseng, H. (2020). An exploratory study of students’ perceptions of learning management system

utilization and learning community. Research in Learning Technology, 28(1), 1-19.
https://doi.org/10.25304/rlt.v28.2423
Lily Award Assessment Project Fall 2022 20

Joo, Y. J., Kim, N., & Kim, N. H. (2016). Factors predicting online university students’ use of a
mobile learning management system (m-LMS). Educational Technology Research and
Development, 64(1), 611-630.

Schuh, J. H., Biddix, J. P., Dean, L. A., & Kinzie, J. (2016). Assessment in student affairs
(Second ed.). Jossey-Bass.

Upcraft, M. L., & Schuh, J. H. (1996). Assessment in student affairs . San Francisco, CA:
Jossey-Bass.

Western Carolina University (n.d.). Lily Community Engagement Award.

https://www.wcu.edu/learn/academic-enrichment/center-for-service-learning/service-learning-
for-students/lily-community-engagement/index.aspx