Exploring The Function of Member Checking
Exploring The Function of Member Checking
3-24-2019
Part of the Educational Methods Commons, Quantitative, Qualitative, Comparative, and Historical
Methodologies Commons, and the Science and Mathematics Education Commons
This How To Article is brought to you for free and open access by the The Qualitative Report at NSUWorks. It has
been accepted for inclusion in The Qualitative Report by an authorized administrator of NSUWorks. For more
information, please contact [email protected].
Exploring the Function of Member Checking
Abstract
Member checking is commonly used in qualitative research as a means to maintain validity; however, little
has been published about the effects the member checking process may have on participants. In this
article, I provide the experiences of two participants in a qualitative case study and argue how member
checking should be used as a reflective space for participants.
Keywords
Qualitative Research, Methodology, Member Checking
This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 International
License.
Qualitative studies are useful when trying to deeply understand a participant’s lived
experiences, the significance of those experiences, and potentially a phenomenon that the
experiences illustrate (Morrow, 2007). When engaging in qualitative research, researchers use
many sources of evidence to increase the validity of the study (Yin, 2014). Freeman, deMarrais,
Preissle, Roulston, and St. Pierre (2007) describe validity as the trustworthiness of the data and
claim the quality of evidence needs to be apparent throughout a study. Triangulation, or cross-
checking data, involves using multiple data sources to verify themes (Lincoln & Guba, 1986;
Maxwell, 2005; Patton, 2015; Stake, 1995; Yin, 2014). Patton (2015) notes studies with only
one source are more vulnerable to errors including loaded interview questions or bias from the
researcher. Maxwell (2005) concurs, saying triangulation “reduces the risk that your
conclusions will reflect only the systematic biases or limitations of a specific collection
method" (p. 93). One important aspect of triangulation is engaging research participants in the
member checking process. Lincoln and Guba (1986) describe member checks as:
Member checking provides a way for the researcher to ensure the accurate portrayal of
participant voices by allowing participants the opportunity to confirm or deny the accuracy and
interpretations of data, thus adding credibility to the qualitative study (Creswell & Miller, 2000;
Lincoln & Guba, 1986; Stake, 1995).
Hallett (2013) cautions member checking “has become part of qualitative research
courses and discussions of ‘best practices’ without much theorizing concerning how
participants experience this practice” (p. 29). Buchbinder (2011) warns that member checking
may not actually help with validity of the study due to the power dynamic between researcher
and participant where participants may just agree with the analysis as they perceive the
researcher as having power and do not want to disagree with the findings. If participants just
agree with the researcher, the member check would not support the validity of the study.
Furthermore, researchers have argued that despite member checking being a validity
measure, the member checking process could cause harm to the participants (e.g., Goldblatt,
Karnieli-Miller, & Neumann, 2010; Hallett, 2013). Therefore, it should either be omitted from
the research process or instituted judiciously, depending on the circumstance of each
participant. Member checking could cause harm during research studies looking at experiences
of marginalized populations or participants who have experienced trauma. Hallett (2013)
speaks of the discord between wanting to create trustworthy data and doing no harm to the
participants in the study, especially because he worked with a marginalized population of
homeless adolescents. There is a counter argument to member checking as causing harm where
researchers have found the member checks and participant interviews to be similar to therapy
for the participants (e.g., Doyle, 2007; Harper & Cole, 2012; Koelsch, 2013; Ortiz, 2001). In a
study of women’s heterosexual experiences that were nonconsensual, confusing, or
problematic, Koelsch found the member checking experience to be reflective for participants.
Koelsch gave participants an individual analysis and an analysis of the group. Each participant
reflected on her individual experience, and some found comfort in knowing there were others
in similar situations. Similarly, in a study with older women on the impact of receiving
community care packages, Doyle (2007) found the member checking process to be therapeutic
for her participants who were able to see their stories from new perspectives and increase their
self-understanding of their experiences with community care packages. As qualitative
researchers gain an understanding of those involved in their study, they should consider the
context of their study while thinking through how the member checking process could
ultimately affect participants.
In regards to the procedure for completing a member check, several researchers give
guidance on how to complete a member check (e.g., Creswell, 2005; Stake, 1995; Yin, 2014).
Stake (1995) refers to participants as “actors” and speaks of the member checking process as
requesting the actor to look at the rough draft in which he or she is featured. The purpose is for
the actor to check for accuracy and possibly to encourage an alternate interpretation. Although
the actor’s feedback can be helpful, whether the actor disagrees or agrees with the analysis, the
researcher does not guarantee that interpretation will end up in the final draft. Stake cautions
often actors will not acknowledge the request from the researcher, however research has been
improved upon by engaging in the member checking process. Creswell (2005) describes the
member checking process as follows:
Member checking is the process in which the researcher asks one or more
participants in the study to check the accuracy of the account. This check
Amber G. Candela 621
involves taking the findings back to the participants and asking them (in writing
or in an interview) about the accuracy of the report. You ask participants about
many aspects of the study such as whether the description is complete and
realistic, if the themes are accurate to include, and if the interpretations are fair
and representative. (p. 252)
According to the National Council of Teachers of Mathematics (2014), “to ensure that
students have the opportunity to engage in high-level thinking, teachers must regularly select
and implement tasks that promote reasoning and problem solving” (p. 17). This type of task
can be classified as having high cognitive demand (Smith & Stein, 2018). However, it was my
experience in classrooms that this was not always occurring. While research exists on teachers
who chose and enacted high cognitive demand tasks (Boston & Smith, 2009; Henningsen &
Stein, 1997; Stein, Grover, & Henningsen, 1996), the researchers did not include teachers’
perspectives on the process of selecting and implementing cognitively demanding tasks to their
students. Thus, my goal was to give voice to middle school mathematics teachers who
attempted to implement high cognitive demand tasks. I wanted to gain their perspectives on
factors affecting their selection and implementation of tasks. I sought to inform those working
with teachers how best to support the selection and implementation of high cognitive demand
mathematical tasks in classrooms. By offering the teachers’ perspectives, those working with
teachers can anticipate possible roadblocks and successes teachers may have when
implementing tasks in the classrooms.
To do this, I conducted a multiple case study (Yin, 2014) to answer questions about
teachers’ perspectives with respect to the phenomenon of implementing cognitively demanding
tasks in a mathematics classroom (Candela, 2016; 2017). In particular, I was interested in
teachers’ perceptions of their challenges and successes when selecting high cognitive demand
tasks as defined by Smith and Stein (2018). As I tried to understand each teacher’s experience,
each represented his or her own case, and cases were then analyzed to determine themes. I then
looked across all the cases for common themes. The research questions guiding the study were
as follows: What are teachers’ perspectives of their classroom practice as they implement high
cognitive demand tasks and how do teachers’ perspectives compare to the researcher’s
perspective? What obstacles and successes do teachers identify when attempting to implement
high cognitive demand tasks?
During the 2013-2104 school year, I worked with three 7th grade mathematics
teachers—Mr. Cone, Mrs. O’Neill, and Mr. Fielder—at a public middle school in the
southeastern United States. The school was a Title 1 school and had approximately 730
students. As reported by the participants, roughly 60% of the students were African-American,
25% were White, 7% were Latino/a, 4% were Asian, and 4% were multi-racial. About 75% of
622 The Qualitative Report 2019
students were eligible to receive free or reduced-price lunches, and about 4% of students were
English language learners.
Mr. Cone was a mathematics and social studies teacher in his fifth year of teaching. Mr.
Cone explained that he added mathematics certification because he enjoyed mathematics and
found many students did not. He wanted the challenge of teaching the subject. Mrs. O’Neill
was a mathematics teacher in her third year of teaching. Mrs. O’Neill said that she never
thought she would be a teacher because her parents pushed her in every direction except
teaching. After 25 years of being a stay-at-home mother and working as a substitute teacher,
she decided she liked being in the middle school and pursued a teaching degree. Mr. Fielder
was a first-year mathematics teacher who had been a student teacher in Mr. Cone’s room the
year before and was certified in middle school mathematics and social studies.
I started working with the teachers during their pre-planning days in July and continued
through December of that year. I provided a four-hour professional development session before
the start of the year and engaged the teachers in activities regarding the nature of high cognitive
demand tasks, including how to classify and select high cognitive demand tasks, how to plan
collaboratively as a team, and how to implement tasks in their classrooms.
I collected data through classroom observations, teacher interviews, and audio
recordings of team meetings. Throughout the semester I attended team planning meetings with
the three teachers where we discussed the implementation of high cognitive demand tasks. I
completed three classroom observations of each teacher implementing high cognitive demand
tasks. To analyze data, I utilized the Mathematics Task Framework (Stein, Grover, &
Henningsen, 1996), the Instructional Quality Assessment (Boston, 2012), and the Five
Practices (Smith & Stein, 2018) as the basis for coding interview transcripts and classroom
observations. The goal for data analysis was to gain teachers’ perspectives of implementing
high cognitive demand tasks in order to answer both research questions. Once I completed my
analysis, I engaged each participant in the member checking process.
Using the guidelines of Creswell (2005), Stake (1995), and Yin (2014), I supplied the
participants with copies of the completed analysis. Specifically, I wanted to find out if the
teachers agreed with the factors I identified as influencing the implementation of high cognitive
demand tasks in their classrooms, whether I had omitted any important factors, or if I had
misrepresented the teacher’s experience in any way. I emailed each teacher a copy of the
analysis, which included each teacher’s individual analysis and a write up of common themes
across all teachers and asked if each could respond with a time to meet and discuss thoughts
on the analysis. Mr. Fielder responded he read over the documents and was fine with my
analysis; he provided no further comments. As such, the focus of this manuscript will be on
how Mr. Cone and Mrs. O’Neill participated in the member check. I set up individual
interviews with Mr. Cone and Mrs. O’Neill and audiotaped those interviews. I transcribed each
interview and analyzed the interviews coding line by line using my original framework. When
I realized there were themes outside of the original framework, I went back through each
teacher’s member check interview line-by-line and found instances where the teacher made a
comment about the member checking process. In a separate document, I created narratives of
these instances, separating the data into instances where the teachers reacted negatively or
positively to the member checking experience. I coded each teacher’s member checking
interviews individually first and then looked for common themes across both cases, making
note of where the teachers had common experiences. I wrote up the analysis of each teacher’s
member checking experience with the themes I found.
Amber G. Candela 623
Mrs. O’Neill responded in an email saying, “Honestly, I am not thrilled at how we come
across. It is very negative and that is not how I had intended for my part to sound.” She
commented that my writing was repetitive related to the teachers’ lack of time and lack of
planning, and thought the overall tone was too negative. However, she did not believe it really
mattered what she thought at that point and expressed how excited she was for me to finish the
research process. I replied in an email that it was not my intention for the teachers to come
across as negative, but the results indicated lack of planning and lack of time contributed to the
teachers choosing to not implement high cognitive demand tasks. I clarified that her opinion
did matter, as the purpose of my research was to accurately portray the teachers’ voices and
asked if she would meet in person and discuss her comments further. When I met with Mrs.
O’Neill in person, she explained:
I know that we expressed to you often our lack of time that was just ours alone
for planning and that kind of thing. So that filtered out into we didn’t have time
to research, to find tasks and all that, and having you do that was awesome. But,
I felt like when you wrote it out, it came across as extremely negative. It [the
analysis] didn’t come out in a very flattering way for the county, for the school,
or for us.
She did not identify any specific parts of the text that were negative; rather, she said she felt
the overall tone was negative. She agreed with my statement that due to the teachers’ many
responsibilities there was not time in the day for quality planning, and she elaborated on her
many roles within the school which included team leader, data team leader, and having a
prospective teacher intern. Mrs. O’Neill commented she was just putting in her “two-cents’
worth” and wanted to make sure I realized the negative tone of the analysis, but ultimately she
said it was up to me if I wanted to change anything. During the member check she revealed a
more deficit perspective of her students as a cause for not implementing tasks, and I wove that
into the analysis, as it was more evident from her member checking interview than from her
original interviews. In response to Mrs. O’Neill’s comments, I fixed some repetitiveness when
talking about lack of time and lack of planning, but overall, I felt the tone of the analysis was
accurate. I also added to the data to support themes around deficit perspectives of students as a
cause to not implement high cognitive demand tasks.
When I met with Mr. Cone in person, he related the member checking experience to
being videotaped saying it was “good and bad at the same time, and very, very honest.” He
commented it was interesting to read my perspective of him in the classroom, even if it was
not exactly how he wanted me to perceive him, but he recognized my perception of his
classroom was helpful in helping him think about how he wanted his classroom to be. He
acknowledged what Mrs. O’Neill had said about the analysis being repetitive but saw the
repetition in my writing as highlighting what the teachers did over and over again and how that
helped him see his ways of being in the classroom and his practices when implementing high
cognitive demand tasks. He remarked on how he was going to use the analysis from the second
research question to help him in the future:
When I read about myself saying I altered things I was like “Oh, I did do that,
lowering the cognitive demand or having lowering expectations for [the
students].” It was something that I say I am going to reread in August before the
first day of school, and maybe every month, to remind myself that I shouldn’t
be doing those things.
624 The Qualitative Report 2019
The session concluded with Mr. Cone affirming what I wrote was an accurate portrayal, and
therefore I did not change any part of my analysis. During the session, I was able to see how
Mr. Cone reflected on his teaching and took the feedback as a way to improve his teaching.
Through the member-checking interview I was able to collect more data on his experience
engaging in the professional development and his perspective on his teaching (Yin, 2014). I
corroborated the findings from the member check with Mr. Cone and incorporated Mr. Cone’s
reaction of the member check being reflective on his teaching in my analysis.
Discussion
Member Checking as Causing Harm. Mrs. O’Neill responded very negatively to the
analysis and thought my portrayal of the teachers was not flattering for the teachers, the school,
and the district. She felt I kept repeating the same sentiment over and over again with relation
to the teachers having a lack of time and resources to plan and implement high cognitive
demand tasks; she read the document as very negative and was concerned that was not the way
I wanted to come across. The nature of the research was to identify factors that promoted or
inhibited teachers’ use of high cognitive demand tasks in the classroom. During analysis, I
found that two major factors in inhibiting teachers from implementing high cognitive demand
tasks were lack of planning time and lack of resources. While I did not set out to negatively
portray the school or district, I did try to point out that many factors that constrain the use of
high cognitive demand tasks stem from school and district requirements and ultimately schools
and districts need to be aware of these possible impediments. In my attempt to highlight barriers
so that others (e.g., school administrators, mathematics educators) could realize what affects
the implementation of high cognitive demand tasks and then address those barriers, I may have
caused harm to Mrs. O’Neill as I was sending back a negative analysis of her situation.
Hallett (2013) contends researchers need to question what unintended harm may befall
participants by participating in the member checking process. While I would never have
excluded Mrs. O’Neill in the member checking process, I should have cautioned her about the
negative tone of the analysis, provided her with the space to reflect on the experience, similar
to what Koelsch (2013) employed, or have been more transparent of my goals initially and not
after she responded to my analyses. While I explained my purpose of trying to enlighten those
who work with mathematics teachers after Mrs. O’Neill had responded to the member check,
Amber G. Candela 625
I am not convinced she was able to see my perspective. I should have been better prepared to
interact with Mrs. O’Neill in a way that would not have made the analysis seem as shocking as
it must have seemed for her. I would hypothesize it was hard for anyone to read a negative
portrayal of thoughts towards students and the school. I hoped her reading of the analysis would
have helped her think differently about her practice, but I could not have expected this to just
happen without scaffolding that process. One way I could have improved this would have been
to first speak with Mrs. O’Neill and then provide an overview of the analysis in which I could
have explained the negative tone of the write up and given my reasons as to why it was so
negative in relation to my purpose with the research.
Member Checking as a Reflective Experience. When I met with Mr. Cone after he
read the analysis, he explained that while it was difficult to read about his teaching overall, he
thought it was productive to have an outsider’s view of his teaching to reflect and try to change
his practice for the better. Mr. Cone was able to take my perspective of his teaching and use
that to shape how he thought about his actions in the classroom and his perceptions of students.
He wanted to use the analysis as a future tool to remind himself of his habits in the classroom
and how he could change those to foster the mathematical growth of his students. While
researchers in the health sciences field have found member checking to be therapeutic to
participants (e.g., Harper & Cole, 2012; Koelsch, 2013; Ortiz, 2001), there is a lack of research
in how member checking affects participants in the field of education. According to Ortiz
(2001), in-depth interviews with participants provided “cathartic opportunities for self-
revelation and introspective opportunities for self-discovery, both of which may possibly
contribute the potential for transformation in self and identity” (p. 193). I would argue this
therapeutic benefit applies to educators and can provide a space for reflection on their practice.
My participants had differing experiences with the member check; however, I would
argue they both experienced a reflection on practice as a result of the member checking process.
While Mrs. O’Neill found my overall tone to be too negative, I hope that reading the analysis
did provide some space for reflection in her teaching. It may not have been a positive reflective
experience, but reading my analysis on her teaching could have been reflective. Mr. Cone used
the member check experience to reflect on his teaching. He was able to take the criticism and
reflect on how to change his practice. In thinking of this as a reflective experience for Mr.
Cone, he was comfortable reading about his teaching, even though parts were not positive, and
took these parts to help him change his teaching. While I have not followed up with Mr. Cone
to see if he has used the analysis each year as a reflective tool, this helps me to think about
using the member check as a reflective tool for teachers.
Member Checking and Power. In the end, Mrs. O’Neill said it was up to me if I
wanted to change the write up, allowing me, the researcher, to make the decision of what was
written. This aligns with prior research on the power dynamic between participants and
researchers (e.g., Buchbinder, 2011) where participants ultimately agree with the researcher
because they see the researcher as the authority. In thinking about my role as researcher in this
experience, and the power dynamic between the teachers and myself, this experience will shape
future member checking interviews. Researchers should be diligent to emphasize to
participants that the interview is a place where they can share their ideas and co-construct the
analysis. Buchbinder (2011) recommends the member check interview should “enhance
reciprocity, equality, and openness, which in turn may allow the communication of more
abundant in-depth data” (p. 119). This happened more with Mr. Cone; however, I caution all
researchers to make this a priority with all member-checking experiences.
626 The Qualitative Report 2019
Conclusion
References
Author Note
Article Citation
Candela, A. G. (2019). Exploring the function of member checking. The Qualitative Report,
24(3), 619-628. Retrieved from https://nsuworks.nova.edu/tqr/vol24/iss3/14