International Journal on Responsibility

Volume 7 Article 1
Issue 1 Artificial Intelligence and Responsibility

2024

Artificial Intelligence Chatbot as a Mathematics Curriculum

Developer: Discovering Preservice Teachers’ Overconfidence in
ChatGPT
Amanda G. Sawyer
James Madison University

Follow this and additional works at: https://commons.lib.jmu.edu/ijr

Part of the Elementary Education and Teaching Commons

This work is licensed under a Creative Commons Attribution-NonCommercial-Share Alike 4.0

International License.

Recommended Citation
Sawyer, Amanda G. (2024) "Artificial Intelligence Chatbot as a Mathematics Curriculum Developer:
Discovering Preservice Teachers’ Overconfidence in ChatGPT," International Journal on Responsibility:
Vol. 7: Iss. 1, Article 1.
DOI: https://doi.org/10.62365/2576-0955.1106
Available at: https://commons.lib.jmu.edu/ijr/vol7/iss1/1

This Article is brought to you for free and open access by the Faculty Publications at JMU Scholarly Commons. It
has been accepted for inclusion in International Journal on Responsibility by an authorized editor of JMU Scholarly
Commons. For more information, please contact [email protected].
International Journal on Responsibility 2024, Vol. X(X)

Artificial Intelligence Chatbot as a Mathematics Curriculum Developer: Discovering

Preservice Teachers’ Overconfidence in ChatGPT

Amanda G. Sawyer1

Abstract

Instructors in many colleges and universities are responsible for supporting their preservice teachers’
understanding of mathematics curriculum to best serve elementary students’ needs. As such, preservice
teachers are taught how to critically analyze curriculum materials. However, with the advent of
technologies like ChatGPT, preservice teachers are utilizing artificial intelligence (AI) tools in new ways
to collate and construct mathematical curricula. As of now, little is known about how to adapt these new
resources for the classroom. ChatGPT, an AI chatbot, can create a mathematics curriculum based on user
questions, which teachers can then adapt to their classrooms. Although this AI chatbot can produce a
quick response, researchers have identified that ChatGPT can produce biased responses and inaccurate
mathematical data. This study evaluates the quality of the mathematics curriculum created by ChatGPT’s
responses, how preservice teachers adapted those resources, and their perceptions of using ChatGPT.
Overall, ChatGPT’s responses tended to construct high levels of cognitive demand with age-inappropriate
text for elementary students. Despite formally teaching how to be critical of resources, the majority of
preservice teachers’ adaptations merely changed visual appeal alone, demonstrating how some preservice
teachers have overconfidence in the abilities of AI tools. This implies that preservice teachers should be
cautious of ChatGPT and be taught specific mathematical prompt engineering techniques to create
innovative tasks.

Keywords: artificial intelligence, mathematics curriculum developers, curriculum adaptation, large

language models, ChatGPT

Introduction

In the United States, teacher education programs in colleges and universities are required to
instruct their preservice teachers, or post-secondary students who aspire to become teachers, on specific
knowledge around mathematics content and mathematics pedagogy (National Council of Teachers of
Mathematics, 2000). The post-secondary instructors of these courses, known as Mathematics Teacher
Educators (MTEs), are responsible for teaching various concepts, from the meaning behind division
algorithms to selecting appropriate mathematical resources that will support the development of effective
classroom teachers in today’s schools. These skills, like adaptation of mathematics resources, are essential
for beginning teachers because many school districts require teachers to develop their own materials
(Dick, Sawyer, 2022). The advent of artificial intelligence (AI) technologies and tools, specifically large
language models (LLMs), adds a new facet to understanding mathematics curriculum. One must consider
the presence of AI tools when incorporating outside resources moving forward.
Although researchers investigated best practices around selecting and curating resources from
virtual resource pools (Dick et al., 2023; Shelton, Archambault, 2019), using AI tools in the classroom is

1 James Madison University, College of Education, Harrisonburg, Virginia 22802, USA; [email protected]

Published by Scholarly Commons, James Madison University. https://commons.lib.jmu.edu/ijr

 Sawyer / International Journal on Responsibility, Vol. X (2024)

so new that this area has yet to be explored in the context of the mathematics curriculum. In this
exploratory study, I investigated the construction of mathematics curriculum materials using the resource
called Chat Generative Pre-Trained Transformer (ChatGPT) 3.5. I examined how preservice teachers in
a mathematics methods course adapted AI tool’s resources, and what can be learned from this process.
Below, I describe what is known about AI tools and effective mathematics education instruction, and I
describe who is responsible for the errors of AI tools in mathematics classrooms. Lastly, I explain what is
known about mathematics curriculum research and how teachers can adapt mathematics resources to
situate this investigation into the known literature in the field.

Responsibility and Artificial Intelligence

ChatGPT, a chatbot AI program, answers users’ questions by providing human-like text

responses based on its training (OpenAI, 2024). As seen in Figure 1, individuals can ask ChatGPT to
create mathematics curriculum resources around a standard, and it provides instant results that can be
used in the classroom. Other AI chatbots can produce similar texts like Google’s Gemini
(https://gemini.google.com/), Claude (https://claude.ai), and Jasper (https://www.jasper.ai/). However,
ChatGPT was selected for this investigation because of its observed popularity among college students.

Question to ChatGPT: Create an engaging worksheet for CCSS.MATH.CONTENT.2.NBT.A.1.

ChatGPT’s response: Creating an engaging worksheet for CCSS.MATH.CONTENT.2.NBT.A.1

(understanding the value of each digit in a two-digit number) can involve fun, visually appealing activities.
Here’s a worksheet that includes engaging content and clear instructions:

**Worksheet: Exploring Two-Digit Numbers**

**Name: ___________________________** **Date: ______________**

**Instructions**:
1. Study each two-digit number and answer the questions below.
2. Pay attention to the value of each digit in the number.
3. Have fun exploring the numbers!

**Questions**:

1. **Number:** 47
- What is the value of the digit 4?
- What is the value of the digit 7?

2. **Number:** 65
- What is the value of the digit 6?
- What is the value of the digit 5?

3. **Number:** 28
- What is the value of the digit 2?
- What is the value of the digit 8?

https://commons.lib.jmu.edu/ijr 2
 Sawyer / International Journal on Responsibility, Vol. X (2024)

4. **Number:** 36
- What is the value of the digit 3?
- What is the value of the digit 6?

5. **Number:** 53
- What is the value of the digit 5?
- What is the value of the digit 3?

**Visual Exercise**:

Look at the two-digit number 49. Color the tens digit (4) in blue and the ones digit (9) in red.

**Number Fun**:

6. **Number:** 72
- Write a short story about the number 72, explaining what each digit represents.

**Challenge**:

7. **Number:** 84
- Can you find any other two-digit numbers where the value of the tens digit is greater than the
value of the ones digit? If so, write them down.
---
This engaging worksheet combines questions about the value of each digit in two-digit numbers with a visual
exercise, a creative writing prompt, and a challenge to explore more numbers. It makes learning about two-digit
numbers interactive and enjoyable for students.

Figure 1. Example of GenAI tool Generated Mathematics Curriculum resources (ChatGPT 3.5, October 11, 2023)

ChatGPT is programmed to learn from its users. AI chatbot explained, “Your input guides my
subsequent responses. If you provide more context or ask follow-up questions, I incorporate that
information to provide you with a more tailored and relevant response” (OpenAI, 2024). Therefore,
teachers can manipulate AI’s construction of mathematics curriculum resources through their
questioning. The process of crafting the users’ text to be understood by the generative AI program is
called prompt engineering (Korzynski et al., 2023). When prompting for ChatGPT, Korzynski et al.
(2023) explained users should include context, instructions, input data, and expected output format;
otherwise, the generative AI would create its own interpretation of the question.
When implementing technology tools in the classroom, teachers are responsible for technoethical
considerations (Krutka et al., 2019). For example, Krutka et al. (2019) suggested teachers should consider
if the tool was designed ethically. Currently, multiple lawsuits have been filed claiming OpenAI trained
ChatGPT from copyrighted materials without consent. Thus, ChatGPT could be plagiarizing materials
to produce its answers (Cho, 2023). Krutka et al. (2019) also had teachers consider if technology “afford[s]
or constrain[s] democracy and justice for all people and groups” (p. 570). Researchers found ChatGPT
can generate “terrible answers that discriminate against gender, race, and minority groups” (Wu, 2023:
25), meaning ChatGPT can perpetuate algorithmic unfairness with biased information (Tasioulas, 2022).

https://commons.lib.jmu.edu/ijr 3
 Sawyer / International Journal on Responsibility, Vol. X (2024)

Considering these statements poses the question: should ChatGPT be used in education?
The United States Department of Education (2023) and the National Council of Teachers of
Mathematics (NCTM) (2024) both provided a position statement saying teacher education needs to
research these generative AI tools despite the technology's current limitations because of its possible
potential in education. They described how using the tool cautiously can enhance and inspire new
educational concepts (USDOE, 2023), but who is morally responsible for inaccuracies and their effect on
elementary students? I argue that the responsibility lies with MTEs and teachers implementing the
curriculum. MTEs must teach preservice teachers and notify inservice teachers about inherent issues with
this technology. Teachers are responsible for vetting and critically curating materials they implement in
their classrooms. Since researchers identified AI tools could take away individual autonomy and restrict
human agency (Coeckelbergh, 2023; Farina et al., 2022), MTEs can take back control by making our
teachers aware of the tool’s inaccuracies. Therefore, transparent processes are needed to create and use
AI tools to support and understand each other. As Coeckelbergh (2023) stated, MTEs have “the
responsibility to make sense, to interpret, and to narrate” (p. 2442). Therefore, this manuscript narrates
and observes how preservice teachers use this technology, so MTEs can better understand how to support
future educators.
AI technology has almost grown enough to mimic educational content, but discretion is still
needed if the technology cannot produce the intrinsic human element in genuine content. Akgun and
Greenhow (2022: 438) identified that “educating future generations of diverse citizens to participate in
the ethical use and development of AI tools will require more professional development for K-12 teachers.”
This is where MTEs are needed: firstly, to identify and contextualize where and how teachers need to be
critical, and secondly, to teach this criticality to their preservice teachers.

Responsibility of Mathematics Education in the United States

MTEs in the United States are responsible for teaching specific mathematics standards to
preservice teachers in their mathematics methods and content courses. Created by national teaching
organizations and reviewed by national accreditation programs, each state has laws regulating the process
to “ensure every classroom has a knowledgeable, engaged, and supported teacher” (VDOE, 2024: 1). The
elementary mathematics methods course implementing the investigation into AI tools was strictly
governed to ensure state regulations were met to support Teacher Licensure. In the United States, an
individual must apply and receive teacher licensure approved by their state department of education to be
a certified teacher. One way to qualify for Teacher Licensure was “to complete an approved Teacher
Education Preparation Program” (VDOE, 2024: 2). For an institute to be approved, the college or
university must obtain and maintain national accreditation from the Council for Accreditation of Educator
Preparation (CAEP) approximately every ten years (Virginia Law, 2024). During accreditation, CAEP
reviewers read narratives and observe syllabi from each course the Teacher Education Program offers to
determine if national standards have been met.
Professional standards for MTEs are constructed by the Association of Mathematics Teacher
Educators (AMTE, 2017). These standards describe the expected mathematics content, practices,
curriculum, and pedagogical knowledge taught in the college classroom. Dispositions towards teaching
mathematics are also addressed, including how to support elementary students as learners and integrate
social contexts within mathematics content. Each standard has subsections that MTEs must meet. For
example, AMTE’s Standard 1 states:

https://commons.lib.jmu.edu/ijr 4
 Sawyer / International Journal on Responsibility, Vol. X (2024)

Well-prepared beginning mathematics teachers possess robust knowledge of mathematical and

statistical concepts that underlie what they encounter in teaching. They engage in appropriate
mathematical and statistical practices and support their students in doing the same. They can
read, analyze, and discuss curriculum, assessment, standards documents, and students’
mathematical productions (AMTE, 2017: 4).

Specifically, it requires MTEs to have their preservice teachers analyze the mathematical
curriculum (C.1.4), plan for effective instruction (C.2.2), and use technology to teach mathematics (C.1.6)
effectively (AMTE, 2017). Another set of standards which can be used for the accreditation process is
constructed by NCTM (NCTM, 2000). These principles describe what excellent mathematics programs
can do to help support:
1. teaching and learning;
2. access and equity;
3. curriculum;
4. tools and technology;
5. assessment; and
6. professionalism (NCTM, 2014).

In particular, NCTM stated, “An excellent mathematics program integrates mathematical tools
and technology as essential resources to help students learn and make sense of mathematical ideas, reason
mathematically, and communicate their mathematical thinking” (NCTM, 2014: 78).
While both standards were constructed before 2014, the effective use of AI technology in
elementary classrooms must be addressed, especially given NCTM’s (2024) AI position statement. It is
the responsibility of MTEs to influence future mathematics students through responsive post-secondary
instruction. Thus, everything implemented in the investigated course was specifically designed to help
improve future mathematics classrooms, specifically lesson curation and adaptation.

Lesson Curation and Adaptation

According to Remillard (1999: 318), teachers’ lesson planning process can be viewed as
“curriculum development” because of the ways teachers take materials and “alter, adapt, or translate” them
for their own classrooms. Lesson adaptation includes insertions, deletions, or substitutions of curricular
materials by teachers to support learning (Davis et al., 2011). When deciding what elements to adapt,
Lederman and Gess-Newsome (1991) found that preservice science teachers focused their lesson
adaptations on two significant concerns: self and student concerns. These preservice teachers described
student-related concerns of student involvement, use of questioning, instructional planning, and
motivation. In contrast, their concerns for self-included more internal issues like self-confidence, board-
work, workload, and physical appearance. These results show different factors teachers bring to the lesson
adaptation process.
Davis et al. (2011) studied two elementary teachers’ adaptation processes and found their
adaptations were influenced by their learning goals and knowledge of their students. They called for
teachers to help design curricula because of the insights they can provide regarding the specific needs of
their students. Despite this positive finding, teachers’ adaptations do not always benefit students (Brown,
2009; Davis et al., 2011). Grossman and Thompson (2008) conducted a longitudinal study on beginning

https://commons.lib.jmu.edu/ijr 5
 Sawyer / International Journal on Responsibility, Vol. X (2024)

English teachers’ use of curriculum materials. They found that new teachers typically selected items that
were easily accessible to them and tended not to adapt materials. However, these same teachers
transitioned towards adapting materials over time. They hypothesized that these initial difficulties in
adapting materials likely stemmed from inadequate instruction during teacher preparation. They called
on teacher education programs to provide more support in preparing teachers to adapt materials
(Grossman, Thompson, 2008).
Researchers have found that some teachers neglect adaptation altogether for fear that their
adaptations will not align with standards (Ding, Carlson, 2013). Consistent with a lack of teacher self-
confidence, a study by Sawyer et al. (2020) revealed that 26 of 72 preservice teachers (PSTs) used materials
in the exact format received from their cooperating teachers. The researchers suggested their lack of
adaptation “shows PSTs seeing themselves as technicians, following directions, and not yet professionals,
making the difficult curating, synthesizing, and adaptations necessary for master teaching” (Sawyer et al.,
2020: 15). Across the literature, researchers have called for additional support for teachers in learning
how to adapt materials for their classroom appropriately (Davis et al., 2011; Grossman, Thompson, 2008;
Sawyer et al., 2020).
Adaptation is also a part of teachers’ process to critically curate the resources they use in the
classroom (Sawyer et al., 2020). Critical Curation Theory (CCT) describes how individuals select and
curate lesson ideas when planning lessons (Sawyer et al., 2020). CCT, as seen in Figure 2, is an open cycle,
meaning individuals can begin the lesson-planning process at any point. In CCT, each cycle element works
together, and curation only occurs once the teacher has a lesson purpose and a classroom resource. Once
a lesson resource has been curated, the adaptation and synthesis process can begin (Sawyer et al., 2020).

Figure 2: Critical Curation Cycle (Sawyer et al., 2020)

https://commons.lib.jmu.edu/ijr 6
 Sawyer / International Journal on Responsibility, Vol. X (2024)

Adaptation in Mathematics

Methods of adaptation vary across elementary school subjects; researchers have investigated
methods of adaptation in reading (e.g., Parsons, 2012), science (e.g., Forbes, 2011), as well as the focus of
this study, mathematics (e.g., Choppin, 2011; Remillard, 1999; Sherin, Drake, 2009). Sherin and Drake
(2009) found mathematics teachers typically adapt their resources in a three-step cycle: omit, replace, and
create. They explain that teachers remove items from a mathematics activity and then replace them with
something they find helpful and believe will better support their students’ learning. If this is not deemed
sufficient, mathematics teachers create new curricular elements to support student learning in their
classroom. In delving into the details of mathematics content adaptations, Choppin (2011: 185) found
when teachers adapt mathematics resources, they often: “Adapted tasks in ways that reduced their
complexity by narrowing the choices available for students and minimizing opportunities for students to
make connections by explaining their strategies and reflecting on other students’ strategies.”
Choppin (2011) also found teachers’ adaptations were influenced by their noticing of students’
thinking and their beliefs about teaching. If teachers only evaluated student thinking for correctness, they
tended to adapt materials by reducing students’ cognitive struggle. If teachers tended to focus on
interpreting details of student thinking, they often adapted materials to consider cognitive trajectory with
a particular emphasis on their mathematical understanding (Choppin, 2011).

Adaptations of Online Mathematics Resources

Adapting resources for mathematics classrooms involves more than just the document; it also
involves where the teachers gained their inspiration, and researchers have identified a growing trend of
individuals selecting materials from trending websites (Sawyer et al., 2020). According to the Bill and
Melinda Gates Foundation (2014), 91% of teachers utilize online resources to help plan their lessons.
Using a sample of 601 elementary mathematics teachers, a survey conducted by Shapiro et al. (2019)
revealed, on average, that 64% of teachers search online weekly for elementary mathematics activities.
Due to the vastly changing landscape of online websites, differences are seen in reports about the
popularity of sites. In 2014, among Pre-K through 5th-grade teachers, the most frequently used sites
were: YouTube (76%), Discovery.com (59%), Scholastic.com (56%), PBS.org (55%), and Pinterest (46%)
(Bill and Melinda Gates Foundation, 2014). Shapiro et al. (2019) found 89% of elementary teachers use
Teachers Pay Teachers, followed by 74% using Pinterest.
Researchers have evaluated the quality of Pinterest pins (Sawyer et al., 2019) and Teachers Pay
Teachers resources (Dick et al., 2019, Dick et al., 2023; Shelton, Archambault, 2019). To determine the
quality of the materials, Dick et al. (2023) categorized the tasks using Stein and Smith’s (1998) Task
Analysis Guide as either Memorization, Procedures Without Connections, Procedures With Connections,
or Doing Mathematics. Stein and Smith (2007) explained that mathematics instruction should include an
even distribution of the different levels. Still, they found that teachers typically lower the levels of
cognitive demand when implementing tasks. As seen in Figure 3, Dick et al. (2019) found less than one
percent of the coded elementary mathematics resources demanded the highest level, Doing Mathematics.
In both Teachers Pay Teachers and Pinterest, they found around 60% of the top 500 free mathematics
activities on these sites required low levels of cognitive demand, being coded as either Memorization or
Procedures Without Connections (Dick et al., 2019; Sawyer et al., 2019). However, all of this is to say
MTEs do not know the level of cognitive demand for resources constructed by AI.
Given the low level of cognitive demand in many online mathematics tasks, the number of

https://commons.lib.jmu.edu/ijr 7
 Sawyer / International Journal on Responsibility, Vol. X (2024)

teachers searching for elementary mathematics resources when planning lessons is concerning. Even
more problematic are Grossman and Thompson’s (2008: 2025) findings, in which new teachers may “latch
onto curriculum materials uncritically.” One of the most effective ways to help preservice teachers think
critically about materials is by engaging them in lesson planning and adaptation (Chai et al., 2010; Colvin,
Tomayko, 2015). Currently, MTEs are attempting to embed critical curation skills into their mathematics
education curriculum (Sawyer et al., 2020). Therefore, Dick and Sawyer (2022) investigated how teachers
adapted resources found from online websites. They created the Mathematical Adaptation Framework,
identifying how teachers categorized adapting online resources into four areas: adapted for different
learners’ needs, adapted for classroom implementation, adapted for mathematics content, and adapted for
visual appeal. They found that 64% of individuals adapted for different learners’ needs, 45% adapted for
classroom implementation, 27% adapted for mathematical content, and 7.4% adapted for visual appeal
(Dick, Sawyer, 2022). In this investigation, I used Dick and Sawyer’s (2022) Mathematics Adaptation
Framework to examine how individuals adapt using another GenAI tool, ChatGPT 3.5, to determine if
the adaptation process was similar.

Figure 3: Level of Cognitive Demand of Virtual Resource Pools

https://commons.lib.jmu.edu/ijr 8
 Sawyer / International Journal on Responsibility, Vol. X (2024)

Current Study

Upon receipt of Institutional Review Board (IRB) approval, I investigated a group of 22 preservice
teachers from a single elementary mathematics methods course in a Mid-Atlantic university to determine
how they use ChatGPT to Critically Curate mathematics curriculum resources. The study asked the
following research questions:

1. What is the level of cognitive demand of ChatGPT’s responses and preservice teachers’
adaptations, and what differences were observed between the resources?
2. How do preservice teachers adapt mathematics resources from ChatGPT?
3. What are preservice teachers’ perspectives on using ChatGPT in their future classrooms?

Context

In preparing the preservice teachers to complete the task of using ChatGPT to create a
mathematics curriculum, a unit was designed and implemented on Critical Curation, or the thoughtful
selection of resources based on one's pedagogical knowledge, content knowledge, personal experiences,
and purpose of the lesson (Sawyer et al., 2020). As seen in Figure 4, MTEs support this process by having
preservice teachers contemplate specific questions while constructing their resources. In that unit, various
ways teachers can adapt their resources were also taught using Dick and Sawyer’s (2022) Mathematical
Adaptation Framework, which describes the following four categories:

1. Adapting for different learners' needs

2. Adapting for classroom implementation
3. Adapting for the mathematical content
4. Adapting for visual appeal.

Figure 4. Questions to Consider to Critically Curate Mathematical Resources

https://commons.lib.jmu.edu/ijr 9
 Sawyer / International Journal on Responsibility, Vol. X (2024)

Lastly, in the unit, the class explored the Critical Curation Cycle (Sawyer et al., 2020) and
demonstrated how to conduct this process. For example, the preservice teachers were asked to create a
lesson goal and find resources by using their textbook or by doing an internet search. Next, the preservice
teachers discussed with their classmates what was found and why it could or could not be implemented
in its current form. This helped demonstrate the process of finding resources using multiple different
methods. Finally, the preservice teachers reflected on how they could adapt the selected resources to
support all their elementary students’ learning needs.

Methods

Below, I describe the participants in the study, how I collected the data using a Qualtrics Survey,
and finally, how I conducted a thematic analysis (Braun, Clake, 2012) to investigate preservice teachers’
use of AI tools to answer the research questions.

Participants and Data Collection

Participants in this study were 22 elementary preservice teachers enrolled in an elementary

mathematics methods course. These university students were all female between the ages of 20 and 21 in
their junior year of their Elementary Education Major. Data for this study includes artifacts from a
classroom activity in an elementary methods course and responses from a Qualtrics survey (Qualtrics,
2019). After completing the Critical Curation unit, preservice teachers were asked to complete the
following steps:

Step 1: Find a mathematics state standard they would like to implement in an elementary
classroom.

Step 2: Determine what goal they want to accomplish for the lesson based on the standard.

Step 3: Ask ChatGPT to create a mathematical task to support that goal.

Step 4: Adapt ChatGPT’s task to create a resource they would implement in an elementary
mathematics classroom.

All preservice teachers in the class agreed to participate in the investigation, and their adapted
resources were analyzed for this project. The preservice teachers were allowed to work in groups to create
their adapted resources. Thus, 15 ChatGPT responses and adaptations were collected. One group’s file
was corrupt. Therefore, only 14 adapted resources were analyzed. While completing the prompt, all
individuals were asked to fill out a Qualtrics survey answering the following questions about their work:

1. What standard did you select to implement for this activity?

2. What was the goal of your mathematical task?
3. Please upload a screenshot of ChatGPT’s response.
4. Please upload the final task you created in class based on ChatGPT’s response.
5. Identify how you adapted the ChaptGPT’s response to become your final product.

https://commons.lib.jmu.edu/ijr 10
 Sawyer / International Journal on Responsibility, Vol. X (2024)

6. What are your perspectives on using ChatGPT in future classrooms?

Data Analysis

To determine the level of cognitive demand of the task created by ChatGPT and the preservice
teachers’ adapted task, thematic analysis (Braun, Clake, 2012) was used with Stein and Smith’s (1998)
Task Analysis Guide (TAG) framework. I used the TAG framework to allow the AI chatbot’s constructed
data to be compared to other research in mathematics curriculum development. To conduct the thematic
analysis, the data was first reviewed for familiarity and then coded for specific elements in the tasks. For
example, the term “multiple representations” was used for a code to identify when the students were asked
to display their mathematics in multiple ways. Once the codes were constructed, they were placed into
themes defined using the TAG framework. Stein and Smith’s (1998) TAG framework included four
categories: Memorization, Procedures Without Connections, Procedures With Connections, and Doing
Mathematics. The data was identified as a Memorization task if the resource was unambiguously asking
elementary students to learn facts unrelated to mathematical meaning. Tasks categorized as the next
level, Procedures Without Connections, were algorithmic in nature without connection to understanding
the mathematical concept. Data was identified as Procedures With Connections if the resource focused on
following an algorithm but had students express their understanding behind the mathematical concept.
Tasks were categorized as Doing Mathematics if they required complex and non-algorithmic thinking
with self-monitoring regulations.
Each resource created by ChatGPT 3.5 included multiple questions, each with its own level of
cognitive demand. Thus, the complete resource was only identified as the maximum level of cognitive
demand. If one activity in the worksheet could be categorized as Doing Mathematics, the entire document
would be classified as Doing Mathematics. For example, Figure 5 has ChatGPT’s response to a preservice
teacher’s question about making a worksheet for kindergarten students decomposing numbers. I
categorized this activity as Procedures With Connections because elementary students make
mathematical connections between multiple constructions of numbers.

ChatGPT’s Response: Certainly! Here’s a kindergarten worksheet to help students practice composing and
decomposing numbers less than or equal to 5 in multiple ways:

**Title: Composing and Decomposing Numbers**

**Instructions:**

1. Look at the number in the star. 2. Use objects, drawings, words, or numbers to show how to make that
number in different ways. 3. Fill in the blanks to complete each set of equations.

**Worksheet:**
1. **Number: 3** Ways to make 3:
- Objects: _____ + _____
- Drawings: _____ + _____
- Words: _____ + _____
- Numbers: _____ + _____
Complete the equations: -1 + _____ = 3 - 2 + _____ = 3 - _____ + 1 = 3 - _____ + 2 = 3

2. **Number: 4** Ways to make 4:

https://commons.lib.jmu.edu/ijr 11
 Sawyer / International Journal on Responsibility, Vol. X (2024)

- Objects: _____ + _____

- Drawings: _____ + _____
- Words: _____ + _____
- Numbers: _____ + _____
Complete the equations: -1 + _____ = 4 - 2 + _____ = 4 - 3 + _____ = 4 - _____ + 1 = 4 - _____ + 2 =
4- _____ + 3 = 4

3. **Number: 2** Ways to make 2:

- Objects: _____ + _____
- Drawings: _____ + _____
- Words: _____ + _____
- Numbers: _____ + _____
Complete the equations: -1 + _____ = 2 - _____ + 1 = 2

4. **Number: 5** Ways to make 5:

- Objects: _____ + _____
- Drawings: _____ + _____
- Words: _____ + _____
- Numbers: _____ + _____
Complete the equations: -1 + _____ = 5 - 2 + _____ = 5 - 3 + _____ = 5 - 4 + _____ = 5 - _____ + 1 =
5 - _____ + 2 = 5- _____ + 3 = 5- _____ + 4 = 5

5. **Number: 1** Ways to make 1:

- Objects: _____
- Drawings: _____
- Words: _____
- Numbers: _____
Complete the equations: - _____ + 1 = 1

6. **Number: 0** Ways to make 0:

- Objects: _____
- Drawings: _____
- Words: _____
- Numbers: _____
Complete the equations: - _____ + 0 = 0

**Bonus Challenge:** Choose a number from 1 to 5 and create your own ways to compose and decompose it!
Have fun learning and exploring numbers!

Figure 5. Example of Procedures With Connections ChatGPT 3.5 Response

To investigate the adaptation, I used a thematic analysis (Braun, Clake, 2012) with Dick and
Sawyer’s (2022) Mathematics Adaptation Framework to allow the field to see if preservice teachers use
AI chatbot curriculum materials like online resources. The data was first reviewed to conduct this
thematic analysis, and then preservice teachers’ adaptations were compared to ChatGPT’s responses.
Specific codes were constructed to identify what changed, as seen in Table 1. After the codes were created,
they were placed in one of the themes identified by Dick and Sawyer’s (2022) Mathematics Adaptation
Framework. After coding the data, I discovered that all the data fit one of the previously defined
categories; thus, no new categories were added to the framework.

https://commons.lib.jmu.edu/ijr 12
 Sawyer / International Journal on Responsibility, Vol. X (2024)

Table 1. Adaptation categories description and open coding keywords (Dick, Sawyer, 2022)

Adaptation Category Open Coding Keywords Description

Adapted for Different Learners English Language Learners The teachers adapted the activities
Needs to meet different learner needs.
Special Education
Level of Difficulty:
Easy/Hard/Difficult/Levels/
Rigor/Advance/Struggle
Differentiation
Scaffold
Adapted for Classroom Manipulatives The teachers adapted the activities
Implementation Directions by adding elements or taking
Time elements away by altering the
Structure implementation of the activity in
Independent/Group/Partner their classroom.
/One-on-One/Centers
Technology
Video/PowerPoint/iPad/Smart
Board/Computer/Tech
Adapted for Mathematics Content Standards The teachers adapted the activities
Common Core/District/State by adding elements or taking
Numbers elements away by altering the
Content mathematical content.
Alignment
Curriculum
Adapted for Format The teachers adapted the visual
Visual Appeal Font aesthetics of the activity.
Design
Size
Picture/Image
Visual

Finally, thematic analysis (Braun, Clake, 2012) was also used with open coding to understand the
preservice teachers’ perspectives on using ChatGPT in their future classrooms. The analysis process
involved reviewing the survey responses multiple times to get acquainted with major patterns, meanings,
and discrepancies. Codes like Cool but Scary, Saves Time, More Details Needed, and Old-School were
constructed. “Cool but Scary” referred to preservice teachers who identified hesitance in the technology
despite acknowledging its unique qualities. “Saves Time” referred to preservice teachers’ belief that the
AI tools would help speed up the construction of mathematics curriculum materials. “More Details
Needed” referred to the preservice teachers’ belief that the responses from the AI tool were vague and did
not provide everything they believed it would need for the classroom. Finally, “Old-School” referred to
the preservice teachers’ view that the AI tool created more traditional resources. The codes were
subdivided into two major themes: Useful Tool (preservice teachers say the value in the tool) and Caution
(preservice teachers believed the AI tool could be misused). All responses fit within the identified two
themes.

Findings

From the preservice teachers’ responses to the survey, I categorized the level of cognitive demand
of ChatGPT’s responses and the preservice teachers’ adaptation to find that ChatGPT’s responses

https://commons.lib.jmu.edu/ijr 13
 Sawyer / International Journal on Responsibility, Vol. X (2024)

represented higher levels of cognitive demand. Still, when the preservice teachers manipulated the
resources, they tended to lower the level of cognitive demand. I also found that preservice teachers
adapted resources from ChatGPT for visual appeal, with the majority of the participants copying the
language verbatim from the chatbot. Finally, I found that the preservice teachers perceived the AI chatbot
as a useful tool but could identify where it could be misused.

Figure 6. Level of Cognitive Demand of ChatGPT 3.5 Responses and Preservice Teachers’ Adaptations

https://commons.lib.jmu.edu/ijr 14
 Sawyer / International Journal on Responsibility, Vol. X (2024)

Levels of Cognitive Demand

Figure 6 shows five of ChatGPT’s responses were considered Procedures Without Connections,
two were Memorization, and eight were Procedures With Connections. ChatGPT’s text responses, which
were Procedures With Connections, like the example seen in Figure 5, asked elementary students to draw
pictures to show their work or explain their mathematical reasoning. Fifty-three percent of ChatGPT’s
responses included a connection to mathematics. Still, none of ChatGPT’s responses could be considered
a Doing Mathematics activity, which is the highest level of cognitive demand. ChatGPT’s responses all
correspond to the preservice teacher’s question; thus, these responses were subject to what was asked by
users. However, when preservice teachers created their adapted resources from ChatGPT’s responses,
only six resources stayed Procedures With Connections, one was Memorization, and seven were
categorized as Procedures Without Connections. This identified that ChatGPT’s responses were changed
to either increase or decrease the activity’s cognitive demand level.
As seen in Table 2, the first column represents the initial level of cognitive demand of the resource
created by ChatGPT, and the second column represents the preservice teacher’s adapted resource’s level
of cognitive demand. The third column represents the frequency of each change. For example, after the
adaptations were constructed, five resources created by ChatGPT’s responses were categorized as
Procedures With Connections and continued to be Procedures With Connections. Four resources stayed
Procedures Without Connections, while only one stayed Memorization through adaptation.

Table 2. Change in Level of Cognitive Demand of ChatGPT’s Responses to Preservice Teachers’ Adaptations.

Level of Cognitive Demand of Level of Cognitive Demand of Frequency

ChatGPT’s Response Preservice Teachers’ Adaptation
Procedures With Connections Procedures With Connections 5 (33.3%)

Procedures Without Connections 3 (20%)

Procedures Without Connections Procedures With Connections 1 (6.7%)

Procedures Without Connections 4 (26.7%)

Memorization Memorization 1 (6.7%)

N/A 1 (6.7%)

As seen in Figure 7, ten of the resources stayed at the same level of cognitive demand. Three
preservice teachers’ adapted resources decreased in their level of cognitive demand from ChatGPT’s
response. The decreased documents started as Procedures With Connections but changed to Procedures
Without Connections. This came from removing pictorial representations or explanations of thinking, as
was initially provided by ChatGPT. One group updated its resource to increase cognitive demand from
Procedures Without Connections to Procedures With Connections. One adapted resource from
ChatGPT’s response, categorized as Memorization, could not be classified because the preservice teachers’
file was corrupt.

https://commons.lib.jmu.edu/ijr 15
 Sawyer / International Journal on Responsibility, Vol. X (2024)

Figure 7. Change in Levels of Cognitive Demand

Adaptation Created by Preservice Teachers

I also investigated the preservice teachers’ adaptations of ChatGPT’s responses using Dick and
Sawyer’s (2022) Mathematical Adaptation Framework. Thus, changes were categorized as adapting for
different learners’ needs, adapting for classroom implementation, adapting for mathematical content, or
adapting for visual appeal. I found that all preservice teachers adapted their resources in at least one area,
as seen in Figure 8, with some resources categorized in multiple places.

Figure 8. Adaptations of ChatGPT 3.5 Responses

https://commons.lib.jmu.edu/ijr 16
 Sawyer / International Journal on Responsibility, Vol. X (2024)

In all 14 resources created by preservice teachers, an adaptation for visual appeal was evident.
Since ChatGPT provides text responses, some elements had to be removed or reformatted. All preservice
teachers showed an understanding of this in their adaptation.
Three resources were identified to be adapted for different learners’ needs. For example, the
language in these resources was changed or added to have the elementary students describe their thinking.
All three of these resources also correlated with the three resources that increased the level of cognitive
demand on Procedures With Connections, as seen above.
Also, three of the resources changed the mathematical content. For example, as seen in Figure 9,
one group had elementary students using their own constructed patterns to understand the beginnings
of Algebra. Another group had to add coin values to assess understanding of currency because ChatGPT's
response provided a broken link to an image, as seen in Figure 1. Two groups created completely different
activities than what was constructed by ChatGPT's responses and, therefore, changed the mathematical
content with the visual appeal.

https://commons.lib.jmu.edu/ijr 17
 Sawyer / International Journal on Responsibility, Vol. X (2024)

Figure 9. Example of Preservice Teacher Resource

https://commons.lib.jmu.edu/ijr 18
 Sawyer / International Journal on Responsibility, Vol. X (2024)

A surprising finding was that nine of the 14 resources created by the preservice teachers only
changed their visual appeal. As seen in Figure 10, ChatGPT’s responses were formatted into a worksheet,
but nothing else was changed. Therefore, 64% of all resources constructed in this activity were taken
almost verbatim from ChatGPT.

ChatGPT 3.5 Response Preservice Teachers’ Adaptation

Question to ChatGPT: create a worksheet for a Understanding Place Value
fourth grade class understanding place value with
nine-digit whole numbers Name:_____________________ Date: ____________

ChatGPT’s Response: Sure, here's a worksheet to Instructions:

help fourth-grade students understand place value 1. Write the value of the digit in the italicized place
with nine-digit whole numbers: **Worksheet: value.
Understanding Place Value with Nine-Digit Whole 2. Express the number in expanded form.
Numbers** **Name: ____________ Date: _____ 3. Compare the numbers using greater than, less than,
** **Instructions:** 1. Write the value of the digit in or equal symbols (>,<,=)
each underlined place. 2. Express the number in
expanded form. 3. Compare the numbers. Example:
**Example:** Number: 4,857,926,301 - Value of the Number: 4,857,926,301
digit in the millions place: 4 - Expanded form: - Value of the digit in the millions place: 4
4,000,000,000 + 800,000 + 50,000 + 7,000 + 900 + 20 - Expanded form: 4,000,000,000 + 800,000 + 50,000 +
+ 6 + 3 + 1 - Comparing with 5,857,926,301: 7,000 + 900 + 20 + 6 + 3 + 1
________ (circle the correct symbol: <, >, or =) - 4,857,926,301 __<____ 5,678,237,903
**1. Number: 9,324,567,812** - Value of the digit in
the tens place: ________ - Expanded form: 9,324,567,812
______________- Comparing with 9,324,567,819: - Value of the digit in the tens place: ________
________ - Expanded form: __________________________
**2. Number: 1,087,654,239** - Value of the digit in - 9,324,567,812 _______ 9,324,567,819
the hundred millions place: ________ - Expanded
form: _____________ - Comparing with 1,087,654,239
1,087,654,238: ________ - Value of the digit in the hundred millions place:
**3. Number: 6,209,384,571** - Value of the digit in ________
the thousands place: ________ - Expanded form: - Expanded form: __________________________
______________ - Comparing with 6,209,384,578: - 1,087,654,239 _____ 1,087,654,238
________
**4. Number: 2,567,890,432** - Value of the digit in 6,209,384,571
the ten millions place: ________ - Expanded form: - Value of the digit in the thousands place: ________
______________- Comparing with 2,567,890,431: - Expanded form: __________________________
________ - 6,209,384,571 ____ 6,209,384,578
**5. Number: 3,410,928,765** - Value of the digit in
the ones place: ________ - Expanded form: 2,567,890,432
____________ - Comparing with 3,410,928,767: - Value of the digit in the ten millions place: ________
________ - Expanded form: __________________________
**6. Number: 8,123,456,789** - Value of the digit in - 2,567,890,432 _____ 2,567,890,431
the billions place: ________ - Expanded form:
_____________ - Comparing with 8,123,456,788: 3,410,928,765
________ - Value of the digit in the ones place: ________
**7. Number: 5,901,234,567** - Value of the digit in - Expanded form: __________________________
the hundred thousands place: ________ - Expanded - 3,410,928,765 _____ 3,410,928,767
form: ____________ - Comparing with 5,901,234,569:
________ 8,123,456,789
**8. Number: 7,654,321,089** - Value of the digit in - Value of the digit in the billions place: ________
the millions place: ________ - Expanded form: - Expanded form: __________________________
____________ - Comparing with 7,654,321,090: - 8,123,456,788 _______ 8,123,456,789
________

https://commons.lib.jmu.edu/ijr 19
 Sawyer / International Journal on Responsibility, Vol. X (2024)

**9. Number: 4,500,000,001** - Value of the digit in Bonus Question:

the billions place: ________ - Expanded form: -Write any nine-digit number: ________________
____________- Comparing with 4,500,000,002: - Value of the digit in the millions place: ________
________ - Expanded form: __________________________
**10. Number: 9,876,543,210** - Value of the digit in - ____________ ______ 9,999,999,999
the ten thousands place: ________ - Expanded form: (your number here) (symbol)
____________ - Comparing with 9,876,543,210:
________

**Bonus Question:** Write a nine-digit number of

your choice, then: - Value of the digit in the millions
place: ________ - Expanded form: ______________ -
Compare it with 9,999,999,999: ________ Remember
to show your work and circle the correct comparison
symbol (<, >, or =) when comparing numbers. Good
luck!
Figure 10. Example of Preservice Teacher Adaptation Similar to ChatGPT 3.5 response

Perceptions of ChatGPT

When I asked preservice teachers to give their perspectives on using ChatGPT in the elementary
classroom, two main themes emerged: Useful Tool and Caution. Preservice teachers identified positive
and negative aspects of using GenAI tools in the classroom.

Useful Tool. Preservice teachers found using ChatGPT as a mathematical curriculum developer “useful
and helpful” (Participant O). With 13 responses, preservice teachers saw ChatGPT as an inspirational
tool. They viewed ChatGPT as “a very helpful resource that teachers can use” (Participant F) because it
could help them create lessons and “inspire new problems for concepts” (Participant E). They identified
appreciating a tool which could consider the mathematics concept and develop activities to go along with
it, stating, “It's like a calculator for math resources” (Participant P) and “We didn't have to think to make
it” (Participant C).
ChatGPT was useful because it provided quick responses, helping teachers save time. Two
participants stated, “ChatGPT can be helpful in quickly finding worksheets and resources that can be
modified for our classroom” (Participant J) and, “I think it’s a really good resource, especially when you
are in a time crunch” (Participant M). These preservice teachers are aware of time limitations as a teacher,
and they viewed ChatGPT as a helpful time saver.

Caution. Preservice teachers ( n = 11) also identified the need for caution when using these tools. They
believed the technology was scary, needed more details, and could be viewed as “Old-School.” For
example, one participant wrote:

I think that it has the potential to greatly benefit teachers. For example, recently on social
media, I have come across younger teachers creating rubrics for the assignments through
ChatGPT. However, I worry that too many young teachers may see this as an easy way
around dedication and hard work which is why I don't mind the fact that I needed to
create my own patterns in the document GPT semi-created. It ensures that the teacher is
still adjusting the content to best fit their individual learners (Participant G).

https://commons.lib.jmu.edu/ijr 20
 Sawyer / International Journal on Responsibility, Vol. X (2024)

Preservice teachers are scared others would misuse ChatGPT. They are worried other teachers
would assume everything they find is accurate. This is a positive result because MTEs want our teachers
to use their professional knowledge and should not take everything straight from AI tools.
Preservice teachers also believed caution was needed because AI tools were insufficient and
needed more details to help them create classroom activities. For example, one participant stated, “It's
okay, I think some things may not be applicable. I would want more detail (which I can add). It is a decent
resource to use in times of need but I'm not sure I would use it” (Participant D). Some preservice teachers
are skeptical of using the tool and understand it does not provide everything needed to support learners.
Preservice teachers viewed ChatGPT’s responses as “old-school,” which means they are similar
to traditional classroom materials with fewer hands-on activities. For example, one participant stated, “It
is very old-school and not innovative when it comes to teaching math, so it should not be relied on”
(Participant E). They viewed ChatGPT as giving less inventive lessons with more traditional activities.
Another participant identified, “I think it is a great bouncing off point - but I think that teachers have to
keep their creativity and spark alive in order to actually use this tool to benefit students” (Participant N).
They believe teachers need to be that new spark to make the activities more alive and less traditional.

Discussion

Three major ideas emerged from the data. First, the prior instruction on Critical Curation was
ineffective in teaching preservice teachers how to adapt ChatGPT’s text. Second, teachers must do more
work to stimulate ChatGPT’s responses to be appropriate for elementary students. Finally, ChatGPT, in
its current form, without prompt engineering, could be another example of technology as innovation
rather than technology that produces innovative materials.

Lack of Critical Curation with Overconfidence in AI

It is the responsibility of MTEs to teach preservice teachers how to analyze the mathematical
curriculum materials used in classrooms critically. Despite preservice teachers being instructed on Critical
Curation before constructing their adaptations, the data indicated that 64% of the adapted preservice
teachers’ resources were only changed for visual appeal. These findings were consistent with Grossman
and Thomspson’s (2008) findings, in which new teachers lacked criticality. The preservice teacher’s
response echoed this lack of Critical Curation, “We didn't have to think to make it (Participant C).” Thus,
preservice teachers appeared to have overconfidence in ChatGPT's abilities. Even though 11 preservice
teachers identified caution with using the tool, they did not heed their own warnings, as the majority of
the adaptations created were almost identical to ChatGPT’s original text. This is concerning because
ChatGPT's responses, despite being a higher-level cognitive demand, were not always appropriate for
elementary students. ChatGPT's responses are written for individuals over 18 (OpenAI, 2023), so the
language is not universally understandable to elementary students. Also, as seen in the example of the
preservice teachers’ adapted resource, a three-page worksheet on coin recognition is not something one
can realistically expect a second-grade student to be able to complete. Thus, when teaching about Critical
Curation, MTEs also need to discuss what is not appropriate for elementary students and what language
should be used. Given the multiple contexts and developmental factors influencing student engagement
and access to curriculum, there is no assurance GenAI tools can account for those requisite nuances needed
for genuine curriculum development.

https://commons.lib.jmu.edu/ijr 21
 Sawyer / International Journal on Responsibility, Vol. X (2024)

More Work is Needed for AI Chatbots

This investigation highlights the need for more information about supporting preservice teachers’
modification of visual appeal. In our unit on Critical Curation, I did not discuss how to format text, space
questions, or omit wording. Since preservice teachers had to focus heavily on visual appeal, other
adaptations like changing for different learners’ needs may become secondary. It is possible preservice
teachers focused so much on changing the appearance that they overlooked other adaptations that needed
to be implemented. Future research would be necessary to determine if adaptations for different learners’
needs could be a secondary consideration for preservice teachers when so much effort is implemented for
visual appeal.

Innovative or Old School

Researchers argue that using AI tools is an innovative way to create new ideas for the classroom
(Ross, 2023). However, preservice teachers viewed ChatGPT’s text responses as promoting more
traditional teaching methods. OpenAI identified that the chatbot creates text based on matches from its
database; if the preservice teacher is not specific about the type of activity they want, it will produce what
it finds from its training alone (OpenAI, 2023). Preservice teachers need to implement prompt engineering
or structuring text in such a way as to allow AI tools to create the best response possible for their
elementary students (Korzynski et al., 2023). If teachers are not using prompt engineering, the push to
implement curricular AI in its current form could be progress through technology instead of the
technology tool creating progressive materials. Otherwise, the tool is the innovative element rather than
the task presented being the innovation. This showed MTEs must research new ways to effectively
prompt the system for mathematics content and teach these concepts to our preservice teachers. Thus,
this exploratory study opened up more questions about how MTEs can teach prompt engineering to
create an innovative mathematics curriculum.

Implications for Responsibility Towards Preservice Teachers’ Overconfidence

Since Critical Curation was not applied and creating visual appeal was more difficult with the text
responses, preservice teachers trusted ChatGPT to make mathematical content and effective teaching
strategies. This illustrates an “unintended and unobvious problem to which this technology might
contribute” (Krutka et al., 2019: 570). Preservice teachers seemed overly confident in AI’s abilities to
construct appropriate elementary mathematics materials, trusting AI tools to be “like a calculator” despite
being told they are not always accurate. This might imply that preservice teachers assume these activities
can be implemented effectively in a classroom. It also implies that MTEs are responsible for making
individuals aware of these issues. Bias and inaccurate information are present in ChatGPT (Wu, 2023),
but it is hard to spot because AI tools state everything as fact without citing sources. Therefore, this
investigation highlights the need for MTEs to provide clear examples of inappropriate, inaccurate, and
biased responses and allow preservice teachers to practice being critical first. MTEs also need to give
examples of AI chatbots’ text responses to teach how to adapt visual appeal to be appropriate for their
classrooms. These practices could help curb their trust in AI technology and show how difficult it can be
to use AI tools in the classroom.
Preservice teachers needed help constructing usable resources from ChatGPT. They identified
ChatGPT as a useful tool, especially for quick lesson plans, but they stated that ChatGPT gave more

https://commons.lib.jmu.edu/ijr 22
 Sawyer / International Journal on Responsibility, Vol. X (2024)

information than necessary. Preservice teachers found the large amount of information provided made it
difficult to select appropriate materials. This implies that ChatGPT 3.5 at the time of the study might not
be the best source for mathematics curriculum resources. Since images are not always available in the free
ChatGPT 3.5’s responses, it would be easier to continue using already constructed curriculum materials
from other sources. Therefore, future research must be completed, and resources must be created to
support teachers’ use of this tool in future classrooms. For example, researchers could investigate what
supports are needed to develop GenAI mathematics curriculum materials, then, from that research, create
a framework to guide teachers through the prompt engineering techniques to create the best possible
resources for the classroom.

Limitations and Future Research

This investigation has several major limitations. The preservice teachers used a single AI chatbot
in this investigation, and the results may differ from those of other AI platforms. I also used ChatGPT
3.5 in September of 2023. ChatGPT 4 is more advanced in some ways, and the tool has been updated since
the implementation of this investigation. Other AI chatbots, like Magic School AI
(https://www.magicschool.ai/), are made specifically for education, which creates lesson plans and
resources. Also, the preservice teachers were not instructed on how to develop effective prompts; they
only constructed their resources based on a limited number of iterations with the tool. This was an initial
exercise in using AI chatbots in mathematics education classes. Thus, more research needs to be done on
supporting prompting techniques for teachers in the field.
Another limitation was the small sample size. Data was collected from a single mathematics
methods course with a minimal subset of participants (n = 14). Therefore, further replication studies are
needed to determine if the findings are consistent across multiple groups of preservice teachers. Future
research may also entail identifying ways preservice teachers can be more critical of using ChatGPT. For
example, researchers can create counterexamples to display in mathematics methods courses,
demonstrating the deficiencies of AI tools. Additionally, researchers may wish to examine responses
ChatGPT will provide without any training. Last, another direction for future research would be to
identify how tasks become a higher level of cognitive demand to determine what kind of mathematical
prompt engineering is needed to create these results.
Professional policies and guidelines need to be constructed to support teachers in this
developmental exploration of AI tools in the classrooms. Teachers need to protect their students from
possible harm from these tools when they are not used correctly; thus, professional development
opportunities need to be provided to both in-service and preservice teachers to raise awareness of the
issues with AI tools and discourage overuse.

Conclusion

This exploratory study suggests diverse AI tools could be a form of inspiration but are not
immediately ready for direct implementation as a curriculum in a mathematics classroom. Creating
genuine and implementable curricular items for the elementary classroom takes more effort than asking
ChatGPT to provide a worksheet. MTEs are aware that preservice teachers will still use AI tools to create
mathematics resources; therefore, it is post-secondary instructors’ responsibility to contextualize this
overconfidence and support the critical use of AI tools in all classrooms.

https://commons.lib.jmu.edu/ijr 23
 Sawyer / International Journal on Responsibility, Vol. X (2024)

About the author

Amanda Gantt Sawyer is Associate Professor of Mathematics Education in the Middle, Secondary, and Mathematics
Education Department in the College of Education at James Madison University, VA (U.S.). She earned a doctoral
degree in mathematics education from the University of Georgia (U.S.), a master’s degree in applied mathematics
from the University of South Carolina (U.S.), and a master’s degree in elementary education from the University of
South Carolina. Her research focuses on mathematics teachers’ selection, critical curation, and implementation of
mathematics resources from modern technology.

References

Akgun, S.,Greenhow, C. (2022) Artificial intelligence in education: Addressing ethical challenges in K-

12 settings. AI and Ethics, 2(3), pp. 431-440. https://doi.org/10.1007/s43681-021-00096-7
Association of Mathematics Teacher Educators. (2017) Standards for preparing teachers of
mathematics. Association of Mathematics Teacher Educators, amte.net/standards, page accessed
1.10.2024.
Bill and Melinda Gates Foundation (2014) America’s teachers on teaching in an era of change. Primary
sources, 3rd ed., http://www.scholastic.com/primarysources/index.html, page accessed
1.10.2024.
Braun, V., Clarke, V. (2012) Thematic Analysis. In H. Cooper (ed.) The APA Handbook of Research
Methods in Psychology, vol. 2: Research Designs, Washington, DC: APA books, pp. 57–71.
https://doi.org/10.4135/9781483384436.n1
Brown, M. W. (2009) The teacher-tool relationship: Theorizing the design and use of curriculum
materials. In J. T. Remillard, B. A. Herbel-Eisenmann, and G. M. Lloyd (eds.), Mathematics
teachers at work: Connecting curriculum materials and classroom instruction. New York: Routledge,
pp. 17–36. https://doi.org/10.4324/9780203884645-11
Chai, C. S., Koh, J. H. L., Tsai, C. C. (2010) Facilitating preservice teachers' development of
technological, pedagogical, and content knowledge. Educational Technology and Society, 13(4), pp.
63-73.
Cho, W. (2023) “The New York Times brings receipts copyright lawsuit OpenAI.” Hollywood Reporter.
23, December. https://www.hollywoodreporter.com/business/business-news/new-york-times-
brings-receipts-copyright-lawsuit-openai-1235775781/, page accessed 4.30.2024.
Choppin, J. (2011) The impact of professional noticing on teachers’ adaptations of challenging tasks.
Mathematical Thinking and Learning, 13(3), pp. 175-197.
https://doi.org/10.1080/10986065.2010.495049
Coeckelbergh, M. (2023) Narrative responsibility and artificial intelligence: How AI challenges human
responsibility and sense-making. AI & SOCIETY, 38(6), pp. 2437-2450.
https://doi.org/10.1007/s00146-021-01375-x
Colvin, J. C., Tomayko, M. C. (2015) Putting TPACK on the radar: A visual quantitative model for
tracking growth of essential teacher knowledge. Contemporary Issues in Technology and Teacher
Education, 15(1), pp. 68-84.
Davis, E. A., Beyer, C., Forbes, C. T., Stevens, S. (2011) Understanding pedagogical design capacity
through teachers’ narratives. Teaching and Teacher Education, 27(4), pp. 797-810.
https://doi.org/10.1016/j.tate.2011.01.005

https://commons.lib.jmu.edu/ijr 24
 Sawyer / International Journal on Responsibility, Vol. X (2024)

Dick, L. K., Sawyer, A. G. (2022) Investigating elementary mathematics teachers’ adaptation of activities
found on virtual resource pools. Proceedings of the 44th annual meeting of the North American
Chapter of the International Group for the Psychology of Mathematics Education.
Dick, L. K., Sawyer, A. G., MacNeille, M., Shapiro, E., Wismer, T. A. (2023) Exploring grades 3–5
mathematics activities found online. Mathematics Teacher: Learning and Teaching PK-12, 116(3),
pp. 174-183. https://doi.org/10.5951/mtlt.2021.0330
Dick, L., Sawyer, A. G., Shapiro, E., Wismer, T. (2019). Influencing the influencers: Discussion of
elementary mathematics online resources. Association of Mathematics Teacher Educators Tech Talk
Blog. Retrieved from: https://amte.net/tech-talk/2019/09/influencing-influencers, page
accessed 1.12.2024.
Ding, M., Carlson, M. A. (2013) Elementary teachers’ learning to construct high-quality mathematics
lesson plans: A use of the IES recommendations. The Elementary School Journal, 113(3), pp. 359-
385. https://doi.org/10.1086/668505
Farina, M., Zhdanov, P., Karimov, A., Lavazza, A. (2022) AI and society: a virtue ethics approach. AI &
SOCIETY, pp. 1-14. https://doi.org/10.1007/s00146-022-01545-5
Forbes, C. T. (2011) Preservice elementary teachers’ adaptation of science curriculum materials for
inquiry‐based elementary science. Science Education, 95(5), pp. 927-955.
https://doi.org/10.1002/sce.20444
Grossman, P., Thompson, C. (2008) Learning from curriculum materials: Scaffolds for new teachers?
Teaching and Teacher Education, 24(8), pp. 2014-2026.
https://doi.org/10.1016/j.tate.2008.05.002
Korzynski, P., Mazurek, G., Krzypkowska, P., Kurasinski, A., (2023) Artificial intelligence prompt
engineering as a new digital competence: Analysis of generative AI technologies such as
ChatGPT. Entrepreneurial Business and Economics Review, 11(3), pp. 25-37.
https://doi.org/10.15678/eber.2023.110302
Krutka, D.G., Heath, M.K., Willet, K.B.S. (2019) Foregrounding technoethics: Toward critical
perspectives in technology and teacher education. Journal of Technology and Teacher Education,
27(4), pp. 555-574.
Lederman, N. G., Gess-Newsome, J. (1991) Metamorphosis, adaptation, or evolution?
Preservice science teachers’ concerns and perceptions of teaching and planning. Science
Education, 75(4), pp. 443-456. https://doi.org/10.1002/sce.3730750406
National Council of Teachers of Mathematics (2000) Principles and standards for school mathematics.
Reston, VA: National Council of Teachers of Mathematics.
National Council of Teachers of Mathematics (2024) Artificial intelligence and mathematics teaching.
National Council of Teachers of Mathematics. https://www.nctm.org/standards-and-
positions/Position-Statements/Artificial-Intelligence-and-Mathematics-Teaching/, page
accessed 4.30.2024.
National Council of Teachers of Mathematics (2014) Principles to actions: Ensuring mathematics success for
all. Reston, VA: National Council of Teachers of Mathematics.
National Council of Teachers of Mathematics (2000) Principles and standards for school mathematics.
Reston, VA: National Council of Teachers of Mathematics.
OpenAI (2023) Privacy policy. OpenAI. 30, June, https://openai.com/policies/privacy-policy, page
accessed 1.10.2024.
OpenAI (2024) ChatGPT (January 10 version) [Large language model]. https://chat.openai.com/chat,

https://commons.lib.jmu.edu/ijr 25
 Sawyer / International Journal on Responsibility, Vol. X (2024)

page accessed 1.10.2024.

Parsons, S. A. (2012) Adaptive teaching in literacy instruction: Case studies of two teachers. Journal of
Literacy Research, 44(2), pp. 149-170. https://doi.org/10.1177/1086296x12440261
Qualtrics (2019) Learn the survey platform. Qualtrics, https://www.qualtrics.com/support/survey-
platform/getting-started/survey-platform-overview/, page accessed 1.10.2024.
Remillard, J. T. (1999) Curriculum materials in mathematics education reform: A framework for
examining teachers' curriculum development. Curriculum Inquiry, 29(3), pp. 315-342.
https://doi.org/10.1111/0362-6784.00130
Ross, E. (2023) Embracing artificial intelligence in the classroom: Generative AI tools can reflect our
failure of imagination and that is when the real learning starts. Usable Knowledge, 20, July,
https://www.gse.harvard.edu/ideas/usable-knowledge/23/07/embracing-artificial-
intelligence-classroom, page accessed 1.12.2024.
Sawyer, A., Dick, L., Shapiro, E., Wismer, T. (2019) The top 500 mathematics pins: An analysis of
elementary mathematics activities on Pinterest. Journal of Technology and Teacher Education,
27(2), pp. 235-263.
Sawyer, A. G., Dredger, K., Myers, J., Barnes, S., Wilson, D. R., Sullivan, J., Sawyer, D. (2020) Teachers
as curators: Investigating elementary preservice teachers’ inspiration for lesson planning.
Journal of Teacher Education, 9(1), pp. 1-19. https://doi.org/10.1177/0022487119879894
Shelton, C.C., Archambault, L.M. (2019) Who are online teacherpreneurs and what do they do? A
survey of content creators on TeachersPayTeachers.com. Journal of Research on Technology in
Education, 51(4), pp. 398-414. https://doi.org/10.1080/15391523.2019.1666757
Sherin, M. G., Drake, C. (2009) Curriculum strategy framework: Investigating patterns in teachers’ use
of a reform‐based elementary mathematics curriculum. Journal of Curriculum Studies, 41(4), pp.
467-500. https://doi.org/10.1080/00220270802696115
Stein, M. K., Smith, M. S. (1998) Mathematical tasks as a framework for reflection: From research to
practice. Mathematics Teaching in the Middle School, 3(4), pp. 268-275.
https://doi.org/10.5951/mtms.3.4.0268
Stein, M. K., Remillard, J., Smith, M. S. (2007) How curriculum influences student learning. Second
handbook of research on mathematics teaching and learning, 1(1), pp. 319-370.
Tasioulas, J. (2022). Artificial intelligence, humanistic ethics. Daedalus, 151(2), pp. 232-243.
https://doi.org/10.1162/daed_a_01912
United States Department of Education (2023) “U.S. Department of Education shares insights and
recommendations for artificial intelligence.” Press Office. 24, May.
https://www.ed.gov/news/press-releases/us-department-education-shares-insights-and-
recommendations-artificial-
intelligence#:~:text=It%20recognizes%20AI%20as%20a,feedback%20loops%2C%20and%20sup
port%20educators, page accessed 4.30.2024.
Virginia Law (2024) 8VAC20-543-20. Accreditation and administering this chapter.
https://law.lis.virginia.gov/admincode/title8/agency20/chapter543/section20/, page accessed
1.10.2024.
Wu, G. (2023) “8 big problems with OpenAI's ChatGPT.” MakeUseOf. 17, September.
https://www.makeuseof.com/openai-chatgpt-biggest-probelms, page accessed 1.10.2024

https://commons.lib.jmu.edu/ijr 26