Can ChatGPT Create a Thoughtful Lesson Plan?
Civics of Tech announcements:
Next “Talking Tech” Monthly Meeting: We hold “Talking Tech” meetings on the first Tuesday of every month from 8-9pm EST/7-8pm CST/6-7pm MST/5-6pm PST. These Zoom meetings will include discussions of current events, new books or articles, and more. Participants can bring topics, articles, or ideas to discuss. Our next meeting is on Tuesday, March 6th. Register on our Events page or click here.
Book Club Reminder: Our next book club will discuss the 2020 book Data Feminism by Catherine D'Ignazio and Lauren F. Klein on Thursday, March 16th, 2023 from 8-9:30pm EST/7-8:30pm CST/6-7:30pm MST/5-6:30pm PST. The book is available in multiple formats, including audiobook, and a paperback version was just released! Register on our Events page or click here.
by Christopher H. Clark & Cathryn van Kessel
In November 2022, a company called Open AI released ChatGPT, an AI “chatbot.” ChatGPT describes itself as “a language model developed by OpenAI, which uses deep learning techniques to generate human-like text based on the input it receives. It is designed to respond to questions, complete text based on the prompt, summarize long text and perform various other language-related tasks” (OpenAI, 2023).
Since its emergence, many leaders and educators in institutes of higher learning have been in a moral panic about students potentially cheating and yet feeling obligated to have students use ChatGPT. This situation is a frustrating one for many reasons outlined in Autumm Caines’ recent cross-post on the Civics of Tech blog.
Technology provides humans with something while it takes something away. Consequently, it is important to ask technoskeptical questions, such as “What does society give up for the benefits of the technology?”
So, from the perspective of teacher educators, we are asking what the benefits and sacrifices for pre-service and early career teachers might be if they use ChatGPT to create lesson plans for their students: Can ChatGPT Create a Thoughtful Lesson Plan?
The short answer seems to be “no.” A longer, more qualified, response would be “no, unless you know what to ask it for.”
Our plan is to try out a number of different topics and try out different prompts, but our first attempt was to see if ChatGPT could generate a thoughtful lesson about Martin Luther King, Jr. We chose this topic both because we know it is included in nearly all high school U.S. History curricula and because, as many thoughtful scholars have noted, it is often covered problematically. We began by thinking about what our criteria would be for a good lesson. Given the topic, we turned to an article by Dr. Ashley Woodson, “We’re just ordinary people: Messianic master narratives and Black youths’ civic agency.” From her insights, we created the following content-based criteria:
● avoids framing an individual as a “savior” or deliverer of a specific group (i.e., depicts them holistically);
● highlights the networks of individuals doing a wide variety of actions that contributed to the Civil Rights Movement;
● views racism as systemic (or avoids framing racism as a personal failing);
● highlights diversity and/or divergent voices within the Civil Rights Movement; and
● connects the Civil Rights Movement to contemporary efforts at social justice
We decided to rate these criteria on a three-point scale, loosely adapted from scales used by the Stanford History Education Group’s Civic Online Reasoning project to rate performance from beginning to mastery. A rating of 1 was given when the desired element was not present, or present in such a way that does not reflect any of the desired content or framings. This rating was also applied when ChatGPT provided vague responses (e.g., use documents that reflect multiple views on the civil rights movement). A rating of 2 meant a developing or partial fulfillment of the criterion. This rating was given when an element was present and shows potential, but does not fully meet the stated requirements or remains problematic in some way. For example, when we asked ChatGPT to include an increased focus on systemic racism, we found that it mentioned systemic racism frequently in the text of the lesson, but that it was vague about sources it would use and that it wasn’t clear if the AI understood the term or if it was just pasting the term into the lesson because we had asked it to. The rating of 3 was for a criterion that was present; i.e., when the lesson produced by ChatGPT had the content element that we are looking for in an acceptable manner.
Our conversations with ChatGPT were conducted with both of us connected over Zoom. We began by simultaneously typing the same prompt (“Create a high school lesson plan about Martin Luther King, Jr.”) into ChatGPT and we compared the results. Interestingly, the AI created different lesson plans for each of us. The AI created a lesson plan for Chris that focused a lot of attention on rhetorical analysis of MLK Jr.’s speeches, while the one created for Cathryn was more on the content of the speeches and what they meant at the time and now. So, the first thing we learned was that ChatGPT will produce different lessons for different people.
We also learned that those lessons can be of different quality, content-wise. We compared the two lessons generated by ChatGPT to our rubric and independently assessed each element. Immediately upon completion of our independent assessments, we discussed each rating and, where points of difference emerged, we deliberated until a consensus scoring was reached. In scoring the lessons, we found that the lesson produced for Chris seemed to spend a lot of time emphasizing rosy narratives of MLK, talking a lot about his “vision” for society and not explicitly situating his work in the context of a broader movement, or noting that this vision was not shared by all within the broader Civil Rights movement. The lesson also did not prompt students to think about why such a movement was even necessary, avoiding mentioning racism altogether. The lesson the AI produced for Cathryn, while still rather vague, at least checked a few more boxes on our rubric, asking students to research other Civil Rights figures and doing a better job of situating King within the broader movement. The lesson did include suggestions for extension activities that connected the King and the Civil Rights Movement to social justice struggles today, but these were implied to be optional and the lesson still did not directly address racism, past or present.
After deciding that the lesson generated on Cathryn’s computer was a stronger starting point, we chatted with ChatGPT to modify that lesson, making five total attempts (figuring that pre-service and early career teachers aren’t going to spend all day getting ChatGPT to modify the lesson plans). We collaboratively assessed each lesson plan according to our rubric and decided how to phrase each follow-up prompt. Our follow-up prompts were: “Modify this lesson plan to include more of a focus on systemic racism,” “Modify this lesson to include people who supported MLK and people within the Civil Rights Movement who disagreed with MLK,” “Keep the emphasis on diverse voices but modify this lesson to show how MLK worked alongside communities and organizations,” and “Modify this lesson to include: systemic racism, people who supported MLK and people within the Civil Rights Movement who disagreed with MLK, and how MLK worked alongside communities and organizations.”
Even with all of these modifications, ChatGPT could not produce a lesson that scored a 3 in all of the criteria. Furthermore, until the last iteration where we explicitly named a lot of the criteria, something was lost where something was gained. For example, the second iteration had an emphasis on systemic racism, and that emphasis was lost in the third lesson plan even though that one met the criteria for more diverse voices. As we read each lesson iteration, we were often impressed with questions or suggestions for activities in one lesson and then disappointed when those activities and questions were lost in response to a new prompt. Despite the AI’s conversational orientation and ability to “remember” previous parts of the conversation, it seems like ChatGPT read our requests for increased emphasis on certain parts of our rubric as overriding our earlier requests (e.g., it interpreted our request for more diverse voices in attempt 3 as something we wanted instead of a focus on systemic racism, as opposed to something we wanted in addition to systemic racism). It wasn’t until we attempted a “kitchen sink” request that included all of our areas of emphasis that we got a lesson that scored high in almost all the areas of our rubric.
The fifth and last lesson plan was the best one—the one where we explicitly named a lot of the rubric criteria. However, it still relied on students (and their teacher) already having a good understanding of what systemic racism is. Students would need some non-ChatGPT instruction of systemic racism to know what that is and perform the lesson plan’s tasks.
ChatGPT users need to be very specific about what they are looking for in a lesson plan about MLK, Jr., and even then they will need to modify the lesson. And how will pre-service and early career teachers know what to be specific about? They will only know what prompts to give the AI from their teacher education and professional development experiences. We need scholars like Dr. Ashley Woodson to point us toward how content can be thoughtful, fostering a nuanced understanding of the Civil Rights Movement while encouraging student agency, and then we will need to follow their advice. Although ChatGPT can help teachers figure out ways of approaching a topic, nothing can replace professional judgment stemming from educational research.
That being said, when paired with professional judgment and research-informed instructions, ChatGPT was able to produce lesson plan ideas that, while not perfect, included insightful questions and interesting activities. A thoughtful teacher would likely be able to combine the stronger elements of each of the lesson iterations, elaborate on the vague areas, and create a meaningful learning experience for their students. On the other hand, asking ChatGPT for a lesson plan on a topic without any additional guidance, as done in some early explorations of the AI’s lesson planning potential and as we did in the first iterations of our instructions, seems to result in lessons that are fairly mediocre and, depending on the topic, may end up perpetuating problematic narratives and discourses.
An over-reliance on ChatGPT for creating lesson plans means that we would be giving up many of the insights provided by educational scholars. Much like using sites like Teachers Pay Teachers or Pinterest (Rodríguez et al., 2020), it might be helpful to see how someone else would make a lesson, but we must use current research to ignore or modify aspects of those lessons if we want to do right by our students.
References
OpenAI. (2023, February 5). Explain briefly what ChatGPT is. ChatGPT [Computer software]. OpenAI.
Civic Online Reasoning (n.d.). Civic Online Reasoning. https://cor.stanford.edu/
Rodríguez, N., Brown, M., & Vickery, A. (2020). Pinning for-profit? Examining elementary preservice teachers’ critical analysis of online social studies resources about Black history. Contemporary Issues in Technology and Teacher Education, 20(3), 497-528.
Will, M. (2023, January 12). We gave ChatGPT common teaching tasks. Here’s how teachers say it did. Education Week. https://www.edweek.org/technology/we-gave-chatgpt-5-common-teaching-tasks-heres-how-teachers-say-it-did/2023/01
Woodson, A. N. (2016). We’re just ordinary people: Messianic master narratives and Black youths’ civic agency. Theory & Research in Social Education, 44(2), 184-211. https://doi.org/10.1080/00933104.2016.1170645