Erik Brynjolfsson and Andrew McAfee in their excellent 2014 book the Second Machine Age pointed to the coming revolution in machine learning and how it would change the way we work. Now with ChatGPT this revolution is here and on campus. Rarely has a technology had such an impact on conversations on higher ed teaching and learning so quickly as ChatGPT and the new wave of AI chat tools. 

The moment I realized it had come for me was when I put midterm exam questions from last year into ChatGPT. The quality of the responses surprised me.

Naturally, this raised quite a few thorny questions. On the surface AI chat tools suddenly appeared to be able to replicate some of the work I was tasking students to do. If students could use these AI chat tools to write essays, solve equations, take multiple choice exams, write computer code, where would that leave faculty? What are our expectations for students if AI can do the work we have been assigning them for the entirety of our teaching careers?  

Clearly these were not questions that I was going to answer overnight, and I still had to show up to teach class tomorrow. So what could I do in the meantime?

An Opportunity to Rethink

Like many of us, I scrambled to educate myself on the topic by following how others were making sense of this sea change in real time. In particular, I found Ezra Klein’s interview with AI researcher and entrepreneur Gary Marcus a useful primer on the limitations of the technology at this moment. 

Marcus argues that while tools like ChatGPT appear on the surface to have made the job of faculty (and perhaps learning) irrelevant, once you explore it more, you realize it is wafer thin. At a technical level tools like ChatGPT (and others like Notion, lambda, Jasper, Quillbot, etc), are predictive models that use vast amounts of data from the internet to predict what the next word or instruction will be in response to a query. Sometimes it gets it right, sometimes it gets it wrong. Why? Because it does not understand the connections between the information. ChatGPT is proving to be a game changer because its responses appear very realistic and therefore correct.

And here is where I began seeing the opportunity for teaching and learning. AI chat is just a tool. It cannot make connections, be reflective, convey emotion, create new knowledge, or discern what is in fact truth or not. In this era of disinformation, AI chat tools provide us with an opportunity as educators to help students learn how to use their education to locate truth, to parse fact from fiction, to critically analyze, to understand how to apply knowledge. These skills are the basis of a liberal arts education that BC is renowned for. Rather than making them irrelevant, AI chat tools have made them ever more important. 

The liberal arts offers a toolkit for responding to the highly plausible answers we can get from AI bots, an approach for critically engaging with them by 1) analyzing why they are right or wrong, 2) building on them to expand their own understanding, and 3) applying basic foundational concepts to more complex work. 

How Should I Respond?

It seemed I had three main ways I might react to the emerging presence of AI in education, beyond just appealing to Boston College’s existing Academic Integrity policy (which will like need some revision):  

1. Ban it: treat it as plagiarism the same way I would any work from an “essay mill,” 
2. Adapt it: allow it to be used as a tool like google search, wikipedia etc, but require students to reference it and not use it to replace their own thinking, or
3. Engage it: integrate it critically and mindfully into our teaching.

When I reflected on why ChatGPT could offer a reasonably competent answer to my exam questions, I realized that the problem wasn’t that students might use ChatGPT to get solid answers to my exam; the real problem was with my questions. They were too straightforward and required little more than regurgitating information. I was not explicit enough with students about how I wanted them to apply this knowledge to the specific case studies we covered in class. 

Using ChatGPT is now forcing me to make the changes I have been meaning to make for a long time: 

1. Tasking students with making judgements on key questions in politics using evidence and political science concepts, and 
2. Giving them the opportunity to organize and share their learning from the course in different digital formats if they want to.

In the end, I decided to take the engagement approach, since this technology is here and is only going to get more advanced and banning it just kicks the can down the road. Why not take this opportunity to explore it with students? 

How I’m Using It

I quickly thought about some ways I could incorporate it into my spring 2023 Political Science course on Populism in a way that would invite my students to explore how ChatGTP might related to how they learn in this course. 

Learning Outcomes

Here is a learning outcome I have added to my course: 

• Students will be able to explore how AI tools can be used to develop critical analysis 

With thanks to Woods College Professor Dr. Erin Baumann for sharing, this is the language I am using in my syllabus: 

As the course objectives above outline, I believe that understanding how to live and work with digital tools and platforms – from collaborative documents to data visualization tools to artificial intelligence tools – is an essential skill for all students in this day and age. In this course I encourage you to use all the tools available to you (and that you are familiar enough with to use efficiently and effectively) to aid your learning. This includes Artificial Intelligence (Al) copywriting and chatbot tools such as ChatGPT, notion.ai, copy.ai, and others. However, as with any other resource you use to aid your work in this course, you must acknowledge any and all Al tools that you use in the development of your work. You must also substantially revise any writing or work produced by an Al tool before submitting it for credit in this course. 

If you use an Al tool at any point in the creation of your work for this course you must include the acknowledgment below in your reference list.

[Name of Al tool] was used in the process of creating this work.

In-Class Discussion

First, I started off by using Chat GPT to generate an answer to a midterm question from last year. When I asked students what grade they would give it, there was almost universal consensus that it deserved an A. I let them know that it would actually fail as it didn’t include references. But even if it had included references, it would only earn a C because it failed to include any examples, it lacked an original argument, and was missing references to the main themes of the course. This got their attention and facilitated an excellent conversation about the expectations I have for their work in the course. Based on this experience, I intend to use this kind of activity in-class to help us sharpen our critical thinking skills. 

Midterm

Second, I will most likely give them a midterm question that tasks them with using an AI Chat tool to generate an answer to a specific question about a key concept. They must then critique it using the main themes of the course. 

Final

Finally, I am thinking about a question for the final that will involve a substantive engagement with an AI Chat tool. Students will have to ask it a question about Populism and keep iterating their questions until they get an answer they are satisfied with. I want students to then reflect on this process and the questions they asked, evaluate the responses, and summarize why they are happy with the answer. 

And to give myself more options for student evaluation beyond essay style questions, I will also offer them a data visualization project option. This is something that I have been thinking about for a long time. The need to respond to Chat GPT has motivated me to finally design this assessment and eventually share it out with students. 

The students are very keen to explore Chat AI tools to help their learning. As I explore with them across these three elements of my course I will report back on this blog.