We invited faculty members in the 2024-2025 Purposeful AI Working Group to reflect on their experiences building custom chatbots that align with their teaching values and enhance student learning. Heather Olins, Associate Professor of the Practice of Biology, shared her thoughts with us, which have been condensed and slightly edited.
How has your view of GenAI in higher education changed over the past year?
I have moved from avoidance to open-mindedness. I’m not quite at ready to embrace this technology, but I think that if I am able to dedicate more time to figuring out how these tools can help me, personally, I will probably get there. I used to use it only in a cautionary or negative way. I’ve created a bad example of an annotated bibliography to demonstrate to students how these tools hallucinate. I’ve used it to identify assessment questions that are too simple for open-book tests (if ChatGPT can answer the question, it’s not good enough). I’ve become more and more aware of students using these tools as I receive emails that are oddly formal and overly complimentary and (on occasion) include identifiers such as <include Professor name here> or as I see infographics, presentations, or other visual work that they clearly did not design themselves. All of this use I would classify as negative or more in the “gotcha” mindset.
But this is not the space in which I want to exist as an educator. I don’t want to focus any more than I need to on ways in which my students might be cheating or taking shortcuts. I would rather focus on ways to help students stay engaged and engage more deeply with the material by practicing metacognition and reflection. I would rather work with students to identify ways in which these tools might help students create work that is more meaningful and creative.
What were the most challenging parts of building an AI chatbot?
The most challenging part of building the chatbot has been not being confident that I can prevent it from providing students with inaccurate information. The chatbot I built is designed to guide students through reviewing an exam and does a good job suggesting strategies for this purpose, as well as asking questions that I would typically ask in conversation. But it also slips easily into content review. Initially, I thought this was bad, but I realized that the content review was very helpful for a particular topic. Since I can never know all the questions students may ask, I worry that I will never be able to fully “vet” whether or not the platform provides useful information, or perhaps might be leading students astray if they veer into content review. Perhaps I could ask students to test it out with me and let me know if they find anything they think might be problematic. However, I don’t think students in an introductory course are able to identify when an AI tool is providing them with inaccurate information, and this is the main concern I have currently. I think only a few students in this introductory course would actually be able to identify the misinformation, which is why I’m not sure whether to move forward with testing the tool. Perhaps there is a way I can pilot it a bit in the lower-stakes summer version of the course. Perhaps I could instruct it to limit its content to one open-access biology textbook, or try to prevent it from conducting any content review, even though that seems to be falling back into the negative ‘AI is bad’ mindset. There appears to be a LOT of potential for giving students something they really want (more practice problems) if I can convince myself that it is not providing inaccurate information.
The other issue I encountered was that, for some reason, the chatbot was incredibly slow to respond to my prompts (or those of my TAs when I asked them to use the tool). I assumed that, given the response times I was seeing, students would give up on using the tool before they completed a meaningful exam review.
Moving Forward: What to Focus on Next Year
Over the next year, I would like to practice using these tools more so that I have a better sense of the way that they can help me individually. I think that if I become a more confident user, I may become convinced that there is real good that comes from specific uses of them. Maybe GenAI can help me:
- Draft the introductory textbook chapter I’ve been procrastinating.
- Stay more on top of personal administrative tasks,
- Come up with more/better higher-level assessment questions, and
- Provide more comprehensive review materials for students based on my lecture slides, etc.
I temper this a bit by the fact that I am someone who teaches extensively about climate change and am deeply concerned by the extent to which moderate gains in reducing energy use over the last few years seem to be rapidly being offset by the immense amount of electricity and water needed to power and cool the processors and data centers behind these tools. That alone is another argument in the greater harm column (of the greater harm vs. greater good debate). At the same time, I want to use these tools a lot more; I also want to be mindful of not having that use become a waste of resources. I’m not sure how to reconcile these things, and I think this has been part of why I haven’t used them more thus far.
