What Happened when my student tested my assignments in chat gpt vs. human students. William Price put ChatGPT to the test for his capstone project to see how it compared to him and his classmates. He shares today's results in this podcast episode.

The Human Side of AI: A Student’s ChatGPT Experience

For his senior project, William tested his teacher's assignments inside ChatGPT to see which would do better: human students or ChatGPT. The results may surprise you. How will we adjust with ChatGPT in the classroom? Let's get real world perspective.

As we continue with the series, '80 Days of AI and AI,' after a much-needed summer break, this interview with my student, William, marks a return to the learning journey. But this journey is not just for me and my students. This learning journey is ours as educators venturing into this rapidly changing area of AI. 

In  April, William Price, one of my senior students, shared his proposal for a capstone project. William was curious: what would happen if he pit ChatGPT against the assignments from our Computer Science class. The question? Could this AI perform as well, or possibly even outperform the human students/ And if so, which assignments would it excel in, and where would it fall short?

William returned to our classroom just a week after graduation to share what he learned, his honest observations about this new tool, and what his generation is saying about it.

While many are talking about the role of AI in education, as a classroom teacher with my students, we are taking a different approach. We're not just discussing it but implementing, testing, and learning from it. We're learning out to supervise AI and when it is effectively used (or not.)

So, on Day 16 of my '80 Days of AI and HI' series, it seems only fitting to showcase one of my inspiring student's contributions to this exploration.

Six questions in an infographic that were shared in the blog post.
Use this infographic with these questions to promote reflection with your teachers and preservice teachers.

Reflection Questions

Essential questions which will help you reflect and discuss this with colleagues aligned with the different levels of Blooms Taxonomy:

  • After reviewing William's steps in the first assignment with ChatGPT, can you identify the action or decision that may have skewed the results? How does this inform our understanding of the importance of effective prompting in teaching? (Understanding)
  • Given William's insights on the tasks where ChatGPT performs at par or better than human students, how should Vicki apply this knowledge to adjust her assignments to align them with learning goals? How should educators adjust their assignments based on this knowledge? (Application)
  • From your understanding of William's reflections, can you identify patterns explaining why students might use ChatGPT for assignments? Are these patterns prevalent or unique to certain circumstances?  (Analyzing)
  • Based on William's experience, can you dissect the factors contributing to ChatGPT potentially earning a student an ‘easy A?” Why might this trend be alarming for us educators? (Evaluating)
  • Using the insights gained from this discussion, can you propose new methods of assignment design that reduce the risk of AI-assisted answers without authentic student learning? (Creating/Synthesis)
  • Create recommendations for teachers to “test” their assignments using AI and share them with colleagues. (Creating/Sythesis)

We have many more questions to ask additionally including how we maintain the integrity of student work while also preparing students for a world increasingly shaped by AI. Our students will be supervising AI and we need to approach what we're teaching with that in mind. We need to examine if AI is being used responsibly in our schools by teachers, students, administrators, and parents. This generation needs us to discuss these questions openly and involve them in many conversations.

Note: If you go through this process of reflection and blog it, please email me at vicki at coolcatteacher dot com. I'd be fascinated to hear what you're learning. 

Just a an editorial note here for all of us educators. I hope some educators will 'dust off' their blogs so that we can again read each other in our RSS readers as the increasingly fragmented and fluid social media landscape can't interrupt our conversations at this pivotal moment in history. We can't afford to lose each other in the dustup between social media sites right now: the conversations are too important. We don't need to thread, blue sky, instagram, Facebook or tweet these conversations but have them accessible to everyone everywhere -- social media connected or not. So yes, I'm going back to Feedly and my RSS reader to follow educators who are blogging about using AI in the classroom. Email me if you're one of those educators! Please!

10 Minute Teacher Podcast - Episode 807

The Human Side of AI: A Student’s ChatGPT Experience

William Price, Class of 2023, Senior Computer Science Student of Mrs. Davis

Subscribe to the Show

10 minute teacher podcas audible
William Price with his senior computer science teacher Vicki Davis

Resources mentioned in episode 807

William Price Bio

William Price graduated from Sherwood Christian Academy in Albany, Georgia in May 2023. William was All-State in Football with over 1,000 all-purpose yards and was part of SCA's state-winning football championship team. William also played basketball and soccer and served as the news anchor for the Sherwood Showstoppers broadcast for two years. He aced Mrs. Vicki Davis' Computer Science class and always sparked great conversations in class. Thanks to William for coming on the show and for his parents coming to his presentation and asking him the tough questions. They all inspire me.

More information may be added to this bio at a later time.

Transcript - Episode 807

00;00;00;00 – 00;00;24;17

Vicki Davis

This is the Ten Minute Teacher podcast with your host, Vicki Davis. I'm so excited today to be talking to my students. William Price Well, technically he's not my student anymore because he graduated last Friday, didn't you, William? Yes, ma'am. Congratulations. William Price, Class of 2023. Now, William was in my computer science class during his senior year. Tough class, right, William?

00;00;24;18 – 00;00;48;29

William

Yes, Very, very tough. 

William's Project

Vicki

And he's not just saying that because it was, and they got to select a final project of their choice, a capstone project. And lots of people did a lot of different cool things, but. William, what was the name of your project?

William

ChatGPT versus Mrs. Davis's computer science class

Mrs. Davis

What was the premise behind your capstone project?

 

00;00;49;00 – 00;01;10;18

William

I logged into Chat GPT and wanted to see how advanced the chatbot was, so I had it do all the tests the computer science class did over the year. 

Analyzing the Major Projects and ChatGPT's Ability to Do Them

Vicki

Yeah, and you are talking about the major projects we did and presented on that. Your parents came was a fantastic presentation. But let's talk through. I think you ran it through four of our major assignments.

 

00;01;10;19 – 00;01;29;16

Vicki:

Can you remember those assignments and talk through and how it did on each of them? 

Comparing Which Computer to Buy

William

For the first assignment, I believe I asked the chatGPT to compare different computers. Yes. 

Vicki

So you had a computer shopping project [PDF] that you did last semester. And I'll tell a little bit about this. In this project, students are given a budget.

 

00;01;29;18 – 00;01;54;02

They have to look up computers online, get the specs for those computers, and then include the specs, and then they had to write a paper on which computer would be the best for their particular major. Okay, so what did you feed into ChatGPT?

 

William

I remember not wanting to be too specific on anything and seeing how well the chatbot could have gathered all that information.

 

00;01;54;02 – 00;02;14;25

Willliam

So I just asked it to compare four computers for me. Give me the pros and cons of both, and then write out a little paragraph at the bottom. And I was asking it just which computer would be better for me to buy. I listed out as a chart because it couldn't do a spreadsheet because it didn't like have access straight into a PDF, put it inside a chart inside chatGPT.

 

00;02;14;27 – 00;02;36;00

William

It didn't give any specific names of the computers, but it just listed out the specs of the computers. 

 

Vicki

Oh, so you didn't even feed in the four computers you found? You let it create four theoretical computers, 

 

William

Yes. 

 

Vicki

Well, and on this one, How did you feel? What one do you feel like? It did it as well as students did?

 

00;02;36;02 – 00;02;52;22

William

No, because the students could go more in-depth on it. But I feel like if I gave it a more specific prompt and asked it to do more, it probably would have done just as well. 

 

Vicki

So I was actually very impressed with what it wrote up. I did not realize that it sort of fabricated the four computers. That’s interesting.

 

00;02;52;28 – 00;03;13;18

Vicki

I actually was pretty impressed with what it wrote. I would think it would have been a solid answer. But there's one thing I want to come to, and I've been writing on 80 days of HI and AI. “HI” is human intelligence. William is probably thinking, Mrs. Davis, I have to come back and hear this again. I want to make the point that he brought his human intelligence into it.

 

00;03;13;18 – 00;03;31;28

Vicki

And he is right. If he fed in the four different computers that he had found, it probably would have given even better answer for the best computer. Of course, it can still be flawed, but I was pretty impressed with the the answer that it gave. So you feel like the human answers were better because they were more specific?

 

00;03;32;00 – 00;03;51;08

William

Yes. 

 

Vicki

And they had real hyperlinks. 

So what was another project that you can remember that you ran the chatbot against? 

Creation of a Website with HTML Code

William

I asked it to create a website out of HTML coding, and I thought it did very well with that. It's about as good as what any student did this year when it came to the HTML project. 

 

Vicki

So what kind of website did you ask it to build?

 

00;03;51;08 – 00;04;14;00

William

I typed in this made up person I forgot the name, but I said to create a website about this person. All the stuff that it likes: favorite movies, places to go to eat. And I think his favorite music. 

 

Vicki

I was impressed with this too, because William and his classmates learned to code HTML by hand, and there will be people on the internet rolling their eyes at that.

 

00;04;14;00 – 00;04;41;00

Vicki

However, I believe personally that understanding how to read and write HTML is the language of the web, and you never know when you're going to come across HTML. So I have found in my experience of teaching HTML by hand for over 20 years now that it does come in handy. But, you found William, what a lot of people are finding that actually ChatGPT is really good at writing HTML. Now, did it add any CSS or not?

 

00;04;41;02 – 00;04;58;13

William

From all the code that I saw? It looked like HTML to me. There was nothing that I didn't learn. 

 

VIcki

But you didn't ask it to write any CSS either? 

 

William

No, mam. 

 

Vicki

Yeah. So I've heard of people having it right? Whole HTML sites, whole mock ups, and this is supposed to be something it's very good at. But you did give it the prompts, so that's great.

 

00;04;58;13 – 00;05;25;21

So in that case, GPT was as good as you and your classmates. Yes. Well, cool. Okay. What was another project? 

Develop a Game in Python (in the Ozaria Platform)

William

I asked it to develop a game in Python or develop a game using Python and Ozaria. It was unable to do it in Ozaria because it cannot access the website, but it did create a game in Python. 

 

Vicki

So in this case, we're using a tool called Ozaria.

 

00;05;25;24 – 00;05;46;10

Vicki

They have another piece called Code Combat to teach Python. And so one thing that William doesn't know is it actually has a little bit of Python code behind the scenes that it inserts. So ChatGPT, I have found, will help kids troubleshoot their Python code but typically won't write it for them because not all features are available to kids as they go through.

 

00;05;46;10 – 00;06;06;22

Vicki

This is one reason I love Ozaria for kids. It kind of keeps them where they are and keeps them from copying off the internet. So William, what was the fourth assignment? 

A Presentation on Emerging Technology

William

I asked ChatGPT to create a short presentation on emerging technology. It was able to basically spit back out what every student wrote earlier in the year, it just it reworded it. It was probably no more than like three paragraphs.

 

00;06;06;28 – 00;06;27;22

Vicki

So did you paste in something like? No? You just said emerging technology?



William

It was able to search all over the internet and find the emerging technology and articles. It wasn't able to post any links. A lot of the stuff that I wrote down I’ve seen when I was doing the project, I’ve seen where sites had said similar things.

 

00;06;27;22 – 00;06;47;27

Vicki

I'm curious, do you remember what emerging technology it picked? 

 

William

It did at least three, AI…

 

Vicki

…. of course, is going to write about itself…

 

William

…virtual currency, blockchain, and stuff like that. Okay. And I can't remember what the last one is. I think it had to do something with transportation. 

 

Vicki

Oh, okay. Like a Hyperloop or something, which we talked about those.

William's Conclusion About ChatGPT's Ability to Perform the Assignments for the Class

00;06;47;29 – 00;07;23;08

Vicki

Okay. So your conclusion that you came to in the class was what? 

 

William

Chat GPT is good, but you can't rely only on ChatGPT. But it's a good steppingstone when it comes to developing things like when it comes to coding and anything regarding Computer Science. If you want that to get the mundane task out of the way, then you can just go in a chat GPT, type in a prompt for you as you get further down to your project, use your own human side to it improve it.

 

00;07;23;08 – 00;07;41;06

Vicki

You gave a number; you said 50%. You felt like on half of the projects it did a reasonably good job. 

 

William

Yes, ma'am. 

 

Vicki

And so this is a word to teachers is that I believe Chat GPT and tools like large language models like that, which we need to understand that there are other models.

Changing How We Write Assignments

00;07;41;06 – 00;08;10;00

Vicki

The Google model is rapidly gaining. There is an open-source, AI large language model that is going to be widely used. All of these large language models can be very, very useful. But you have to write your assignments and your tasks in ways that require human intelligence

 

It's like when Google came out, and we had to stop asking kids to go copy out of the encyclopedia at their house because otherwise, they were just going to copy off Google and turn it in, which they did at the beginning.

William's Thoughts on Student Use of ChatGPT to Complete Assignments

00;08;10;00 – 00;08;35;24

Vicki

And now you're seeing Chat GPT be the same thing. 

So, William, your parents were here watching. And tell us about the question your dad asked, which is a question every parent is asking these days. 

 

William

It was about how teachers can see if a student is using chat GPT or not. 

 

Vicki

Yeah, for cheating and what you thought about kids using Chat GPT for academic work. So that's kind of the question everybody's asking what are your thoughts?

 

00;08;35;24 – 00;09;01;18

William

There are ways to detect Chat GPT, but I believe that is going to be kind of like Google Chat. GPT. I don't think it's going to be used as much for cheating but more as kind of a search engine. Get in, like I said, mundane, every day, easy task out of the way. 

 

Vicki

When you first started hearing other students, whether at this school or other schools, talk about chat GPT and how they could use it to be dishonest, what did you hear others start saying about it?

 

00;09;01;19 – 00;09;22;02

William

Usually, they were joking. I said, Oh, ChatGPT, I'll just get it to do my whole paper and something like that without looking over it. But there are just some people out there that are going to it. Like if they're feeling stressed out and they just want to get an easy A or whatever, they might just ask Chat GPT to do this for them.

 

00;09;22;05 – 00;09;43;14

Vicki

So what do you think? What do you think about that? Is that a problem? What are your thoughts? 

 

William

It is a problem after the pandemic, people have. I've been trying to find an easy way out when it comes to things because, having an extended summer break and coming back to school, students have been lazy. But it's going to come back to bite them.

 

00;09;43;17 – 00;10;01;21

William

Teachers, I go to find a way to make students actually have to do the work themselves and not just have a bot do everything for them. 

Integrity in Today's World

Vicki

Thank you. William. William’s already graduated, and he and I talked last week, and I said, “Hey, will you give me a call?” He says, “Sure, I'll email you.” And he did. And he kept his promise, and it's a whole week after graduation.

 

00;10;01;21 – 00;10;20;28

Vicki

He's back here at the school, keeping his promises. And, you know, that's a piece of human intelligence that's really important, is keeping your commitments, having relationships with human beings. It's been great to interview you. I'm excited for this show to come out for people to see one of my many amazing students. And William, I'm really excited about your future.

 

00;10;20;28 – 00;10;36;28

Vicki

Yeah, I've already told you. You know, I think you're brilliant. I think you’re going to be extremely successful. And I hope you'll keep me updated. 

 

John

You've been listening to the Ten Minute Teacher podcast. If you like this program, you can find more at coolcatteacher.com. If you wish to see more content by Vicki Davis, you can find her on Facebook and Twitter @coolcatteacher

 

00;10;37;00 – 00;10;47;09

Thank you for listening.




Never Miss a Podcast Episode

Subscribe to get our podcast episodes by email.

Powered by ConvertKit
Facebook
Twitter
LinkedIn
Pinterest
Vicki Davis

Vicki Davis

Vicki Davis is a full-time classroom teacher and IT Director in Georgia, USA. She is Mom of three, wife of one, and loves talking about the wise, transformational use of technology for teaching and doing good in the world. She hosts the 10 Minute Teacher Podcast which interviews teachers around the world about remarkable classroom practices to inspire and help teachers. Vicki focuses on what unites us -- a quest for truly remarkable life-changing teaching and learning. The goal of her work is to provide actionable, encouraging, relevant ideas for teachers that are grounded in the truth and shared with love. Vicki has been teaching since 2002 and blogging since 2005. Vicki has spoken around the world to inspire and help teachers reach their students. She is passionate about helping every child find purpose, passion, and meaning in life with a lifelong commitment to the joy and responsibility of learning. If you talk to Vicki for very long, she will encourage you to "Relate to Educate" or "innovate like a turtle" or to be "a remarkable teacher." She loves to talk to teachers who love their students and are trying to do their best. Twitter is her favorite place to share and she loves to make homemade sourdough bread and cinnamon rolls and enjoys running half marathons with her sisters. You can usually find her laughing with her students or digging into a book.

All Posts »

2 comments

James Abela July 26, 2023 - 10:18 am

Yeah, I’ve noticed that socials are a bit all over the place. I’m using good old Flipboard to keep an eye on stuff. https://flipboard.com/@jamesabelaelt/ai-in-education-io30f0rjz?from=share&utm_source=flipboard&utm_medium=curator_share

Reply
Vicki Davis July 27, 2023 - 8:19 pm

I’m back to sharing on my flipboard magazines as well!

Reply

Leave a Reply

[script_19]

This site uses Akismet to reduce spam. Learn how your comment data is processed.

The Cool Cat Teacher Blog
Vicki Davis writes The Cool Cat Teacher Blog for classroom teachers everywhere
-
00:00
00:00
Update Required Flash plugin
-
00:00
00:00