A few days ago I had a little problem. I had a text file with a lot of lines I didn’t want. Specifically, it was a computer generated transcript with timing on just about ever other line. It made reading the document harder and didn’t provide me with information I needed or wanted. So I decided to clean it up. Obviously, I could write some pretty trivial code to do it for me.
This being the age of AI, I thought I would let AI write it for me. I use Visual Studio (the full blown version) and GitHub Copilot is available to me there. I thought about asking it something like this:
Create some code to read a file and create a new file that doesn't;t include lines containing a user specified word or string.
That's the sort of open ended instruction I might give students.I might use something like that in Advanced Placement CS A if I were still teaching it. It's fine as far as it goes. More on that prompt later in the post by the way.
I wanted something a bit more specific. Something that worked the way I like to work. That meant a Windows Forms application. I decided to give AI what I would include as scaffolding for a student project. Some easy steps to provide a more gentle path for students. Or in this case, something particular I wanted to see.
What I gave GitHub Copilot was:
Create a C# method that:
1. Uses a dialog box to request and save an input file name
2. Uses a dialog box to request and save an output file name
3. Copies a string from a text box
4. Opens the input file and output file
5. Reads the input file and copies lines that do not include the saved string
6. Closes both files when finished reading the input file
7. Displays the message “Completed” in a message box
This gave me exactly what I wanted. That’s not a surprise really. AI tends to be good at simple projects like this. The thing I realized is that the prompt worked well and I got the results I wanted because I knew what to ask. I easily broke the task down into specific steps and was explicit about what I wanted. Yea me.
After I got my file cleaned (the reason I started this) I went back to make the program more flexible. For example, what if I wanted to keep lines that included a specific string rather than skip it?
The AI in Visual Studio was very helpful here. Or rather it tried to be helpful. It suggested all sorts of code based on the assumptions it was making about my intent. It got things wrong as often as it got things correct. Maybe it would have done better if I gave it specific prompts but maybe it wouldn't. One thing is clear though. I knew enough code to know that it was wrong and how to fix it. Experience matters.
Coming back to my initial prompt. The AI did a very good job. It does everything I asked an more. All sorts of error checking (file exist?) and even asks if the user wants the comparison of stings to be case sensitive or not. As with the other prompt, the code is nicely commented. It created a console app which is not my favorite way to work but is the way many, perhaps most, student projects are created.
I can’t help but think about what all this means in the context of CS education.
Could students use these prompts to have AI write code for them? Absolutely. Would the results be obvious to someone grading the work that the code was not student written? Also, I think, pretty obviously. The comments alone would be a good clue. The error handling, something we tend not to cover very well because of time constraints would be an even stronger clue. Yes, I have had students add error handling but when they do it tends not to be unexpected. We’ve all had those students who do more. Often they have experience prior to coming to class.
So what is an educator to do? Ban the use of AI! Have you met today’s students? They are past masters of getting around the rules. I had students from China who laughed, literally laughed, at the attempts of a national government to limit what people do on the Internet.
I think we can ask students not to use AI to complete projects. Some will listen. Some will not. In either case, I think it is up to us to make the case that they need the experience of writing their own code. Every professional developer I talk to knows that their prior experience allows them to use AI more effectively. The people making the most effective, productive results using AI have written code on their own.
Here is what I think is the most important skill in software development today. Being able to analyze a problem and break it down into its component pieces. Compare the two prompts I used above. Admittedly, this is a simple problem and it is easy to break down. Which one is easy to produce code for? Which one is more likely to solve the problem in a way that the user wants to operate?
If a student cannot break down a “toy” problem like those we tend to assign in early courses how can they be expected to break down larger problems professionally? Or even non-professionally to solve problems in their career.
Circling back to the tweaks I made to AI generated code for a minute. Being able to read and understand code is key to understanding where code went wrong. AI tools have their own assumptions and they are not mind readers. Communication, between user and AI just like between people, requires some level of common understanding. Programming languages are the languages of computer science.
I am reminded of stories I have read about people with tattoos in languages they don’t understand later finding out that the tattoo doesn’t say what they wanted it to say. We had better keep current and fluent in computer languages unless we want computers to get totally out of control.
