Grumpy Gamer

I’ve been very critical of AI but have never really used it in depth and I feel that needs to change. Don’t criticize what you don’t know.

I’m going to ignore the moral, ethical and privacy implementation of AI and just focus on the practical. Morally and ethically, AI is a train wreck. But I’m not going to focus on that.

I’m also going to only focus on AI for programming as that is my area of expertise, at least in this case.

I’ve done a lot of looking into how LLM’s work and I get the basics in the same way that I understand quantum physics. Enough to make me entertaining at a cocktail party, but not much after that.

I am starting a new game pro-type project using Raylib and it feels like this is a good place to experiment with AI. So for the next 30 days I’m going to use AI for programming and see how it goes.

I’ve been programming C since 1987 and C++ since 1993 so I have a lot of experience (although const in all but it’s basic form still messes me up, but I thinks that true for a lot of people).

I am using Claude AI (basic subscription) cli for general programming, Co-pilot in VSCode and Gemini for quick unrelated to programming questions.

Gemini

I was only using Gemini on the free trier that everyone with a Goggle account gets and it was worthless for programming, so we’ll skip it.

Copilot

Copilot comes integrated with VSCode, so it was easy to try out and doesn’t cost anything (I already pay for Github).

Copilot treads this line between being somewhat useful and maddeningly annoying.

Copilot is good at realizing I’m going to make the same small change to the next 10 lines and offer (mostly correctly) to make it for me. It’s not making big AI decision about my code base, just that I’m going to add Ox in front of a list of ints or that I’m going to fill out an enum. Good stuff. I do find that useful in the same way Intellisense can be useful.

What Copilot is not good at is making larger AI decisions about what I’m trying to add right after the if(. I’ve found it to be mostly wrong. It’s double or triple irritating that it interrupts my flow by popping up a huge block of text that is wrong with the excitement of a poorly trained intern on their first day.

I wish I could keep the first part and disable the second, but I have not found a way to do that. What it needs is a char limit to the amount of new text it will try and insert. If it’s about 20 characters then STFU.

Claude Code

Again, ignoring the ethics of the company that make Claude Code.

What Claude code is good at is doing rote things that don’t need a lot of (ironically) “Intelligence”.

I asked it to make be a .yml file for Github actions for building Mac, Linux and Windows version of my game and it did a 99% right version. Only thing wrong was the way it named the executable, but anyone could make that mistake. Maybe I could have been clearer.

Which brings me to a point. The clearer you are the better AI is. But at some point I’ve spent so much time writing the spec that I could have just done it myself. And it’s not just a high level spec, you have to get into the weeds.

I asked Claude Code to write a .py program that did texture packing, taking a folder of .png files and making a sprite sheet with a .json file describing where everything was. Basically what TexturePacker does.

After a few seconds I had to stop it because it was freely pulling in python packages that I didn’t have.

I then told it to use Pillow (PIL) and nothing else that wasn’t standard.

After a few minutes it had a runnable program, but not correct. It was putting the .json and sheet .png in different places despite me tell it they should be together. It was also processing the .png files twice, once to check for changes and another before adding them. I called it out and it apologized profusely and fix it.

There were several other instances where it wrote c++ code that was technically correct, but horrible inefficient like passing std::string around when it could have done std::string &. I caught this, but a junior dev might not have.

I had it write some c++ code to read the sprite sheet it had created and it took 5 failed attempts to get the origins right for rotation and scaling. I finally fixed it myself but it was a frustrating back and forth. Are new devs being trained to think this is normal?

I also had it write code so billboard sprites in my 3D particle system were always camera-facing. It never got that right and had generated such mess of code that I just scrapped the idea. Math is not my strong point so I didn’t feel like I could just fix it.

I also had a instance where a file was being read from the wrong path and instead of prepending the right path it tried to completely rewrite my library.

Ironically it also had a problem with const. It recompiled the program three times randomly changing where const appeared. I feel for ya.

I have spent a lot time over this experiment correcting AI.

I could go on and on. But I won’t.

Conclusions

AI for coding is very impressive. It is a lot more then a fancy auto-complete.

But it can also be very wrong and you don’t really know it unless you look closely and have the experience to do that and I fear that new devs won’t ever get that experience. I hear about new devs freaking out when they run out of tokens for AI. They are lost without it.

I have a few friends who work at companies where their managers are telling them to use AI. This isn’t coming from “it’s a useful tool” place, but rather that AI feels like magic to some managers who don’t program.

Someone once said that “AI feels amazing when it’s talking about a subject you know nothing about, but is laughably wrong when you do”. I couldn’t agree more.

I doubt I’ll use AI beyond this 30 day experiment. I spend too much time correcting it plus I enjoy programming.

Someone else also said “I want AI to do my laundry so I can make Art, not make my Art so I can do laundry”.

I also worry that AI was trained on real programmers, but now it will just be training on itself. There is no good end to that story.

I blame the I in AI. It’s not Intelligent and it’s important to remember that. It’s a really good encyclopedia of (stolen) knowledge but AI is not figuring things out on it’s own.

Autopilots in modern planes are amazing. They can take off and land the plane. But pilots are also legally requited to hand fly the plane several times a month. Training requires them to hand fly the simulators or simulators with broken autopilots. Pilots use the auto pilot as a tool not a crutch. It’s also not called a AI Pilot (yet).

More conclusions

AI will get better. It might even get as good as we see in Star Trek. I know I felt like Geordi programming the computer in engineering at times.

I grew up in a time where everyone was afraid computers would take their jobs. And they were right. Computers did. There are fewer longshoremen because of automated ports, short-haul commercial flights don’t need three pilots anymore, there aren’t rooms full of accountants hand writing figures into a book.

That will happen with AI.

Let’s just hope it also doesn’t make us too stupid in the process.

Of course, someone from 1793 would say the same thing about me as I could not survive the night in the wilderness.

In the meantime, I’m going to enjoy programming and not being a baby sitter to an AI programmer.