A first summary and recommendations after a few days of work

I notice I'm diving to deep into details about the game I'm converting to Javascript with ChatGPT.Maybe much is not so interesting for you, dear reader.

So here's a summary without such details, along with a few recommendations for working with it that might help you when doing the same.

Working on modular and independent classes is magic

ChatGPT is extraordinarily good at working on single classes that don't have too many dependencies to other classes. Tasks like creating new methods, either by converting a method from Java to Javascript, or by understanding a specification text and implementing it, are perfect for ChatGPT. On these tasks it performs on a superhuman level.

With the AI, I got stuff done in Javascript (where I'm a beginner) in 1/10th of the time that I get similar stuff done in Java (where I'm an expert). 

Limitation: it doesn't really think and comes up with patterns / algorithms that do the job but don't perform perfect and can be optimized easily by a human.

 

Example: let's assume you have a number of N cities with coordinates, and you have a new coordinate (x,y) and you want to find out what the closest city is.

 

If your N is not too large and you don't need great performance, just iterating the list of your cities, measuring the distance of each to your (x,y) and remembering the closest one does the job. Interestingly, ChatGPT then talks about euclidian distance and even computes it correctly as sqrt(x*x + y*y), but fails to understand that if this distance is doing the job, then x*x + y*y does the job just as well and you can spare N sqrt calls. 

 

Also it has to be said that sometimes the code it generates is fairly unusable and you have to explain more, which can be tedious.

Exploration for APIs is magic

I don't know how many hours of my life I have invested in figuring out where to find a certain API, how to use that API. This time is pretty much reduced to zero. With ChatGPT it is replaced by understanding why the API works the way it does and whether there are limitations.

An example: today I wanted to store a single property cross-session in my Javascript app, and I wanted to write code that checks whether a word that a user enters is an english word.

Before this would have taken me somewhere between 1 and 2 hours, I guess. Much stackoverflowing, looking for sites with APIs to check for words.

With ChatGPT it took 10 minutes incl. thorough testing. You just tell it what you want, it writes the code for you, you ask it why this code does what it should, you understand it. With the word API I also needed to check the datamuse API for limitations on usage, done.

Not only is 10 minutes less than 2 hours, the 2 hours are also not the most joyful time; it's simply not fun to dig into lengthy API docs, reading stackoverflow answers, so this is, for me, a game changer.

Complex class dependencies may be overwhelming it

NB this is about ChatGPT 3.5; I have not yet had the same effect with ChatGPT 4 yet.

When I had a good number off classes that were using similar concepts, it started confusing what is where and giving me code that didn't make sense, putting methods in the wrong class etc. The reason might also be that the conversation got longer and longer, maybe it loses control over it over time?

I learned from that that a beginner software developer might run into serious issues when trusting ChatGPT too much. YOU stay in charge. YOU are the "last line of defense". If you generate 10k lines with ChatGPT and all of a sudden it doesn't understand what it's doing, then there is noone but you to save the day.

So whatever you do, make sure that you understand what ChatGPT has generated for you.

Code reviews are cool

My experiments with telling ChatGPT to have a look at a class and offering suggestions on how to improve it were very positive. It finds stuff to improve that makes sense to improve. Do this.

Refactoring is a mixed bag

Stuff like pulling members up to a common base class is done easier with an IDE that supports refactoring. I had to do a lot of manual rework on ChatGPT's recommendations.

ChatGPT likes to work on single classes

If the best solution is to enhance a superclass of the class you want to change, then ChatGPT (4) typically misses this. I had a situation in which I could either overwrite 3 members of a class or add a setter method to a single member in its super-super class. The later approach is the right one, but ChatGPT didn't see that.

Learning a new programming language with ChatGPT is GREAT

Before, I learned a language in a fairly painful way. I read books or tutorials, started with a hello world, extended it by this and that, understood compiler errors, fixed stuff, understood in the debugger why that API wouldn't want my input, etc. 

Now I'm learning Javascript by asking the AI to do what I want, see how the code looks like, ask it about the code (e.g. "can you explain the difference between default export and named exports"), and I feel that my language skills are growing much faster and in a much less tedious and annoying way.

And as sometimes ChatGPT messes things up, I need to jump in, fix stuff by hand, ask it why that stuff didn't work as expected, get corrections, so I also learn the low level stuff.

I'm not sure if I could develop a Javascript app after that week with ChatGPT without the help of an AI; some of the syntax still is "read-only" for me; I can understand it but I can't re-create it. I'm also not sure if I will ever need this part.

Needed: repository integration

Overall, the whole thing is fun and makes me more productive. But what is badly missing is a single source of truth, a code repository that me and ChatGPT share and know, where ChatGPT does something like pull requests for its code changes and I review and update them, and where ChatGPT knows exactly what the code is like.

It's tedious to change a few lines in the code and tell ChatGPT about it. But if I'm not doing this, the AI and I are talking about different classes, with the expected consequences.

I've read that some IDEs are about to include co-pilots. This is the way to go.