Unlocking AI Coding Assistants Part 2: Generating Code

AI coding assistants can help you build working code faster, eliminate manual repetition, and even propose solutions you might not have considered. In this blog, we'll explore how AI tools can become a powerful coding ally, saving you time, boosting creativity, and making your work smoother and more efficient. Enjoy!

Introduction

This article is the second in the series, with emphasis on generating code. The first part can be read here: "Unlocking AI Coding Assistants Part 1: Real-World Use Cases."

Similar to Part 1, some tasks are executed with the help of an AI coding assistant and the responses are evaluated. Different techniques are applied, which can be used to improve the responses when necessary. 

The tasks are executed with the IntelliJ IDEA DevoxxGenie AI coding assistant.

Two setups are used:

1. Ollama as inference engine and qwen2.5-coder:7b runs as a model. This runs on CPU only.
2. LMStudio as inference engine and qwen2.5-coder:7b runs as a model. This runs on GPU only.

The sources used in this blog are available at GitHub.

Prerequisites

Prerequisites for reading this blog are:

• Basic coding knowledge.
• Basic knowledge of AI coding assistants.
• Basic knowledge of DevoxxGenie. For more information, you can read my previous blog, ''DevoxxGenie: Your AI Assistant for IntelliJ IDEA," or watch the conference talk given at Devoxx. 

Task: Generate Javadoc

The goal is to see whether AI can be helpful in generating javadoc.

The setup LMStudio, qwen2.5-coder, GPU is used.

The source file Refactor.java does not contain any javadoc.

Prompt

Open the file and enter the following prompt.

write javadoc for public classes and methods in order that it clearly explains its functionality

Response

The response can be viewed here.

Response Analysis

The response is quite good and useful:

• The generated javadoc is correct.
• The javadoc for the methods could be a bit more elaborate.
• A constructor is added, which was not asked to do so.
• The prompt asked to write javadoc for public classes and methods only, but this instruction is ignored.

Task: Generate Names

The goal is to see whether AI can be helpful with generating names for classes, methods, variables, etc.

The setup LMStudio, qwen2.5-coder, GPU is used.

The source file Refactor.java contains some names, which can be improved.

Prompt

Open the file and enter the following prompt.

give 3 suggestions for a better method name for the parseData methods

Response

The response can be viewed here.

Response Analysis

The suggestions are quite good and an improvement over the current names.

Prompt

Let's find out whether a better name for the class can be found.

Open the file and enter the following prompt.

give 3 suggestions for a better class name for the Refactor.java class

Response

The response can be viewed here.

Response Analysis

The suggestions are quite good, but also some Example Usage are given, which wasn't really asked to do so.

Prompt

Enter the same prompt, but change the temperature to 0.7 in the DevoxxGenie large language model (LLM) settings. This instructs the LLM to be more creative.

Open the file and enter the prompt.

give 3 suggestions for a better class name for the Refactor.java class

Response

The response can be viewed here.

Response Analysis

For some reason, the response is very short now. But the suggestions seem to be better than with a temperature of 0.

Task: Generate Docker Compose File

The goal is to see whether AI can be helpful in converting a docker run command into a Docker Compose file.

The setup Ollama, qwen2.5-coder, CPU is used.

Open WebUI, an offline AI interface, can be started by means of the following docker command.

docker run -d -p 3000:8080 -v open-webui:/app/backend/data --name open-webui ghcr.io/open-webui/open-webui:main

Prompt

Open the file and enter the prompt.

create a docker compose file for this command

Response

The response can be viewed here.

Response Analysis

The response is correct. Besides that, an explanation is given about the different components in the Docker Compose file and how it can be executed.

Task: Generate Cron Expression

The goal is to see whether AI can be helpful in generating Spring Scheduling Cron expressions. Note that Spring Scheduling Cron expressions differ from regular crontab expressions. An extra component in order to define the seconds is available in Spring Scheduling Cron expressions.

The setup LMStudio, qwen2.5-coder, GPU is used.

Prompt

Enter the prompt.

generate a spring scheduling cron expression which runs every 3 days at 0 AM, but not on Sundays

Response

The response can be viewed here.

Response Analysis

The LLM does know how to format a valid Spring Scheduling cron expression. The cron expression will run always on Monday, Wednesday, Friday and Saturday. That is not what we meant, because when it runs on Friday, it should not run on Saturday.

Prompt

Write a follow-up prompt.

It will run always on Friday and Saturday, this is not every 3 days. The task should only run every 3 days, excluding (thus skipping) Sundays.

Response

The response can be viewed here.

Response Analysis

No real improvement, the expression has changed in such a way that it now runs every three hours. The explanation the LLM gives, does not correspond to this new cron expression.

Prompt

This challenge is quite difficult because it is a tricky one: you want to run it every three days, but not on Sundays. But what to do when the next schedule would run on Sunday? Do you want it to just skip Sunday or do you want it to run the next day (meaning Monday)?

Let's try another prompt indicating that Sundays should be skipped. Create a new chat window in order to start anew.

generate a spring scheduling cron expression which runs every 3 days at 0 AM, Sundays should be skipped.

Response

The response can be viewed here.

Response Analysis

This response is correct this time. And it also shows the problem it was struggling with: it depends in this case which day you want to start, so the LLM is able to create a cron expression, which satisfies the requirements.

The conclusion is that it does matter which words, phrasing you use in the prompt and sometimes it is better to start anew with a new prompt instead of iterating on previous responses.

Task: Refactor Code

The goal is to see whether AI can be helpful with refactoring code.

The setup LMStudio, qwen2.5-coder, GPU is used.

Take a look at the processMessage method in the Refactor.java class. This code screams for refactoring. Let's see how AI can help us with that.

Prompt

Add the entire code directory to the Prompt Context. Enter the following prompt.

DevoxxGenie will expand this to the following prompt.

Review the selected code, can it be improved or are there any bugs?

Response

The response can be viewed here.

Response Analysis

Some general improvements are suggested, but overall the code did not improve a lot.

Prompt

Open a new chat window and add the source directory as Prompt Context again. Provide more clear instructions of what you expect. This is based on AI-Assisted Software Development.

Open a new chat window. Add the entire code directory to the Prompt Context. Enter the following prompt.

Please review the following code for quality and potential issues: Refactor.processMessage(RefactorMessage refactorMessage) 
In your review, please consider: 
1. Code style and adherence to best practices 
2. Potential bugs or edge cases not handled 
3. Performance optimizations 
4. Security vulnerabilities 
5. Readability and maintainability 

For each issue found, please: 
1. Explain the problem 
2. Suggest a fix 
3. Provide a brief rationale for the suggested change 

Additionally, are there any overall improvements or refactoring suggestions you would make for this code?

Response

The response can be viewed here.

Response Analysis

This response seems to be better:

1. It is seen that the method is too long and too complex—this is a good conclusion.
2. The input variable refactorMessage should be checked for null—this is correct.
3. The LLM suggests to use a DateTimeFormatter to parse the date, but in this specific situation, this is not applicable. The occurrenceTime is a range separated by a slash, e.g. "2014-03-01T13:00:00Z/2015-05-11T15:30:00Z."
4. It suggests to validate the input data, which is a good suggestion.
5. A suggestion is given to use early returns, but the fix given is already present in the code.
6. Also, overall improvements and suggestions are given (e.g. to write unit tests).

Prompt

The most interesting suggestion is to break the code down into smaller, more manageable methods. However, no real fix was suggested. So, let's enter a follow-up prompt.

Break the method down into smaller more manageable methods

Response

The response can be viewed here.

Response Analysis

The suggested code is not entirely correct.

1. findSingleData and findMultiData are not exactly improvements.
2. parseOccurrenceTime is not parsing the same way as in the original code.
3. parseLocation makes uses of a constructor with arguments, which do not exist.
4. parseData makes also use of constructors with arguments, which do not exist.

However, the overall simplification and readability improvements do make sense.

Prompt

Let's use Anthropic Claude 3.5 Sonnet to verify whether a cloud LLM is able to produce better results.

Open a new chat window and add the entire code directory to the Prompt Context. Use the same prompt as before.

Please review the following code for quality and potential issues: Refactor.processMessage(RefactorMessage refactorMessage)
In your review, please consider:
1. Code style and adherence to best practices 
2. Potential bugs or edge cases not handled 
3. Performance optimizations 
4. Security vulnerabilities 
5. Readability and maintainability

For each issue found, please:
1. Explain the problem 
2. Suggest a fix 
3. Provide a brief rationale for the suggested change

Additionally, are there any overall improvements or refactoring suggestions you would make for this code?

Response

The response can be viewed here.

Response Analysis

Some good suggestions are given:

1. Complex conditional logic—this is correct and should be changed.
2. The method is too long and should be split into smaller methods—that is correct.
3. Some problems with null references and input validation are found—that is correct.
4. Some overall suggestions are given.

Overall, the suggestions are of a similar level compared to a local LLM.

Overall Conclusion For Task Refactor Code

AI can give good suggestions and improvements for existing code. However, the code cannot be taken as-is into your code base. It is better to apply some of the suggestions yourself and prompt again to see whether the code has improved.

The suggestions between a cloud provider or a local LLM running on a GPU do not differ very much from each other.

Conclusion

From the examples used in this blog, the following conclusions can be drawn:

1. AI coding assistants are good at generating javadoc, generating names, small coding tasks.
2. By default, it is wise to set the temperature to 0 for coding tasks (0 instructs the LLM to be factual). However, for more creative tasks like generating names, it is advised to increase the temperature in order to get better results.
3. Iterating on a previous prompt provides more context to the LLM and provides better results.
4. Changing words can make a difference (see the Spring Scheduling Cron expression example) when prompting. Sometimes it is better to start anew (without chat history) but with different phrasing in order to get better responses.
5. Start with short prompts is often an easy way to interact with an LLM. But do not hesitate to give clear instructions about what you expect in a response. Some good software development prompts can be found in AI-Assisted Software Development.
6. Always review and understand the responses. Suggested code fixes are often not entirely correct but do point you in the right direction.
7. Nowadays, local LLMs are able to generate responses closely to the level of cloud LLMs.