Developing a Next.js App with Cline and Local LLM: Using Qwen2.5-Coder-32B-Instruct (Part 2)

Introduction

Who is this article for?

• Those interested in local LLMs
• Those interested in Cline
• Those interested in the performance of Qwen2.5-Coder-32B
• Those interested in app development using LLMs

Development environment for this app

1. Ubuntu 24.04.1 LTS (Core i5-8400 + 16GB Main Memory)
2. Mac Studio (M2 Ultra 128GB) # ollama server

Overview

Following up on the previous article, I will continue to verify how much app development is possible using Qwen2.5-Coder-32B-Instruct.

https://huggingface.co/Qwen/Qwen2.5-Coder-32B-Instruct

I have divided the content into the first and second parts as follows:

• Part 1 (Previous article)
  • Shaberi3 benchmark evaluation
  • Simple html + css + javascript app using Open WebUI
  • OSS version of Bolt.new

https://zenn.dev/robustonian/articles/qwen25_coder_32b

• Part 2 (This article)
  • Next.js app equipped with API and DB functions

So, this time I will challenge myself with more complex app development compared to the previous time.
Specifically, I will build a basic Next.js app featuring API and DB functions.

Also, I decided to use Cline, which became a hot topic about a week ago, as my development environment.
By the way, I learned about Cline from the following tweet by Kinopee-san.

https://x.com/kinopee_ai/status/1855959024505856413

Next.js App Development with Cline + ollama

Now, let's start development by installing Cline from the VS Code extensions.

Setting

• My development environment consisted of VS Code on Ubuntu and an ollama server on Mac, so I changed the Base URL and selected qwen2.5-coder:32b-instruct-q4_K_M-16k as the model.

As mentioned in the previous article, setting num_ctx too high makes prompt processing take longer, so I chose num_ctx = 16k.
The quantization size q4_K_M also seemed fine for now, so I proceeded with these settings.

Although it might slightly decrease performance, I'm not very confident in my English, so I added "Please respond in Japanese" to the Custom Instruction.

Also, I checked Always approve read-only operations. This setting is quite important because by default, it asks for permission every time it reads a file.

Ideally, I would like it to complete tasks fully automatically, so I want to approve all file writes and command executions as well, but I couldn't figure out how to do that.

If anyone knows how, I would appreciate a comment.

My usual editor is VS Code, but since Cline operates semi-automatically, it's difficult to do other work like writing an article while using it. Having two VS Code windows open is also confusing, so I ultimately decided to run Cline in Cursor (free version).

Task

Now, let's give it a task. Again, I'll make an unreasonable request.

Task

Make an interesting app using Next.js.

Below, I will explain step-by-step (within the scope of my understanding) how the app was built in response to this unreasonable request.

Phase 1: TODO App

First, it created a Next.js development environment with the project name ollama_app.

For the "interesting application," it seems it decided to create a simple TODO app.

As before, its ability to handle unreasonable requests doesn't seem very high, but I'll allow it for now.

The reason the project name is ollama_app is likely because I created a directory called ollama_db with the intention of making an app using ollama and a database, and started the conversation with Cline using that as the root.

Here is the resulting app. It's way too plain! 🤣

Phase 2: Adding a feature to display random quotes

So, I gave it some feedback. Activating the "power harassment prompt."

Note: Since it was an error I could handle myself, I manually added "use client";.

Cline's (Qwen2.5-Coder-32B) choice was to add a feature that displays random quotes.

An error caused by random numbers occurred here, but after providing the error message, it fixed it automatically.

Task Completed is displayed, and it seems to want to finish the task, but I won't let it (lol).

The result of the second phase is here. It has become a slightly more interesting app.

Phase 3

Since there is no database functionality in the second phase, all tasks disappear if the browser is closed or refreshed. So, I decided to have it add database functionality next.

I have almost no knowledge about databases, so I tried installing sqlite3 at first, but since I couldn't fully handle the errors, I eventually adopted better-sqlite3, which was suggested by Cline.

By this point, the time it takes to generate the next text has become quite long, so I was doing some manual debugging to some extent.

After a few rounds of back-and-forth, I managed to implement the database functionality for now.

Checking the db file with DB Browser for SQLite.

Phase 4

Next, I decided to refine the UI a bit more.

Just when I thought the errors had stopped, I was greeted with a screen like this... Is the CSS not being recognized?

I tried various things but reached a dead end, so I gave up on leaving it entirely to Cline and suggested introducing shadcn-ui.

Installing shadcn-ui sometimes fails even with Claude, and as expected, Qwen2.5-Coder-32B also failed, so I installed it myself instead.

After that, it couldn't properly add the button, input, and label components either, so I handled those as well.

I encountered errors like these.

$ npx shadcn-ui@latest init
sh: 1: shadcn-ui: not found

$ npx shadcn-ui@latest add button input label
sh: 1: shadcn-ui: not found

After a few more rounds of interaction, I was able to complete the app with an improved UI.

Phase 5

I could have finished at Phase 4, but I decided to go all the way and try integrating it with ollama. Since I felt that prompt processing was taking a considerable amount of time, I closed the thread and started with a new task.

After repeating the process while giving several instructions, I successfully created the Next.js app integrated with ollama and a database, which was my original goal!

Here is the final result.

Thoughts on Challenging App Development Using Cline + Local LLM

It's quite debatable whether I managed to create a truly useful app, but I was able to implement features like DB and ollama integration in a Next.js app without relying on Claude or ChatGPT. It took some time, but I think this is quite an achievement.

I'll jot down some notes and reflections from the process.

• Regarding development time and requirement definition

  • It likely took more than 5 hours to create this app.
  • If I had given tasks based on solid requirement definitions, it might have been built a bit faster.

• For practical use, you should use Claude unless you are in an environment where development can only be done locally

  • Claude 3.5 Sonnet has somewhat high API fees, but it's incredibly fast and requires fewer iterations. Plus, it produces high-quality results.
  • In the case of local LLM + Cline, you have to consider the electricity cost for long-term local LLM usage and the fact that it ties up human attention and time because you have to click "Approve" many times during development. (This would be somewhat resolved if fully automatic approval becomes possible.)
  • Given the above, there is almost no reason not to use Claude for actual app development.
  • At most, I feel it is useful if you want to develop apps in an environment handling highly confidential information, even if it takes more time.

• Opportunities for growth are provided

  • I found this to be an unexpected benefit, but in reality, I feel I learned more about Next.js app development while using Qwen2.5-Coder-32B + Cline than I did with Claude.
  • This is because Claude 3.5 Sonnet rarely stops due to errors and its output is very fast.
  • On the other hand, Qwen2.5-Coder-32B + Cline frequently encounters errors and its output is just slow. It's so slow that you feel like looking things up and fixing them yourself while waiting.
  • By using it repeatedly like that, you start to be able to act ahead of Cline to some extent. In other words, you grow yourself.
  • Conversely, once you can do yourself what Qwen2.5-Coder-32B + Cline can do, there will finally be little left to gain.
    → Once you reach this stage, unless you have plenty of time to kill, you should buy time and quality with API fees.

Summary

In this article, I covered the following:

• Process of Next.js app development using Cline and Qwen2.5-Coder-32B

  • Introduced the process from a basic TODO app to building an app with API and DB integration.

• Setup and precautions for the Cline development environment

  • Configuration of VS Code and ollama server, selection of quantization size, and optimization of prompt processing.

• Interactions with Cline and UI screen transitions at each development stage

• Comparison between local LLMs and cloud LLMs

  • Local LLM: Many learning opportunities but slow processing.
  • Cloud LLM: High speed and high quality but high cost.

• Insights gained through development

  • Skill improvement: Manual error correction as a learning opportunity.
  • Efficiency challenges: Considering time and quality, cloud LLMs should be used for practical work.

Although it takes time, it seems that developing a Next.js app with API and DB functionality is possible using Cline and Qwen2.5-Coder-32B for small-scale projects.
If everything can be approved automatically, the possibility of completing tasks fully automatically becomes realistic, so I hope for the addition of a "Full Auto Mode" option. In that case, I could try throwing tasks in at night and seeing the results later.

Well, if I were to ask for more, as shown in this article, it would be great if it could git commit after certain changes, save the deliverables at each stage, and summarize the production process in an easy-to-understand way.
This might be achievable just by tweaking Custom Instructions, but I'll try verifying that if I have another opportunity.

Thank you for reading until the end. I look forward to seeing you in the next post.