Adapting Ollama Models for Cline Compatibility

There are two challenges in getting Ollama models to work with Cline.

The first is the input context limit. In many default Ollama model settings, Cline's system prompt gets truncated, so increasing the num_ctx parameter is necessary. While there are plans for Cline to dynamically provide Ollama API parameters, for now, one workaround is to build the model yourself.

The second is the prompt format mismatch. Cline requires a unique XML format for inputs and outputs; Tools must be defined in the system prompt, and the model's response needs to include the Tool calls and their arguments. You can address this by overwriting the TEMPLATE definition in Ollama to comply with Cline's specifications.

As an example, I adjusted llama3.1:8b.

Here, I made the following changes:

• Changed PARAMETER num_ctx from 2048 to 20480.
• Added # Tools and <tool_call> blocks to the TEMPLATE.
  The TEMPLATE format uses the Go template syntax, and descriptions of each directive can be found here:
  https://github.com/ollama/ollama/blob/main/docs/template.md

To use this, build it with the following command:

llama create llama3.1-cline:8b -f ./lama3.1-cline-8b.Modelfile

If you want to use models other than llama3.1:8b, the basic approach remains adjusting this TEMPLATE section. However, you must base it on a model that supports Tool calling.
To view the original TEMPLATE data for a model, run the ollama show command.

❯ ollama show qwen2.5:latest --template
{{- if .Messages }}
{{- if or .System .Tools }}<|im_start|>system
{{- if .System }}
{{ .System }}
{{- end }}
{{- if .Tools }}

...

Use this content as a reference to write your Modelfile.
To see the system prompt that is actually being sent, start the server with OLLAMA_DEBUG=1 ollama start.

APPENDIX: Cline's Architecture

1. Task Analysis: Suppose a user gives a task like "Read and display the contents of the src/main.js file." Cline first analyzes the task and selects tools to gather the necessary information.
2. Tool Selection: The model determines that the read_file tool is best suited for this task.
3. Parameter Setting: The model determines that the path parameter should be src/main.js, as provided by the user.
4. Tool Execution: Cline parses the tool call in XML format within the model's response.

   <read_file>
     <path>src/main.js</path>
   </read_file>
   

5. Waiting for Results: Cline executes the read_file process to retrieve the file contents.
6. Result Processing: After receiving the file content, Cline processes it and uses the attempt_completion tool to present the result to the user.
7. Completion: Cline notifies the user that the task is complete and provides additional commands if necessary.

 <attempt_completion>
  <result>The content of the src/main.js file is as follows: {file content}</result>
 </attempt_completion>

Once you understand this operating principle, you can investigate the execution flow for other tasks using the following system prompt definitions:
https://github.com/cline/cline/blob/a7e9d473752c668d7040d2f6ff9e03123c3334d6/src/core/prompts/system.ts

References

https://github.com/cline/cline/discussions/241
https://zenn.dev/kun432/scraps/f87398fffd51a7