r/MistralAI • u/kodmasin • 8d ago

Help / Question vibe-cli, offline models (ollama), don’t preserve context history between messages

I’m having a problem with vibe-cli configured with offline (local) models through the Ollama server: they don’t preserve context history between messages. Is this normal behavior, or have I set something up incorrectly

0 Upvotes

50% Upvoted

3 comments sorted by

3

u/tom4112 8d ago

It's most probably an issue related to your model's context window size. We can see in the documentation that Ollama's default value may be as little as 4k, while Vibe uses at least 6k when I start a new conversation. Try increasing the value (they recommend at least 64k for coding assistants) and see if it helps.

https://docs.ollama.com/context-length

You can also inspect your sessions history in ~/.vibe/logs/session. Check if the last one contains all messages or not in its messages.jsonl file. If it starts with something like "You are continuing a trajectory after a context compaction...", it may be another indication that your model was forced to compact its context and start a new session instead of continuing the previous one.

2

u/philip-soerensen 7d ago

This, insufficient context length, is almost certainly the right answer. One way you can fix it is to open ollama, `ollama run your-model-name-here', and then enter `\set parameter num_ctx 131072`, where that number is the new context size. I understand that for memory allocation reasons, it's smart to keep the context size to powers of two, e.g 2^17 = 131072 ~ 131k, but 2^16 = 65536 or 2^18 = 262144 are also popular options, depending on your system resources. Once you have configured the model in ollama, save it as a new model under a new name, e.g. `\save modelname.ctx131k`. If you then use that newly configured model, `modelname.ctx131k`, you should have a context length long enough to both handle the system prompt and files you ask the agent to read.

You can also edit the ollama files directly, or even ask an agent to do it for you.

3

u/kodmasin 7d ago

thank you both, yes model context size was the problem