Random freezing on Ubuntu

[mercurial-moon]

KoboldCpp 1.109.2 running on Ubuntu 24.04 desktop, also has OpenWebUI in docker.
Model is Qwen3.5 122B A10B UD IQ4

I'm having issue of KoboldCpp randomly freezing under specific conditions.

As long as users on the lan use OpenWebUI with KoboldCpp things work well, no issues whatsoever.
But as soon as one user start a custom workflow and sends requests to the OpenAI compatible endpoint and simultaneously some users are using OpenWebUI wierd things happen.
Some times Koboldcpp freezes and the server console looks like this

"max_completion_tokens": 20000, "stream": true, "stream_options": {"include_usage": true}}

[SmartCache RNN Match of 14705 tokens in slot 3. Already saved in slot 1, switching...]
state_read_data: reading state
state_read_data: - reading memory module

KV Load SaveState 3: Restored KV with 14705 tokens.

Processing Prompt (0 / 0 tokens)
Generating (4734 / 20000 tokens)
(EOS token triggered! ID:248046)state_write_data: writing state
state_write_data: - writing memory module
state_write_data: writing state
state_write_data: - writing memory module

KV Save State 7: Created SaveState of 19439 tokens, costing 391 MB.

[12:03:36] CtxLimit:19439/262144, Amt:4734/20000, Init:0.08s, Process:0.00s (infT/s), Generate:75.48s (62.72T/s), Total:75.48s       

and we have to shutdown Kobold (using ctrl -c) and then restart it.
So the output is mostly that stats line at the end and no answer text during the freeze.

Under some conditions the LLM spits out repetitive statements when are workflow is running.
I enabled debug mode but didn't see any errors there.

When I start Koboldcpp i see some errors

Enabled APIs: KoboldCppApi OpenAiApi OllamaApi
Starting Kobold API on port 5001 at http://localhost:5001/api/
Starting OpenAI Compatible API on port 5001 at http://localhost:5001/v1/
Starting llama.cpp secondary WebUI at http://localhost:5001/lcpp/
======
Please connect to custom endpoint at http://localhost:5001
Gtk-Message: 12:15:29.785: Not loading module "atk-bridge": The functionality is provided by GTK natively. Please try to not load it.
[16404, Main Thread] WARNING: GTK+ module /snap/firefox/6565/gnome-platform/usr/lib/gtk-2.0/modules/libcanberra-gtk-module.so cannot be loaded.
GTK+ 2.x symbols detected. Using GTK+ 2.x and GTK+ 3 in the same process is not supported.: 'glib warning', file /build/firefox/parts/firefox/build/toolkit/xre/nsSigHandlers.cpp:201

(firefox_firefox:16404): Gtk-WARNING **: 12:15:29.849: GTK+ module /snap/firefox/6565/gnome-platform/usr/lib/gtk-2.0/modules/libcanberra-gtk-module.so cannot be loaded.
GTK+ 2.x symbols detected. Using GTK+ 2.x and GTK+ 3 in the same process is not supported.
Gtk-Message: 12:15:29.849: Failed to load module "canberra-gtk-module"
[16404, Main Thread] WARNING: GTK+ module /snap/firefox/6565/gnome-platform/usr/lib/gtk-2.0/modules/libcanberra-gtk-module.so cannot be loaded.
GTK+ 2.x symbols detected. Using GTK+ 2.x and GTK+ 3 in the same process is not supported.: 'glib warning', file /build/firefox/parts/firefox/build/toolkit/xre/nsSigHandlers.cpp:201

(firefox_firefox:16404): Gtk-WARNING **: 12:15:29.851: GTK+ module /snap/firefox/6565/gnome-platform/usr/lib/gtk-2.0/modules/libcanberra-gtk-module.so cannot be loaded.
GTK+ 2.x symbols detected. Using GTK+ 2.x and GTK+ 3 in the same process is not supported.
Gtk-Message: 12:15:29.851: Failed to load module "canberra-gtk-module"

[LostRuins]

The errors seem to be unrelated, possibly just GNOME being wonky.

Could you provide steps to repro it? Preferably, copy a request payload (the terminal logs, which will show the API input) and share it here so that I can try to trigger the freeze.

But as soon as one user start a custom workflow and sends requests to the OpenAI compatible endpoint and simultaneously some users are using OpenWebUI wierd things happen.

An example of the custom workflow/payload would be helpful

[mercurial-moon]

Thanks, for the reply.. I'd need a bit more time to collect the logs, since I'm deep into a deadline that I have to meet... but as a temp fix to completing my tasks I'm considering to move to windows and test the setup over there and see if the same issue persists.

[henk717]

I actually expect the freeze is resource related. In that log you are using smartcache (I assume because its an RNN) and that does mean that for each snapshot it makes your ram usage increases especially if they are at high context.

Good place to start would be to monitor your (video) memory when this happens. The freeze might be a slowdown due to swapping.

[mercurial-moon]

I usually used KoboldCpp on windows. But lately shifted Ubuntu Desktop.
On windows I had not noticed the Smart Cache logs, even RNN, I thought its a now feature in 1.109
Also I just kept the default settings in KoboldCpp and just changed the config to set context and a few other values.
Yes when I start my workflow the input tokens would be high typically > 30K tokens.