LLM Session Chat

Persistent multi-turn chat with local GGUF models using file-based history.

By kantan-kanto·Created 7 months ago·Updated about a month ago· 27

LLM Session Chat

media

assistant_text

◄user_text►

◄session_iddefault►

◄model(No GGUF models found in models/LLM/)►

◄mmproj(Auto-detect)►

◄system_promptYou are a helpful assistant.►

◄max_tokens512►

◄temperature0.70►

◄top_p0.90►

◄n_gpu_layers0►

◄n_ctx4096►

◄persistent_cacheoff►

◄runtime_cacheLlamaTrieCache►

◄log_leveltiming►

◄suppress_backend_logstrue►

◄repeat_penalty1.12►

◄repeat_last_n256►

◄rewrite_continuetrue►

◄max_turns12►

◄summarize_old_historytrue►

◄summary_chunk_turns3►

◄max_tokens_summary128►

◄summary_max_chars1500►

◄dynamic_max_tokenstrue►

◄min_generation_tokens96►

◄safety_margin_tokens64►

◄history_dir►

◄reset_sessionfalse►

◄stream_to_consolefalse►

◄enable_thinkingfalse►

CategoryLLM/Session

Inputs (30)

Name	Type	Default	Description
user_text	STRING		User message for this turn
session_id	STRING	default	Session ID (maps to a history file). Same ID continues the chat.
model	COMBO	(No GGUF models found in models/LLM/)	GGUF model file in models/LLM/
mmproj	COMBO	(Auto-detect)	Manual selection is recommended.
system_prompt	STRING	You are a helpful assistant.	System prompt (conversation policy). Saved into the history file.
max_tokens	INT	5121–32768	Maximum tokens to generate for this turn
temperature	FLOAT	0.700–2	Sampling temperature
top_p	FLOAT	0.900.05–1	Nucleus sampling (top_p). Lower = safer/more conservative.
n_gpu_layers	INT	0-1–200	Number of layers to offload to GPU. 0=CPU. -1=all.
n_ctx	INT	4096512–131072	Context length (must be supported by the model)
mediaopt	*		Optional IMAGE tensor/batch or AUDIO input for this turn only (never saved to history)
persistent_cacheopt	COMBO	off	Persistent cache backend. LlamaDiskCache stores cache data under a session-specific cache directory in output/llm_session_sessions/cache/.
runtime_cacheopt	COMBO	LlamaTrieCache	Runtime cache backend. KV_cache uses save_state/load_state, RAM/Trie use llama.cpp cache in memory.
log_levelopt	COMBO	timing	Console logging verbosity for LLM Session Chat.
suppress_backend_logsopt	BOOLEAN	true	Suppress backend stdout/stderr during generation.
repeat_penaltyopt	FLOAT	1.121–2	Repetition penalty to reduce looping outputs (especially on continue).
repeat_last_nopt	INT	2560–4096	Apply repeat_penalty over the last N tokens. 0 disables.
rewrite_continueopt	BOOLEAN	true	Rewrite inputs starting with 'continue' into an explicit continuation instruction to reduce repetition.
max_turnsopt	INT	120–200	Keep only the last N turns in live context. 0 means no prior turns.
summarize_old_historyopt	BOOLEAN	true	Summarize overflow turns into a rolling summary when turns exceed max_turns.
summary_chunk_turnsopt	INT	31–50	Summarize overflow in chunks of this many turns (reduces summary frequency).
max_tokens_summaryopt	INT	12816–2048	Max tokens for summary generation (kept small for speed).
summary_max_charsopt	INT	1500200–20000	If the rolling summary exceeds this size, it will be re-summarized to stay compact.
dynamic_max_tokensopt	BOOLEAN	true	Dynamically shrink max_tokens (and/or turns) when prompt would exceed n_ctx.
min_generation_tokensopt	INT	961–4096	Minimum tokens to allow for generation when dynamic_max_tokens is enabled.
safety_margin_tokensopt	INT	640–2048	Token margin reserved to reduce the chance of exceeding n_ctx.
history_diropt	STRING		Optional directory for history files and session-scoped disk caches. Empty uses output/llm_session_sessions/
reset_sessionopt	BOOLEAN	false	If true, overwrite existing session history file with a fresh session. Session disk cache is kept.
stream_to_consoleopt	BOOLEAN	false	Stream tokens to console while generating.
enable_thinkingopt	BOOLEAN	false	Enable model thinking/reasoning output for supported chat formats.

Outputs (1)

Name	Type	Description
assistant_text	STRING	—