Skip to content

Misc. bug: The web UI of llama-server is not displaying correctly. #13428

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ZUIcat opened this issue May 10, 2025 · 6 comments
Open

Misc. bug: The web UI of llama-server is not displaying correctly. #13428

ZUIcat opened this issue May 10, 2025 · 6 comments

Comments

@ZUIcat
Copy link

ZUIcat commented May 10, 2025

Name and Version

build: 5335 (d891942) with MSVC 19.43.34810.0 for x64

Operating systems

Windows

Which llama.cpp modules do you know to be affected?

llama-server

Command line

".\llama.cpp_release\llama-server.exe" -m ".\_model\Qwen3-30B-A3B-Q4_K_M__bartowski.gguf" -a "Qwen3-30B-A3B-Q4_K_M__bartowski" --host 0.0.0.0 --port 8090 --slots --props --metrics -np 1 -c 20480 -ngl 999 -ctk f16 -ctv f16 --no-mmap --keep 0 --cache-reuse 256  --jinja --chat-template-file ".\_model\Qwen3-30B-A3B-Q4_K_M__bartowski.jinja"

Problem description & steps to reproduce

A simple issue: This PR made some UI changes, but some buttons aren't displaying correctly on my end (though they remain clickable).

Image

Image

Image

First Bad Commit

No response

Relevant log output

".\llama.cpp_release\llama-server.exe" -m ".\_model\Qwen3-30B-A3B-Q4_K_M__bartowski.gguf" -a "Qwen3-30B-A3B-Q4_K_M__bartowski" --host 0.0.0.0 --port 8090 --slots --props --metrics -np 1 -c 20480 -ngl 999 -ctk f16 -ctv f16 --no-mmap --keep 0 --cache-reuse 256 --jinja --chat-template-file ".\_model\Qwen3-30B-A3B-Q4_K_M__bartowski.jinja"

load_backend: loaded RPC backend from ggml-rpc.dll
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon 780M Graphics (AMD proprietary driver) | uma: 1 | fp16: 1 | warp size: 64 | shared memory: 32768 | int dot: 1 | matrix cores: KHR_coopmat
load_backend: loaded Vulkan backend from ggml-vulkan.dll
load_backend: loaded CPU backend from ggml-cpu-icelake.dll
build: 5335 (d8919424) with MSVC 19.43.34810.0 for x64
system info: n_threads = 8, n_threads_batch = 8, total_threads = 16

system_info: n_threads = 8 (n_threads_batch = 8) / 16 | CPU : SSE3 = 1 | SSSE3 = 1 | AVX = 1 | AVX2 = 1 | F16C = 1 | FMA = 1 | BMI2 = 1 | AVX512 = 1 | AVX512_VBMI = 1 | AVX512_VNNI = 1 | LLAMAFILE = 1 | OPENMP = 1 | AARCH64_REPACK = 1 |

main: binding port with default address family
main: HTTP server is listening, hostname: 0.0.0.0, port: 8090, http threads: 15
main: loading model
srv    load_model: loading model '.\_model\Qwen3-30B-A3B-Q4_K_M__bartowski.gguf'
llama_model_load_from_file_impl: using device Vulkan0 (AMD Radeon 780M Graphics) - 16128 MiB free
llama_model_loader: loaded meta data with 41 key-value pairs and 579 tensors from .\_model\Qwen3-30B-A3B-Q4_K_M__bartowski.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv   0:                       general.architecture str              = qwen3moe
llama_model_loader: - kv   1:                               general.type str              = model
llama_model_loader: - kv   2:                               general.name str              = Qwen3 30B A3B
llama_model_loader: - kv   3:                           general.basename str              = Qwen3
llama_model_loader: - kv   4:                         general.size_label str              = 30B-A3B
llama_model_loader: - kv   5:                            general.license str              = apache-2.0
llama_model_loader: - kv   6:                       general.license.link str              = https://huggingface.co/Qwen/Qwen3-30B...
llama_model_loader: - kv   7:                   general.base_model.count u32              = 1
llama_model_loader: - kv   8:                  general.base_model.0.name str              = Qwen3 30B A3B Base
llama_model_loader: - kv   9:          general.base_model.0.organization str              = Qwen
llama_model_loader: - kv  10:              general.base_model.0.repo_url str              = https://huggingface.co/Qwen/Qwen3-30B...
llama_model_loader: - kv  11:                               general.tags arr[str,1]       = ["text-generation"]
llama_model_loader: - kv  12:                       qwen3moe.block_count u32              = 48
llama_model_loader: - kv  13:                    qwen3moe.context_length u32              = 32768
llama_model_loader: - kv  14:                  qwen3moe.embedding_length u32              = 2048
llama_model_loader: - kv  15:               qwen3moe.feed_forward_length u32              = 6144
llama_model_loader: - kv  16:              qwen3moe.attention.head_count u32              = 32
llama_model_loader: - kv  17:           qwen3moe.attention.head_count_kv u32              = 4
llama_model_loader: - kv  18:                    qwen3moe.rope.freq_base f32              = 1000000.000000
llama_model_loader: - kv  19:  qwen3moe.attention.layer_norm_rms_epsilon f32              = 0.000001
llama_model_loader: - kv  20:                 qwen3moe.expert_used_count u32              = 8
llama_model_loader: - kv  21:              qwen3moe.attention.key_length u32              = 128
llama_model_loader: - kv  22:            qwen3moe.attention.value_length u32              = 128
llama_model_loader: - kv  23:                      qwen3moe.expert_count u32              = 128
llama_model_loader: - kv  24:        qwen3moe.expert_feed_forward_length u32              = 768
llama_model_loader: - kv  25:                       tokenizer.ggml.model str              = gpt2
llama_model_loader: - kv  26:                         tokenizer.ggml.pre str              = qwen2
llama_model_loader: - kv  27:                      tokenizer.ggml.tokens arr[str,151936]  = ["!", "\"", "#", "$", "%", "&", "'", ...
llama_model_loader: - kv  28:                  tokenizer.ggml.token_type arr[i32,151936]  = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, ...
llama_model_loader: - kv  29:                      tokenizer.ggml.merges arr[str,151387]  = ["Ġ Ġ", "ĠĠ ĠĠ", "i n", "Ġ t",...
llama_model_loader: - kv  30:                tokenizer.ggml.eos_token_id u32              = 151645
llama_model_loader: - kv  31:            tokenizer.ggml.padding_token_id u32              = 151643
llama_model_loader: - kv  32:                tokenizer.ggml.bos_token_id u32              = 151643
llama_model_loader: - kv  33:               tokenizer.ggml.add_bos_token bool             = false
llama_model_loader: - kv  34:                    tokenizer.chat_template str              = {%- if tools %}\n    {{- '<|im_start|>...
llama_model_loader: - kv  35:               general.quantization_version u32              = 2
llama_model_loader: - kv  36:                          general.file_type u32              = 15
llama_model_loader: - kv  37:                      quantize.imatrix.file str              = /models_out/Qwen3-30B-A3B-GGUF/Qwen_Q...
llama_model_loader: - kv  38:                   quantize.imatrix.dataset str              = /training_data/calibration_datav3.txt
llama_model_loader: - kv  39:             quantize.imatrix.entries_count i32              = 384
llama_model_loader: - kv  40:              quantize.imatrix.chunks_count i32              = 209
llama_model_loader: - type  f32:  241 tensors
llama_model_loader: - type q8_0:   48 tensors
llama_model_loader: - type q4_K:  193 tensors
llama_model_loader: - type q5_K:   48 tensors
llama_model_loader: - type q6_K:   49 tensors
print_info: file format = GGUF V3 (latest)
print_info: file type   = Q4_K - Medium
print_info: file size   = 17.35 GiB (4.88 BPW)
load: special tokens cache size = 26
load: token to piece cache size = 0.9311 MB
print_info: arch             = qwen3moe
print_info: vocab_only       = 0
print_info: n_ctx_train      = 32768
print_info: n_embd           = 2048
print_info: n_layer          = 48
print_info: n_head           = 32
print_info: n_head_kv        = 4
print_info: n_rot            = 128
print_info: n_swa            = 0
print_info: n_swa_pattern    = 1
print_info: n_embd_head_k    = 128
print_info: n_embd_head_v    = 128
print_info: n_gqa            = 8
print_info: n_embd_k_gqa     = 512
print_info: n_embd_v_gqa     = 512
print_info: f_norm_eps       = 0.0e+00
print_info: f_norm_rms_eps   = 1.0e-06
print_info: f_clamp_kqv      = 0.0e+00
print_info: f_max_alibi_bias = 0.0e+00
print_info: f_logit_scale    = 0.0e+00
print_info: f_attn_scale     = 0.0e+00
print_info: n_ff             = 6144
print_info: n_expert         = 128
print_info: n_expert_used    = 8
print_info: causal attn      = 1
print_info: pooling type     = 0
print_info: rope type        = 2
print_info: rope scaling     = linear
print_info: freq_base_train  = 1000000.0
print_info: freq_scale_train = 1
print_info: n_ctx_orig_yarn  = 32768
print_info: rope_finetuned   = unknown
print_info: ssm_d_conv       = 0
print_info: ssm_d_inner      = 0
print_info: ssm_d_state      = 0
print_info: ssm_dt_rank      = 0
print_info: ssm_dt_b_c_rms   = 0
print_info: model type       = 30B.A3B
print_info: model params     = 30.53 B
print_info: general.name     = Qwen3 30B A3B
print_info: n_ff_exp         = 768
print_info: vocab type       = BPE
print_info: n_vocab          = 151936
print_info: n_merges         = 151387
print_info: BOS token        = 151643 '<|endoftext|>'
print_info: EOS token        = 151645 '<|im_end|>'
print_info: EOT token        = 151645 '<|im_end|>'
print_info: PAD token        = 151643 '<|endoftext|>'
print_info: LF token         = 198 'Ċ'
print_info: FIM PRE token    = 151659 '<|fim_prefix|>'
print_info: FIM SUF token    = 151661 '<|fim_suffix|>'
print_info: FIM MID token    = 151660 '<|fim_middle|>'
print_info: FIM PAD token    = 151662 '<|fim_pad|>'
print_info: FIM REP token    = 151663 '<|repo_name|>'
print_info: FIM SEP token    = 151664 '<|file_sep|>'
print_info: EOG token        = 151643 '<|endoftext|>'
print_info: EOG token        = 151645 '<|im_end|>'
print_info: EOG token        = 151662 '<|fim_pad|>'
print_info: EOG token        = 151663 '<|repo_name|>'
print_info: EOG token        = 151664 '<|file_sep|>'
print_info: max token length = 256
load_tensors: loading model tensors, this can take a while... (mmap = false)
load_tensors: offloading 48 repeating layers to GPU
load_tensors: offloading output layer to GPU
load_tensors: offloaded 49/49 layers to GPU
load_tensors:      Vulkan0 model buffer size = 17596.42 MiB
load_tensors:          CPU model buffer size =   166.92 MiB
....................................................................................................
llama_context: constructing llama_context
llama_context: n_seq_max     = 1
llama_context: n_ctx         = 20480
llama_context: n_ctx_per_seq = 20480
llama_context: n_batch       = 2048
llama_context: n_ubatch      = 512
llama_context: causal_attn   = 1
llama_context: flash_attn    = 0
llama_context: freq_base     = 1000000.0
llama_context: freq_scale    = 1
llama_context: n_ctx_per_seq (20480) < n_ctx_train (32768) -- the full capacity of the model will not be utilized
llama_context: Vulkan_Host  output buffer size =     0.58 MiB
llama_kv_cache_unified: kv_size = 20480, type_k = 'f16', type_v = 'f16', n_layer = 48, can_shift = 1, padding = 32
llama_kv_cache_unified:    Vulkan0 KV buffer size =  1920.00 MiB
llama_kv_cache_unified: KV self size  = 1920.00 MiB, K (f16):  960.00 MiB, V (f16):  960.00 MiB
llama_context:    Vulkan0 compute buffer size =  1344.00 MiB
llama_context: Vulkan_Host compute buffer size =    44.01 MiB
llama_context: graph nodes  = 3126
llama_context: graph splits = 2
common_init_from_params: setting dry_penalty_last_n to ctx_size = 20480
common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
srv          init: initializing slots, n_slots = 1
slot         init: id  0 | task -1 | new slot n_ctx_slot = 20480
main: model loaded
main: chat template, chat_template: {%- if tools %}
    {{- '<|im_start|>system\n' }}
    {%- if messages[0].role == 'system' %}
        {{- messages[0].content + '\n\n' }}
    {%- endif %}
    {{- "# Tools\n\nYou may call one or more functions to assist with the user query.\n\nYou are provided with function signatures within <tools></tools> XML tags:\n<tools>" }}
    {%- for tool in tools %}
        {{- "\n" }}
        {{- tool | tojson }}
    {%- endfor %}
    {{- "\n</tools>\n\nFor each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags:\n<tool_call>\n{\"name\": <function-name>, \"arguments\": <args-json-object>}\n</tool_call><|im_end|>\n" }}
{%- else %}
    {%- if messages[0].role == 'system' %}
        {{- '<|im_start|>system\n' + messages[0].content + '<|im_end|>\n' }}
    {%- endif %}
{%- endif %}

{%- set ns = namespace(multi_step_tool=true, last_query_index=0) %}
{%- for message in messages %}
    {%- if ns.multi_step_tool %}
        {%- if message.role == "user" %}
            {%- if not (message.content[0:14] == '<tool_response>' and message.content[-15:] == '</tool_response>') %}
                {%- set ns.multi_step_tool = false %}
                {%- set ns.last_query_index = loop.index0 %}
            {%- endif %}
        {%- endif %}
    {%- endif %}
{%- endfor %}

{%- for message in messages %}
    {%- if message.role == "user" or (message.role == "system" and loop.index0 != 0) %}
        {{- '<|im_start|>' + message.role + '\n' + message.content + '<|im_end|>\n' }}
    {%- elif message.role == "assistant" %}
        {%- set content = message.content %}
        {%- set reasoning_content = '' %}
        {%- if message.reasoning_content is defined and message.reasoning_content %}
            {%- set reasoning_content = message.reasoning_content %}
        {%- else %}
            {%- if '<think>' in message.content and '</think>' in message.content %}
                {%- set think_start = message.content.find('<think>') + 7 %}
                {%- set think_end = message.content.find('</think>') %}
                {%- set reasoning_content = message.content[think_start:think_end] %}
                {%- set content = message.content[think_end + 8:] %}
            {%- endif %}
        {%- endif %}
        {%- if loop.index0 > ns.last_query_index %}
            {%- if loop.last or (not loop.last and reasoning_content) %}
                {{- '<|im_start|>' + message.role + '\n<think>\n' + reasoning_content + '\n</think>\n\n' + content }}
            {%- else %}
                {{- '<|im_start|>' + message.role + '\n' + content }}
            {%- endif %}
        {%- else %}
            {{- '<|im_start|>' + message.role + '\n' + content }}
        {%- endif %}
        {%- if message.tool_calls %}
            {%- for tool_call in message.tool_calls %}
                {{- '\n' }}
                {%- if tool_call.function %}
                    {%- set tool_call = tool_call.function %}
                {%- endif %}
                {{- '<tool_call>\n{"name": "' + tool_call.name + '", "arguments": ' }}
                {%- if tool_call.arguments is string %}
                    {{- tool_call.arguments }}
                {%- else %}
                    {{- tool_call.arguments | tojson }}
                {%- endif %}
                {{- '}\n</tool_call>' }}
            {%- endfor %}
        {%- endif %}
        {{- '<|im_end|>\n' }}
    {%- elif message.role == "tool" %}
        {%- set prev_role = (loop.index0 > 0) and (messages[loop.index0 - 1].role) or '' %}
        {%- if loop.index0 == 0 or prev_role != "tool" %}
            {{- '<|im_start|>user\n' }}
        {%- endif %}
        {{- '<tool_response>\n' + message.content + '\n</tool_response>' }}
        {%- set next_role = (loop.index0 + 1 < messages|length) and (messages[loop.index0 + 1].role) or '' %}
        {%- if loop.index0 == messages|length - 1 or next_role != "tool" %}
            {{- '<|im_end|>\n' }}
        {%- endif %}
    {%- endif %}
{%- endfor %}

{%- if add_generation_prompt %}
    {{- '<|im_start|>assistant\n' }}
    {%- if enable_thinking is defined and enable_thinking == false %}
        {{- '<think>\n\n</think>\n\n' }}
    {%- endif %}
{%- endif %}
, example_format: '<|im_start|>system
You are a helpful assistant<|im_end|>
<|im_start|>user
Hello<|im_end|>
<|im_start|>assistant
Hi there<|im_end|>
<|im_start|>user
How are you?<|im_end|>
<|im_start|>assistant
'
main: server is listening on http://0.0.0.0:8090 - starting the main loop
srv  update_slots: all slots are idle
@ayestoro

This comment has been minimized.

@fivemeyestoreserver

This comment has been minimized.

@ngxson
Copy link
Collaborator

ngxson commented May 10, 2025

It's a browser-specific bug. You should try another browser, or try disabling plugins that may conflict with website's display

@ZUIcat
Copy link
Author

ZUIcat commented May 10, 2025

It's a browser-specific bug. You should try another browser, or try disabling plugins that may conflict with website's display

I'm using Microsoft Edge, which should be a widely used browser. I'm not familiar with web front-end, but I just tried something myself—for example, the regenerate button:

<button class="btn-mini show-on-hover w-8 h-8 flex items-center justify-center"><svg xmlns="http://www.w3.org/2000/svg" fill="none" viewBox="0 0 24 24" stroke-width="1.5" stroke="currentColor" aria-hidden="true" data-slot="icon" class="h-4 w-4"><path stroke-linecap="round" stroke-linejoin="round" d="M16.023 9.348h4.992v-.001M2.985 19.644v-4.992m0 0h4.992m-4.993 0 3.181 3.183a8.25 8.25 0 0 0 13.803-3.7M4.031 9.865a8.25 8.25 0 0 1 13.803-3.7l3.181 3.182m0-4.991v4.99"></path></svg></button>

The button appears when removing show-on-hover from its class.

@ZUIcat
Copy link
Author

ZUIcat commented May 10, 2025

Then I noticed that show-on-hover seems to have opacity: 0. After manually changing it to 1, several buttons became visible.

.show-on-hover {
    opacity: 0;
}

It seems like some "dynamic opacity change on hover" code isn't executing properly.

@ngxson
Copy link
Collaborator

ngxson commented May 10, 2025

Can you simply try other browsers to see it's the same? If it only happens on Edge, we can simply adds CSS rule to make the button always show on Edge

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

4 participants