deepseek-r1-awq overflow #3180

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Open

3 of 4 tasks

taishan1994 opened this issue Apr 16, 2025 · 0 comments

taishan1994 commented Apr 16, 2025 •

edited

Loading

System Info

tgi:3.2.3

Information

Docker
The CLI directly

Tasks

An officially supported command
My own modifications

Reproduction

Input a long text, concatenate the contents of user and assistant
truncate part of the text. Set the length of the output truncated text.
The model directly outputs "", and the finish_reason shows the reason as "length".

Expected behavior

It will also occur.: The model suddenly starts giving random answers during the inference process.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment