Will MNN Chat app support ”Speculative Decoding“？ #3412

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Open

BeetSoup128 opened this issue May 1, 2025 · 1 comment

Labels

BeetSoup128 commented May 1, 2025

请问是否有计划下一步加入Speculative Decoding？MNN框架下的LLM拥有高度的一致性，相对大内存与较低的算力更适合双模型同时加载。

Collaborator

jxt1234 commented May 3, 2025

正在实现中，预计本月会支持

jxt1234 added Suggestion llm labels

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment