llm 支持输入参数为input_embeds #3509
Open
llm 支持输入参数为input_embeds #3509
+47
−2
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
本次以新增接口的方式修改的。
hf transformer的Qwen2的forward函数 参数中既支持input_ids形式，同时也支持inputs_embeds 形式。 这样其实通用性更高。 比如我的这个场景： 我自定义模型 VIT + LLM， VIT部分的输出给LLM作为输入时，已经是inputs_embeds的结果了而不是input_ids， 但是咱们的MNN的LLM的forward接口是没有这个参数的
hf transformer这两个参数使用方式和逻辑都是很清晰的，我想参考hf代码，优化下咱们MNN的forward接口