-
Notifications
You must be signed in to change notification settings - Fork 365
Add english prompts for GEM/wikilingua (18 languages) #765
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add english prompts for GEM/wikilingua (18 languages) #765
Conversation
@haileyschoelkopf I can assist the Indonesian |
@afaji That would be great! Thank you |
https://github.com/haileyschoelkopf/promptsource/pull/1/files <- Indonesian prompt for wikilingua |
Looks good! Should the |
I could definitely make that change if you think it's better! My idea was to have a prompt that didn't indicate the language (since these dataset splits all have the same source and target language) to see if it degraded performance when summary language wasn't explicitly specified, but that might not be desirable for the hackathon evals. |
Yes, I think we should change it (or remove if that makes the prompt redundant with others?) It's a potentially interesting comparison to make, but balanced against the overall criteria for prompts in promptsource, I think it's better to have the task explicitly stated. CCing @awebson to see if he agrees? |
Of course, makes sense! I've made the changes and explicitly stated language for the |
Co-authored-by: Stephen Bach <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great! Thanks!
Added 5 English prompts for each of the 18 languages in GEM/wikilingua!
Will need others to write non-english prompts for this dataset if time permits.