Skip to content

Commit 1b35aa0

Browse files
haileyschoelkopfstephenbach
haileyschoelkopf
and
stephenbach
authored
Add english prompts for GEM/wikilingua (18 languages) (#765)
* add english prompts for GEM/wikilingua (18 languages) * (typo) add missing target to 'summarize_above_ar' * (fix) update target formatting * make language more explicit in prompts * fix typo Co-authored-by: Stephen Bach <[email protected]> Co-authored-by: Stephen Bach <[email protected]>
1 parent 4695489 commit 1b35aa0

18 files changed

+1449
-0
lines changed

‎promptsource/templates/GEM/wiki_lingua/ar/templates.yaml

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
dataset: GEM/wiki_lingua
2+
subset: ar
3+
templates:
4+
26a2c187-0667-41bf-b375-da0436aba830: !Template
5+
answer_choices: null
6+
id: 26a2c187-0667-41bf-b375-da0436aba830
7+
jinja: '{{source}}
8+
9+
10+
TL;DR in Arabic: ||| {{target}}'
11+
metadata: !TemplateMetadata
12+
choices_in_prompt: false
13+
metrics:
14+
- ROUGE
15+
- BLEU
16+
original_task: true
17+
name: tldr_ar
18+
reference: xsum templates
19+
4f05d015-f132-41ad-a2da-75eb1e650c13: !Template
20+
answer_choices: null
21+
id: 4f05d015-f132-41ad-a2da-75eb1e650c13
22+
jinja: "First, read the Arabic article below. \n\n{{source}}\n\nNow, please write\
23+
\ a short abstract for it in Arabic. ||| {{target}}\n\n"
24+
metadata: !TemplateMetadata
25+
choices_in_prompt: false
26+
metrics:
27+
- ROUGE
28+
- BLEU
29+
original_task: true
30+
name: write_abstract_ar
31+
reference: xsum templates
32+
578e4464-fe13-4eff-960d-0ac1c430e8f7: !Template
33+
answer_choices: null
34+
id: 578e4464-fe13-4eff-960d-0ac1c430e8f7
35+
jinja: '{{source}}
36+
37+
38+
===
39+
40+
41+
Write a summary of the text above in Arabic. ||| {{target}}'
42+
metadata: !TemplateMetadata
43+
choices_in_prompt: false
44+
metrics:
45+
- ROUGE
46+
- BLEU
47+
original_task: true
48+
name: summarize_above_ar
49+
reference: xsum templates
50+
c3288886-c6b6-465e-acb4-fe2ea3fcd002: !Template
51+
answer_choices: null
52+
id: c3288886-c6b6-465e-acb4-fe2ea3fcd002
53+
jinja: 'Article in Arabic: {{source}}
54+
55+
56+
Summary in Arabic: ||| {{target}}'
57+
metadata: !TemplateMetadata
58+
choices_in_prompt: false
59+
metrics:
60+
- ROUGE
61+
- BLEU
62+
original_task: true
63+
name: article_summary_ar
64+
reference: xsum templates
65+
f09797cd-252b-4817-9f85-92b5c349b67b: !Template
66+
answer_choices: null
67+
id: f09797cd-252b-4817-9f85-92b5c349b67b
68+
jinja: '{{source}}
69+
70+
71+
How would you rephrase that briefly in Arabic? ||| {{target}}'
72+
metadata: !TemplateMetadata
73+
choices_in_prompt: false
74+
metrics:
75+
- ROUGE
76+
- BLEU
77+
original_task: true
78+
name: rephrase_ar
79+
reference: xsum templates

‎promptsource/templates/GEM/wiki_lingua/cs/templates.yaml

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
dataset: GEM/wiki_lingua
2+
subset: cs
3+
templates:
4+
6cb95f93-b6b7-4da8-a27f-e334d30ed856: !Template
5+
answer_choices: null
6+
id: 6cb95f93-b6b7-4da8-a27f-e334d30ed856
7+
jinja: '{{source}}
8+
9+
10+
How would you rephrase that briefly in Czech? ||| {{target}}'
11+
metadata: !TemplateMetadata
12+
choices_in_prompt: false
13+
metrics:
14+
- ROUGE
15+
- BLEU
16+
original_task: true
17+
name: rephrase_cs
18+
reference: xsum templates
19+
7d5c5019-7728-4052-9a2b-434646682398: !Template
20+
answer_choices: null
21+
id: 7d5c5019-7728-4052-9a2b-434646682398
22+
jinja: 'Article in Czech: {{source}}
23+
24+
25+
Summary in Czech: ||| {{target}}'
26+
metadata: !TemplateMetadata
27+
choices_in_prompt: false
28+
metrics:
29+
- ROUGE
30+
- BLEU
31+
original_task: true
32+
name: article_summary_cs
33+
reference: xsum templates
34+
7f2bd973-52c0-486c-ab3b-913892dfee92: !Template
35+
answer_choices: null
36+
id: 7f2bd973-52c0-486c-ab3b-913892dfee92
37+
jinja: "First, read the Czech article below.\n\n{{source}} \n\nNow, please write\
38+
\ a short abstract for it in Czech. ||| {{target}}"
39+
metadata: !TemplateMetadata
40+
choices_in_prompt: false
41+
metrics:
42+
- ROUGE
43+
- BLEU
44+
original_task: true
45+
name: write_abstract_cs
46+
reference: xsum templates
47+
a43cb97f-eeca-403c-85e0-1f1f83725900: !Template
48+
answer_choices: null
49+
id: a43cb97f-eeca-403c-85e0-1f1f83725900
50+
jinja: '{{source}}
51+
52+
53+
TL;DR in Czech: ||| {{target}}'
54+
metadata: !TemplateMetadata
55+
choices_in_prompt: false
56+
metrics:
57+
- ROUGE
58+
- BLEU
59+
original_task: true
60+
name: tldr_cs
61+
reference: xsum templates
62+
d8d4f3e8-88cd-471a-a29c-17e5822d779e: !Template
63+
answer_choices: null
64+
id: d8d4f3e8-88cd-471a-a29c-17e5822d779e
65+
jinja: '{{source}}
66+
67+
68+
===
69+
70+
71+
Write a summary of the text above in Czech: ||| {{target}}'
72+
metadata: !TemplateMetadata
73+
choices_in_prompt: false
74+
metrics:
75+
- ROUGE
76+
- BLEU
77+
original_task: true
78+
name: summarize_above_cs
79+
reference: xsum templates

‎promptsource/templates/GEM/wiki_lingua/de/templates.yaml

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
dataset: GEM/wiki_lingua
2+
subset: de
3+
templates:
4+
039c2189-9fb2-4afb-b690-251af7ee89df: !Template
5+
answer_choices: null
6+
id: 039c2189-9fb2-4afb-b690-251af7ee89df
7+
jinja: '{{source}}
8+
9+
10+
===
11+
12+
13+
Write a summary of the text above in German: ||| {{target}}'
14+
metadata: !TemplateMetadata
15+
choices_in_prompt: false
16+
metrics:
17+
- ROUGE
18+
- BLEU
19+
original_task: true
20+
name: summarize_above_de
21+
reference: xsum templates
22+
1f3a6173-9741-4ada-98fc-44b4ac78dec2: !Template
23+
answer_choices: null
24+
id: 1f3a6173-9741-4ada-98fc-44b4ac78dec2
25+
jinja: '{{source}}
26+
27+
28+
TL;DR in German: ||| {{target}}'
29+
metadata: !TemplateMetadata
30+
choices_in_prompt: false
31+
metrics:
32+
- ROUGE
33+
- BLEU
34+
original_task: true
35+
name: tldr_de
36+
reference: xsum templates
37+
2977b652-d313-4a3b-b197-f9e0e5e468db: !Template
38+
answer_choices: null
39+
id: 2977b652-d313-4a3b-b197-f9e0e5e468db
40+
jinja: "First, read the German article below. \n\n{{source}}\n\nNow, please write\
41+
\ a short abstract for it in German. ||| {{target}}"
42+
metadata: !TemplateMetadata
43+
choices_in_prompt: false
44+
metrics:
45+
- ROUGE
46+
- BLEU
47+
original_task: true
48+
name: write_abstract_de
49+
reference: xsum templates
50+
6ef08ab1-5d00-4d13-876f-e06c3bd96747: !Template
51+
answer_choices: null
52+
id: 6ef08ab1-5d00-4d13-876f-e06c3bd96747
53+
jinja: 'Article in German: {{source}}
54+
55+
56+
Summary in German: ||| {{target}}'
57+
metadata: !TemplateMetadata
58+
choices_in_prompt: false
59+
metrics:
60+
- ROUGE
61+
- BLEU
62+
original_task: true
63+
name: article_summary_de
64+
reference: xsum templates
65+
fd7fa7ca-b87f-4ecd-bc89-d5ee6deca03d: !Template
66+
answer_choices: null
67+
id: fd7fa7ca-b87f-4ecd-bc89-d5ee6deca03d
68+
jinja: '{{source}}
69+
70+
71+
How would you rephrase that briefly in German? ||| {{target}}'
72+
metadata: !TemplateMetadata
73+
choices_in_prompt: false
74+
metrics:
75+
- ROUGE
76+
- BLEU
77+
original_task: true
78+
name: rephrase_de
79+
reference: xsum templates

‎promptsource/templates/GEM/wiki_lingua/en/templates.yaml

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
dataset: GEM/wiki_lingua
2+
subset: en
3+
templates:
4+
088288f3-7516-4cf7-9406-0e082053bf54: !Template
5+
answer_choices: null
6+
id: 088288f3-7516-4cf7-9406-0e082053bf54
7+
jinja: '{{source}}
8+
9+
10+
===
11+
12+
13+
Write a summary of the text above in English : ||| {{target}}'
14+
metadata: !TemplateMetadata
15+
choices_in_prompt: false
16+
metrics:
17+
- ROUGE
18+
- BLEU
19+
original_task: true
20+
name: summarize_above_en
21+
reference: xsum DOC_write_summary_of_above template
22+
2038df7b-5420-4a33-87ec-09715419deef: !Template
23+
answer_choices: null
24+
id: 2038df7b-5420-4a33-87ec-09715419deef
25+
jinja: 'Article in English: {{source}}
26+
27+
28+
Summary in English: ||| {{target}}'
29+
metadata: !TemplateMetadata
30+
choices_in_prompt: false
31+
metrics:
32+
- ROUGE
33+
- BLEU
34+
original_task: true
35+
name: article_summary_en
36+
reference: xsum 'article_DOC_summary' template
37+
753f0a46-aeff-4cd2-932c-8548897cebe5: !Template
38+
answer_choices: null
39+
id: 753f0a46-aeff-4cd2-932c-8548897cebe5
40+
jinja: '{{source}}
41+
42+
43+
How would you rephrase that briefly in English? ||| {{target}}'
44+
metadata: !TemplateMetadata
45+
choices_in_prompt: false
46+
metrics:
47+
- ROUGE
48+
- BLEU
49+
original_task: true
50+
name: rephrase_en
51+
reference: xsum 'DOC_how_would_you_rephrase_few_words' template
52+
d3c5baa3-5e37-46f8-b1b2-5b834181c9da: !Template
53+
answer_choices: null
54+
id: d3c5baa3-5e37-46f8-b1b2-5b834181c9da
55+
jinja: '{{source}}
56+
57+
58+
TL;DR in English: ||| {{target}}'
59+
metadata: !TemplateMetadata
60+
choices_in_prompt: false
61+
metrics:
62+
- ROUGE
63+
- BLEU
64+
original_task: true
65+
name: tldr_en
66+
reference: xsum 'DOC_tldr' template
67+
dff7b314-7385-4855-bb90-253073a34fde: !Template
68+
answer_choices: null
69+
id: dff7b314-7385-4855-bb90-253073a34fde
70+
jinja: "First, read the English article below.\n\n{{source}} \n\nNow, please write\
71+
\ a short abstract for it in English. ||| {{target}}"
72+
metadata: !TemplateMetadata
73+
choices_in_prompt: false
74+
metrics:
75+
- ROUGE
76+
- BLEU
77+
original_task: true
78+
name: write_abstract_en
79+
reference: xsum 'read_below_DOC_write_abstract' template

‎promptsource/templates/GEM/wiki_lingua/es/templates.yaml

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
dataset: GEM/wiki_lingua
2+
subset: es
3+
templates:
4+
0bcbc702-a23b-45a1-8c79-67919d8ff2df: !Template
5+
answer_choices: null
6+
id: 0bcbc702-a23b-45a1-8c79-67919d8ff2df
7+
jinja: '{{source}}
8+
9+
10+
===
11+
12+
13+
Write a summary of the text above in Spanish: ||| {{target}}'
14+
metadata: !TemplateMetadata
15+
choices_in_prompt: false
16+
metrics:
17+
- ROUGE
18+
- BLEU
19+
original_task: true
20+
name: summarize_above_es
21+
reference: xsum templates
22+
3c79eb35-ae2f-4e0d-b50c-3088e32ab16e: !Template
23+
answer_choices: null
24+
id: 3c79eb35-ae2f-4e0d-b50c-3088e32ab16e
25+
jinja: "First, read the Spanish article below.\n\n{{source}} \n\nNow, please write\
26+
\ a short abstract for it in Spanish. ||| {{target}}"
27+
metadata: !TemplateMetadata
28+
choices_in_prompt: false
29+
metrics:
30+
- ROUGE
31+
- BLEU
32+
original_task: true
33+
name: write_abstract_es
34+
reference: xsum templates
35+
59be0be3-dcf3-4413-8ec8-f8a68c326bb6: !Template
36+
answer_choices: null
37+
id: 59be0be3-dcf3-4413-8ec8-f8a68c326bb6
38+
jinja: '{{source}}
39+
40+
41+
TL;DR in Spanish: ||| {{target}}'
42+
metadata: !TemplateMetadata
43+
choices_in_prompt: false
44+
metrics:
45+
- ROUGE
46+
- BLEU
47+
original_task: true
48+
name: tldr_es
49+
reference: xsum templates
50+
96c3d1f4-2e7d-468e-aca3-faa6519f768d: !Template
51+
answer_choices: null
52+
id: 96c3d1f4-2e7d-468e-aca3-faa6519f768d
53+
jinja: 'Article in Spanish: {{source}}
54+
55+
56+
Summary in Spanish: ||| {{target}}'
57+
metadata: !TemplateMetadata
58+
choices_in_prompt: false
59+
metrics:
60+
- ROUGE
61+
- BLEU
62+
original_task: true
63+
name: article_summary_es
64+
reference: xsum templates
65+
bca06c7a-d447-4fd9-a5b3-b789dcd4048a: !Template
66+
answer_choices: null
67+
id: bca06c7a-d447-4fd9-a5b3-b789dcd4048a
68+
jinja: '{{source}}
69+
70+
71+
How would you rephrase that briefly in Spanish? ||| {{target}}'
72+
metadata: !TemplateMetadata
73+
choices_in_prompt: false
74+
metrics:
75+
- ROUGE
76+
- BLEU
77+
original_task: true
78+
name: rephrase_es
79+
reference: xsum templates

0 commit comments

Comments
 (0)