You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{
"messages": [
{
"content": "Your task is to analyze the content and respond to the user's request with structured data that strictly conforms to the given JSON schema.\n\n--\n\nSchema:\n{\"description\":\"A text-based answer and relation with optional metadata.\\n\\\", \"properties\": {\"content\":{\"description\":\"The main content of the answer\",\"type\":\"string\"},\"metadata\":{\"description\":\"Additional metadata about the answer\",\"properties\":{\"language\":{\"description\":\"Language of the answer\",\"type\":\"string\"},\"source\":{\"description\":\"Source of the answer\",\"type\":\"string\"}},\"type\":\"object\"},\"relation\":{\"description\":\"A single lower cased word that describes the relation of the answer to the input entity\",\"type\":\"string\"}}, \"required\":[\"content\",\"relation\"], \"title\":\"answer\", \"type\":\"object\"}\n",
"role": "system"
},
{
"content": "Write an accurate description of the movie in a single paragraph. Use as many data fields in the description as you see fit.\n\n-- \n \nadult: false\nbadrop_path: /zOpe0eHsq0A2NvNyBbtT6sj53qV.jpg\nbelongs_to_collection: %{\"backdrop_path\" => \"/ggDVF2DhqIQjJI0zUqsWMS2WTcF.jpg\", \"id\" => 720879, \"name\" => \"Sonic the Hedgehog Collection\", \"poster_path\" => \"/1qtAVIWnx4vxmFEnZ3fDDbtMt8o.jpg\"}\nbudget: 122000000genres:\nAction, Science Fiction, Comedy, Family\n\nhomepage: https://www.sonicthehedgehogmovie.com\nid: 939243\nimdb_id: tt18259086\noriginal_language: en\noriginal_title: Sonic the Hedgehog 3\noverview: Sonic, Knuckles, and Tails reunite against a powerful new adversary, Shadow, a mysterious villain with powers unlike anything they have faced before. With their abilities outmatched in every way, Team Sonic must seek out an unlikely alliance in hopes of stopping Shadow and protecting the planet.\npopularity: 1976.892\nposter_path: /d8Ryb8AunYAuycVKDp5HpdWPKgC.jpgproduction_companies:\nParamount Pictures, Original Film, Marza Animation Planet, SEGA, Blur Studio, SEGA of Americaproduction_countries:\nUnited States of America, Japan\n\nrelease_date: 2024-12-19\nrevenue: 472518457\nruntime: 110spoken_languages:\nEnglish\n\nstatus: Released\ntagline: New adventure. New rival.\ntitle: Sonic the Hedgehog 3\nvideo: false\nvote_average: 7.7\nvote_count: 2047\n",
"role": "user"
}
],
"model": "google/gemma-3-12b-it",
"response_format": {
"type": "json_object",
"value": {
"description": "A text-based answer and relation with optional metadata.\n",
"properties": {
"content": {
"description": "The main content of the answer",
"type": "string"
},
"metadata": {
"description": "Additional metadata about the answer",
"properties": {
"language": {
"description": "Language of the answer",
"type": "string"
},
"source": {
"description": "Source of the answer",
"type": "string"
}
},
"type": "object"
},
"relation": {
"description": "A single lower cased word that describes the relation of the answer to the input entity",
"type": "string"
}
},
"required": [
"content",
"relation"
],
"title": "answer",
"type": "object"
}
},
"temperature": 0
}
Output:
I get the following output:
{
"content": ": Sonic the Hedgehog 3, released in 2024, is an American-Japanese action-comedy-science fiction film with a budget of $122 million. Featuring English spoken language, the movie follows Sonic, Knuckles, and Tails as they team up to face a formidable new enemy, Shadow, in an effort to protect the planet. It is part of the 'Sonic the Hedgehog Collection' and has garnered a vote average of 7.7 from 2047 votes, achieving a global revenue of $472,518,457. The film's tagline is 'New adventure. New rival.'",
"metadata": {
"language": "en",
"source": "movie database"
},
"relation": "sequel"
}
While it mostly looks ok, there is a strange ":" at the beginning of the text, If I use the same prompt with other engine (for example gemma 3 12b running on lm studio) I get a clean output without the ":"
Another example:
{
"messages": [
{
"content": "Your task is to analyze the content and respond to the user's request with structured data that strictly conforms to the given JSON schema.\n\n--\n\nSchema:\n{\"description\":\"A text-based answer and relation with optional metadata.\\n\",\"properties\":{\"content\":{\"description\":\"The main content of the answer\",\"type\":\"string\"},\"metadata\":{\"description\":\"Additional metadata about the answer\",\"properties\":{\"language\":{\"description\":\"Language of the answer\",\"type\":\"string\"},\"source\":{\"description\":\"Source of the answer\",\"type\":\"string\"}},\"type\":\"object\"},\"relation\":{\"description\":\"A single lower cased word that describes the relation of the answer to the input entity\",\"type\":\"string\"}},\"required\":[\"content\",\"relation\"],\"title\":\"answer\",\"type\":\"object\"}",
"role": "system"
},
{
"content": "Write an accurate description of the movie in a single paragraph. Use as many data fields in the description as you see fit.\n\n-- \n\nadult: false\nbackdrop_path: /hctfNoMpQGfn3VXdBYb9p0P5bT5.jpg\nbelongs_to_collection: \nbudget: 20000000genres:\nDrama\n\nhomepage: https://a24films.com/films/babygirl\nid: 1097549\nimdb_id: tt30057084\noriginal_language: en\noriginal_title: Babygirl\noverview: A high-powered CEO puts her career and family on the line when she begins a torrid affair with her much younger intern.\npopularity: 213.381\nposter_path: /A8HbTd0FemZyFCh5qvJFpHGiwF8.jpgproduction_companies:\nA24, 2AM, Man Up Filmproduction_countries:\nUnited States of America, Netherlands\n\nrelease_date: 2024-12-25\nrevenue: 60934894\nruntime: 115spoken_languages:\nEnglish\n\nstatus: Released\ntagline: Get exactly what you want.\ntitle: Babygirl\nvideo: false\nvote_average: 5.778\nvote_count: 511",
"role": "user"
}
],
"model": "lmstudio-community/gemma-3-12b-it",
"response_format": {
"value": {
"description": "A text-based answer and relation with optional metadata.\n",
"properties": {
"content": {
"description": "The main content of the answer",
"type": "string"
},
"metadata": {
"description": "Additional metadata about the answer",
"properties": {
"language": {
"description": "Language of the answer",
"type": "string"
},
"source": {
"description": "Source of the answer",
"type": "string"
}
},
"type": "object"
},
"relation": {
"description": "A single lower cased word that describes the relation of the answer to the input entity",
"type": "string"
}
},
"required": [
"content",
"relation"
],
"title": "answer",
"type": "object"
},
"type": "json_object"
},
"temperature": 0
}
For the second output i think the issue is partially Gemma 3 12b itself because when I tried with another engine it output the same thing just without the ":"
Expected behavior
I've tried switching to phi4 on TGI and got a clean output like the following:
{
"content": "Sonic the Hedgehog 3, released on December 19, 2024, is an action-packed, science fiction comedy, and family film. Directed by Paramount Pictures and other production companies, it reunites Sonic, Knuckles, and Tails as they face a formidable new adversary, Shadow, who possesses powers unlike anything they have encountered before. With their abilities outmatched, Team Sonic seeks an unlikely alliance to stop Shadow and protect the planet. The film, part of the 'Sonic the Hedgehog Collection,' boasts a budget of $122 million and grossed $472.5 million worldwide. It features an ensemble cast and is available in English, with a runtime of 110 minutes. The movie has received a vote average of 7.7 from 2047 votes, indicating a positive reception from audiences. For more information, visit the official website at https://www.sonicthehedgehogmovie.com.",
"metadata": {
"language": "en",
"source": "Movie Database"
},
"relation": "description"
}
Second Example with Phi4
{
"content": "Babygirl is a drama film released on December 25, 2024, directed by A24 Films. The movie follows a high-powered CEO who risks her career and family by engaging in a passionate affair with her much younger intern. Set against a backdrop of corporate power dynamics and personal dilemmas, the film explores themes of desire, ambition, and the consequences of crossing professional boundaries. With a budget of $20,000,000, the film grossed $60,934,894 and has a runtime of 115 minutes. It is primarily in English and was produced in the United States and the Netherlands. The film has received a positive reception, with a vote average of 5.778 from 511 votes. The tagline 'Get exactly what you want' encapsulates the film's exploration of personal desires and the lengths one might go to achieve them.",
"metadata": {
"language": "English",
"source": "Movie Database"
},
"relation": "description"
}
The text was updated successfully, but these errors were encountered:
System Info
{ "model_id": "google/gemma-3-12b-it", "model_sha": "96b6f1eccf38110c56df3a15bffe176da04bfd80", "model_pipeline_tag": "image-text-to-text", "max_concurrent_requests": 128, "max_best_of": 2, "max_stop_sequences": 4, "max_input_tokens": 7349, "max_total_tokens": 7350, "validation_workers": 2, "max_client_batch_size": 4, "router": "text-generation-router", "version": "3.2.3", "sha": "a1f3ebe17ca1cbe5f8a52291a7339a40eb2830ae", "docker_label": "sha-a1f3ebe" }Information
Tasks
Reproduction
Steps to Reproduce:
{ "messages": [ { "content": "Your task is to analyze the content and respond to the user's request with structured data that strictly conforms to the given JSON schema.\n\n--\n\nSchema:\n{\"description\":\"A text-based answer and relation with optional metadata.\\n\\\", \"properties\": {\"content\":{\"description\":\"The main content of the answer\",\"type\":\"string\"},\"metadata\":{\"description\":\"Additional metadata about the answer\",\"properties\":{\"language\":{\"description\":\"Language of the answer\",\"type\":\"string\"},\"source\":{\"description\":\"Source of the answer\",\"type\":\"string\"}},\"type\":\"object\"},\"relation\":{\"description\":\"A single lower cased word that describes the relation of the answer to the input entity\",\"type\":\"string\"}}, \"required\":[\"content\",\"relation\"], \"title\":\"answer\", \"type\":\"object\"}\n", "role": "system" }, { "content": "Write an accurate description of the movie in a single paragraph. Use as many data fields in the description as you see fit.\n\n-- \n \nadult: false\nbadrop_path: /zOpe0eHsq0A2NvNyBbtT6sj53qV.jpg\nbelongs_to_collection: %{\"backdrop_path\" => \"/ggDVF2DhqIQjJI0zUqsWMS2WTcF.jpg\", \"id\" => 720879, \"name\" => \"Sonic the Hedgehog Collection\", \"poster_path\" => \"/1qtAVIWnx4vxmFEnZ3fDDbtMt8o.jpg\"}\nbudget: 122000000genres:\nAction, Science Fiction, Comedy, Family\n\nhomepage: https://www.sonicthehedgehogmovie.com\nid: 939243\nimdb_id: tt18259086\noriginal_language: en\noriginal_title: Sonic the Hedgehog 3\noverview: Sonic, Knuckles, and Tails reunite against a powerful new adversary, Shadow, a mysterious villain with powers unlike anything they have faced before. With their abilities outmatched in every way, Team Sonic must seek out an unlikely alliance in hopes of stopping Shadow and protecting the planet.\npopularity: 1976.892\nposter_path: /d8Ryb8AunYAuycVKDp5HpdWPKgC.jpgproduction_companies:\nParamount Pictures, Original Film, Marza Animation Planet, SEGA, Blur Studio, SEGA of Americaproduction_countries:\nUnited States of America, Japan\n\nrelease_date: 2024-12-19\nrevenue: 472518457\nruntime: 110spoken_languages:\nEnglish\n\nstatus: Released\ntagline: New adventure. New rival.\ntitle: Sonic the Hedgehog 3\nvideo: false\nvote_average: 7.7\nvote_count: 2047\n", "role": "user" } ], "model": "google/gemma-3-12b-it", "response_format": { "type": "json_object", "value": { "description": "A text-based answer and relation with optional metadata.\n", "properties": { "content": { "description": "The main content of the answer", "type": "string" }, "metadata": { "description": "Additional metadata about the answer", "properties": { "language": { "description": "Language of the answer", "type": "string" }, "source": { "description": "Source of the answer", "type": "string" } }, "type": "object" }, "relation": { "description": "A single lower cased word that describes the relation of the answer to the input entity", "type": "string" } }, "required": [ "content", "relation" ], "title": "answer", "type": "object" } }, "temperature": 0 }Output:
I get the following output:
{ "content": ": Sonic the Hedgehog 3, released in 2024, is an American-Japanese action-comedy-science fiction film with a budget of $122 million. Featuring English spoken language, the movie follows Sonic, Knuckles, and Tails as they team up to face a formidable new enemy, Shadow, in an effort to protect the planet. It is part of the 'Sonic the Hedgehog Collection' and has garnered a vote average of 7.7 from 2047 votes, achieving a global revenue of $472,518,457. The film's tagline is 'New adventure. New rival.'", "metadata": { "language": "en", "source": "movie database" }, "relation": "sequel" }While it mostly looks ok, there is a strange ":" at the beginning of the text, If I use the same prompt with other engine (for example gemma 3 12b running on lm studio) I get a clean output without the ":"
Another example:
{ "messages": [ { "content": "Your task is to analyze the content and respond to the user's request with structured data that strictly conforms to the given JSON schema.\n\n--\n\nSchema:\n{\"description\":\"A text-based answer and relation with optional metadata.\\n\",\"properties\":{\"content\":{\"description\":\"The main content of the answer\",\"type\":\"string\"},\"metadata\":{\"description\":\"Additional metadata about the answer\",\"properties\":{\"language\":{\"description\":\"Language of the answer\",\"type\":\"string\"},\"source\":{\"description\":\"Source of the answer\",\"type\":\"string\"}},\"type\":\"object\"},\"relation\":{\"description\":\"A single lower cased word that describes the relation of the answer to the input entity\",\"type\":\"string\"}},\"required\":[\"content\",\"relation\"],\"title\":\"answer\",\"type\":\"object\"}", "role": "system" }, { "content": "Write an accurate description of the movie in a single paragraph. Use as many data fields in the description as you see fit.\n\n-- \n \nadult: false\nbackdrop_path: /hctfNoMpQGfn3VXdBYb9p0P5bT5.jpg\nbelongs_to_collection: \nbudget: 20000000genres:\nDrama\n\nhomepage: https://a24films.com/films/babygirl\nid: 1097549\nimdb_id: tt30057084\noriginal_language: en\noriginal_title: Babygirl\noverview: A high-powered CEO puts her career and family on the line when she begins a torrid affair with her much younger intern.\npopularity: 213.381\nposter_path: /A8HbTd0FemZyFCh5qvJFpHGiwF8.jpgproduction_companies:\nA24, 2AM, Man Up Filmproduction_countries:\nUnited States of America, Netherlands\n\nrelease_date: 2024-12-25\nrevenue: 60934894\nruntime: 115spoken_languages:\nEnglish\n\nstatus: Released\ntagline: Get exactly what you want.\ntitle: Babygirl\nvideo: false\nvote_average: 5.778\nvote_count: 511", "role": "user" } ], "model": "lmstudio-community/gemma-3-12b-it", "response_format": { "value": { "description": "A text-based answer and relation with optional metadata.\n", "properties": { "content": { "description": "The main content of the answer", "type": "string" }, "metadata": { "description": "Additional metadata about the answer", "properties": { "language": { "description": "Language of the answer", "type": "string" }, "source": { "description": "Source of the answer", "type": "string" } }, "type": "object" }, "relation": { "description": "A single lower cased word that describes the relation of the answer to the input entity", "type": "string" } }, "required": [ "content", "relation" ], "title": "answer", "type": "object" }, "type": "json_object" }, "temperature": 0 }Output:
{ "content": ":Babygirl", "metadata": { "language": "en", "source": "TMDB" }, "relation": "about" }For the second output i think the issue is partially Gemma 3 12b itself because when I tried with another engine it output the same thing just without the ":"
Expected behavior
I've tried switching to phi4 on TGI and got a clean output like the following:
{ "content": "Sonic the Hedgehog 3, released on December 19, 2024, is an action-packed, science fiction comedy, and family film. Directed by Paramount Pictures and other production companies, it reunites Sonic, Knuckles, and Tails as they face a formidable new adversary, Shadow, who possesses powers unlike anything they have encountered before. With their abilities outmatched, Team Sonic seeks an unlikely alliance to stop Shadow and protect the planet. The film, part of the 'Sonic the Hedgehog Collection,' boasts a budget of $122 million and grossed $472.5 million worldwide. It features an ensemble cast and is available in English, with a runtime of 110 minutes. The movie has received a vote average of 7.7 from 2047 votes, indicating a positive reception from audiences. For more information, visit the official website at https://www.sonicthehedgehogmovie.com.", "metadata": { "language": "en", "source": "Movie Database" }, "relation": "description" }Second Example with Phi4
{ "content": "Babygirl is a drama film released on December 25, 2024, directed by A24 Films. The movie follows a high-powered CEO who risks her career and family by engaging in a passionate affair with her much younger intern. Set against a backdrop of corporate power dynamics and personal dilemmas, the film explores themes of desire, ambition, and the consequences of crossing professional boundaries. With a budget of $20,000,000, the film grossed $60,934,894 and has a runtime of 115 minutes. It is primarily in English and was produced in the United States and the Netherlands. The film has received a positive reception, with a vote average of 5.778 from 511 votes. The tagline 'Get exactly what you want' encapsulates the film's exploration of personal desires and the lengths one might go to achieve them.", "metadata": { "language": "English", "source": "Movie Database" }, "relation": "description" }The text was updated successfully, but these errors were encountered: