Add phase failures to search responses

[thecoop]

Second part of #116796, following on from #121784

Expose reranker failure information as part of a search response, in a new phase_failures field

Resolves #116796

[elasticsearchmachine]

Hi @thecoop, I've created a changelog YAML for you.

[elasticsearchmachine]

Hi @thecoop, I've updated the changelog YAML for you.

[benwtrent]

I think this idea has merit.

I would like to "see" what it looks like in an API response. I also don't think "subsidiaryFailures" is a good name. They key is that these are failures that would not "fail" the result that are unrelated to remote clusters or individual shards.

Looking at the API as it is now, I think the response would look like:

{
  "took": 20,
  "timed_out": false,
  "_shards": {
...
  },
  "subsidiaryFailures": [ { "phase": "rerank", "exception": "blah blah"} ],
  "hits": {
...
  }
}

maybe it should be called "phase_failures" or "additional_failures" or "partial_failures"? The key thing is that the failures:

• Are unrelated to a shard or cluster
• Do effect the search response, but not to the extent that the search is considered completely failed.

[thecoop]

I like phase_failures. Although maybe it should be a bit more general than phase, which has a specific meaning for searches? operation_failures?

An example JSON structure can be seen https://github.com/elastic/elasticsearch/pull/122788/files#diff-e69bede2f385a8806c0d1784ed73c218fc172666cb9fbdf8fdde728bb4f12e57. I'll of course add more tests, docs, and examples once the core idea has been cleared.

[javanna]

Do you have an example of a failure that would fall in this new bucket? Are these coord level errors that don't fit into shard failures perhaps ? I have a hard time wrapping my head around this new concept. Also, where are these failures expected to be consumed?

[thecoop]

This is used here. For background, see #116796 and #121784

[github-actions]

Warning

It looks like this PR modifies one or more .asciidoc files. These files are being migrated to Markdown, and any changes merged now will be lost. See the migration guide for details.

[elasticsearchmachine]

Pinging @elastic/es-search-relevance (Team:Search Relevance)

[github-actions]

Warning

It looks like this PR modifies one or more .asciidoc files. These files are being migrated to Markdown, and any changes merged now will be lost. See the migration guide for details.

[elasticsearchmachine]

Hi @thecoop, I've created a changelog YAML for you.

[benwtrent]

I think this is good. I have some name changes that need addressed and we should have a yaml test.

One other thing is we need to ensure docs & specs are updated to include this change.

Also, could you post what the response looks like when there is a failure?