[BUG] Explain API endpoint mixes indices with metadata

[jurajseffer]

Describe the bug
Documentation at https://opendistro.github.io/for-elasticsearch-docs/docs/im/ism/api/#explain-index states that a response for the explain API endpoint looks like this:

{
  "index_1": {
    "index.opendistro.index_state_management.policy_id": "policy_1"
  }
}

However, when I use a wildcard _opendistro/_ism/explain/* search to investigate all indices, I also get total_managed_indices in the response mixed up in the response object together with the indices.E.g.

{
  "index_1": {
    "index.opendistro.index_state_management.policy_id": "policy_1"
  },
  "total_managed_indices": 1
}

This makes for harder and unexpected response parsing. I think the results of indices should be separated from metadata, e.g. under a subkey or such so that iteration over the results doesn't have to have exceptions for what sort of key it is (index entry vs metadata).

To Reproduce
Query _opendistro/_ism/explain/* in (AWS) ES 7.9

Expected behavior
Separation of result items from metadata.

{
  "indices" : {
    "index_1": {
      "index.opendistro.index_state_management.policy_id": "policy_1"
    }
  },
  "total_managed_indices": 1
}

[bowenlan-amzn]

Sorry for the inconvenience, but we are not able to change the structure of already released API response structure.

we need this total_managed_indices in our frontend, but didn't notice this could break user experience.

[dbbaughe]

Hey @jurajseffer,

Thanks for opening the issue, agree it's not helpful to have them mixed like this (and would cause an issue with an index named total_managed_indices most likely). Unfortunately we cannot change the structure of the API response at this point anymore since it would break the parsing for everyone using it. Ideally this should have been caught during CR so we could have perhaps populated an x-header instead of the response body. For now I would suggest to handle the total_managed_indices explicitly in your parsing logic and perhaps we can look into supporting a v2 version of the API with the suggested format and keep around the v1 for users still using it.