Contract test: Handle responses with no body

[savage-alex]

204 is a popular status code for PUT, POST and DELETE operations and MUST not have a response body says the RFC7231

In portman contract tests it correctly omits any body or schema checks if there is no content in the and API definition for the 2xx response. What it currently does not to is add a check for response body to be empty style test such as
pm.test("response should be empty", function () {
pm.response.to.not.be.withBody;
});

Is this something that would be useful to add to ensure we are not returning bodies when the contact is saying we shall not?

[nicklloyd]

Definitively a good suggestion

[savage-alex]

I like this proposition and it covers my needs.
My only possible query is not all responses are jsonBodies, we are validating that there is no body whatsoever - is there a case where we would have a response that is something else all together? such as return a png perhaps? It would have a responseBody but its not a jsonBody.

[thim81]

We could replace "JsonBody" with "responseBody" as configuration (while deprecating JsonBody) and use the OpenAPI content-type to validate if the API response body is matching OAS defined.

[savage-alex]

@nicklloyd @AliKhalili any thoughts?

[thim81]

For more context, we are talking about a contract test setting

[nicklloyd]

We could replace "JsonBody" with "responseBody" as configuration (while deprecating JsonBody) and use the OpenAPI content-type to validate if the API response body is matching OAS defined.

I'm a fan :)

[thim81]

I'm going to link this discussion to an issue, so that we can create a PR for and close it afterwards, because this has become a valuable feature request.

[thim81]

@savage-alex I have created PR that extend the "jsonBody" contract tests, that in case of the expected 204, a testResponseBodyEmpty check is added.

So concrete, jsonBody has become a bit smarter, does this makes sense for you? With regards to the Portman and expected tests?

[savage-alex]

Hi, yes makes total sense. This will add another great safety net.

[thim81]

There is currently no control over the behaviour of "jsonBody", are you ok with this?
I expect it would make sense to allow control to use the of testResponseBodyEmpty in variations as well, but I'm looking for some users input on how you would like to control this, in the config.

[savage-alex]

I think the lack of control is fine. 204 is banned by the RFC from returning a body. I would suggest if thats the target response code from the API def (or the variation) then this test could be enabled automatically. We arent doing something unusual here. Are there cases we would expect to return the 204 AND expect a body and would therefor want to exclude this test?