How does portman handle form encoded array query params?

[jpjpjp]

I have an endpoint that supports a form encoded query param ids.

This is the spec definition:

paths:
  /categories
    get:
      parameters:
        - name: ids
          in: query
          description: A form serialized array of category IDs to filter the
            results. <br>
            IDs that do not exist are ignored.
          required: false
          schema:
            type: array
            items:
              type: integer
          style: form
          examples:
            child_ids:
              value:
                - 315162
                - 315163
                - 315164
            top level:
              value:
                - 83
                - 84

In my first attempt to write a test that used this category I just tried setting a single ids parameter:

operations:
    - openApiOperationId: getAllCategories
      openApiResponse: '200'
      variations:
        - name: OneIdsParameter
          operationPreRequestScripts:
            - scripts:
              - file:./specs/v2/tests/scripts/categories/pre-skipGetCategoryGroupIfNone.js
          overwrites:
            - overwriteRequestQueryParams:
                - key: ids
                  value: "{{_categoryGroupId}}"

I was surprised that regardless of the type of variable I passed into the "value" field, that when the generated, the test looked like this in postman:

GET {{baseUrl}}/categories?ids={{_categoryGroupId}}&ids={{_categoryGroupId}}&ids={{_categoryGroupId}}

So, the same variable is passed three times as the form encoded parameter.

In my ideal world, the behavior would be as follows:

1. If the variable is a single entity (ie: string or integer), the result is simply:
   GET {{baseUrl}}/categories?ids={{_categoryGroupId}

2. If the variable, let's call it idList, looks like a list, the result is
   GET {{baseUrl}}/categories?ids={{idList[0}}&ids={{idList[1]}}&ids={{idList[2]}}

However, I realize that portman may not know the current value of the Postman variable when generating the test. In this case, I think its reasonable that the test writer create a pre-request script to evaluate the variable and create new individual list entry variables, ie:

Prerequest script:

// Skip the test if a category group was not found
if (!pm.collectionVariables.get("_listOfAllCategoryIds")) {
    // If _categoryGroupId is not set, skip to the next request
    console.log(`Skipping ${pm.info.requestName} since no list of categories was available`)
    pm.execution.skipRequest();
} else {
   // Define variables for the first 3 elements of the list if it is a list
    _listOfAllCategoryIds = json.Parse(pm.collectionVariables.get("_listOfAllCategoryIds"))
    pm.collectionVariables.set("_categoryId1", _listOfAllCategoryIds[0] || 0);
    pm.collectionVariables.set("_categoryId2", _listOfAllCategoryIds[1] || 0);
    pm.collectionVariables.set("_categoryId3", _listOfAllCategoryIds[2] || 0);
}

and then write a test that looks more like this:

operations:
    - openApiOperationId: getAllCategories
      openApiResponse: '200'
      variations:
        - name: OneIdsParameter
          operationPreRequestScripts:
            - scripts:
              - file:./specs/v2/tests/scripts/categories/pre-skipGetCategoryGroupIfNone.js
          overwrites:
            - overwriteRequestQueryParams:
                - key: ids
                  value: "{{_categoryId1}}"
                - key: ids
                  value: "{{_categoryId2}}"
                - key: ids
                  value: "{{_categoryId3}}"

However, the request generated from this test definition looks like this:

GET {{baseUrl}}/categories?ids={{_categoryId3}}&ids={{_categoryId3}}&ids={{_categoryId3}}

So it seems like portman is generating the same 3 values for any value that is listed as a form encoded array (at least given my spec which does include 3 ids in the first example!)

Is this a corner case, not well handled yet by Portman?

[thim81]

hi @jpjpjp

It is a corner case, which we did not ran into before.
We will try to extend query param logic to support multiple query params with the same name.

[thim81]

hi @jpjpjp

We are working on solution, which should result in the handling of form encoded array query params.

This is the result

See the PR for more details.

[thim81]

hi @jpjpjp

We just release Portman v1.30, which allows you to overwrite of form encoded array query params.

Thanks for sharing this edge case.

[jpjpjp]

Hi @thim81,

I'm afraid there is an extra corner in my corner case that is still causing an issue. A form-encoded query param can be of variable length, so a user should be able to make requests with 0, 1, 2, etc. instances of the query param. Portman v1.30 handles test cases where the number of form-encoded query params set matches the number of query params set in the first example in the spec (3 in my case) but fails if there is a different number.

Here is an example set of tests that will throw an error:

operations:
    - openApiOperationId: getAllCategories
      openApiResponse: '200'
      variations:
        - name: TwoGoodOneBadIdsParameter
          operationPreRequestScripts:
            - scripts:
              - file:./specs/v2/tests/scripts/categories/pre-generateExistingCatetoryIds.js
          overwrites:
            - overwriteRequestQueryParams:
                - key: ids
                  value: "{{_categoryId1}}"
                - key: ids
                  value: "{{_categoryId2}}"
                - key: ids
                  value: "{{_nonExistingCategoryId}}"
          tests:
            $ref: ../contracts/variation-contract-tests.yaml 
            contentTests:
              - responseBodyTests:
                  - key: categories
                    maxLength: 2
                  - key: categories[0].id
                    value: "{{_categoryId1}}"
                  - key: categories[1].id
                    value: "{{_categoryId2}}"
    - openApiOperationId: getAllCategories
      openApiResponse: '404'
      variations:
        - name: AllBadIdsParameter
          operationPreRequestScripts:
            - scripts:
              - file:./specs/v2/tests/scripts/categories/pre-generateExistingCatetoryIds.js
          overwrites:
            - overwriteRequestQueryParams:
                - key: ids
                  value: "{{_nonExistingCategoryId}}"
                - key: ids
                  value: "{{_nonExistingCategoryId2}}"
          tests:
            $ref: ../contracts/variation-contract-tests.yaml 
            contentTests:
              - responseBodyTests:
                  - key: message
                    value: Not Found
                  - key: errors
                    minLength: 2
                  - key: errors[0].errMsg
                    contains: here is no category with the id

Notice that the first request sets three instances of the ids param, but the second instance only sets two. This causes the following runtime error when generating the tests:

Uncaught TypeError TypeError: Cannot read properties of undefined (reading 'value')
    at <anonymous> (../portman/dist/application/overwrites/overwriteRequestQueryParams.js:61:38)

I had hoped to try fixing this myself and perhaps submit a PR, but I think the problem may extend beyond just the code in overwriteRequestQueryParams.js.

The issue appears to be that the heart of the overwriteRequestQueryParams function is a loop that iterates with:

pmOperation.item.request.url.query.each(pmQueryParam => {
}

But from what I could tell when iterating through my requests in the debugger, the query object was the same for all the requests: one instance of the optional format parameter and three instances of the optional ids parameter. This means that tests that try to set more than 3 instances of the query param will only set the first three, and of course, tests that try to set one or two instances will generate that runtime error.

I was reluctant to mess with the code that sets up the pmOperation object, so I thought I'd report my findings here. I realize that this is a very unique corner case so if you aren't able to support it, that is completely reasonable! I can adjust my tests with an understanding of the limitations.

[jpjpjp]

I've attached a simple spec and portman config file to reproduce the issue.
configs.zip

The "ThreeParamsWork" test generates a postman query as expected.
The "FourthParamIsIgnored" test will only generate the first 3 instances of the form encoded ids query param
If you comment out the "TwoParamCauseACrash" test and rerun portman, you should get a runtime error.

Hope this makes sense!

[thim81]

hi @jpjpjp

We are working on the support of the mentioned cases.
What would be the expected behaviour for "remove:true"?

1 config will remove all "ids"

[
      {
        key: 'name',
        value: 'newName',
        overwrite: true
      },
      {
        key: 'ids',
        remove: true // remove all IDS params
      }
]

or

[
      {
        key: 'name',
        value: 'newName',
        overwrite: true
      },
      {
        key: 'ids',
        remove: true  //remove the 1st IDS param
      },
      {
        key: 'ids',
        remove: true //remove the 2nd IDS param
      }
]

Should it be:

• 1 remove:true would remove all "ids"?
• or you would want to remove a specific "ids" (1ste or 2nd or 3rd) from the array of "ids"

For consistency and clarity, I would prefer the 1st scenario, since the "remove" indicates that you are not expecting any "ids".

[jpjpjp]

I agree that remove:true should remove all instances of the form encoded query parameter.

Form encoding is really a way to handle lists. Most libraries expect the user to pass in a single list to the api call which is then converted to this style: ?ids=1&ids=2.

This is the default style for lists in the OpenAPI spec but is also possible to pass in pipe seperated lists or indexed lists, ie: ?ids[0]=1&id[1]=2

https://swagger.io/docs/specification/serialization/

[thim81]

hi @jpjpjp

Have a look at the new PR, which will handle your 3 variations.
Does this match your expectations?

[thim81]

hi @jpjpjp

We just released Portman 1.30.1, which contains the PR to extend the handling form encoded array query params.
Let us know if this helped your case.

[jpjpjp]

I'm happy to report that 1.30.1 resolves both the issues that were present in the sample configs.zip:

-- Specifying more values than 3 (the number of values in the first example) for a form encoded query param resulted in a request where the values after the first 3 instances were ignored. This is no longer the case. All query params and values specified in the test are set in the generated request.
-- Specifying fewer values than 3 caused a runtime error. This is no longer the case! The generated request matches what was specified in the test.

Thanks as always @thim81

[thim81]

About the other Query Parameter Styles, this how they get converted as Postman request query params:

openapi: 3.0.0
info:
  title: 640 - Query Parameter Styles Example API
  version: 1.0.0
paths:
  /example:
    get:
      summary: Demonstrates different query parameter styles
      parameters:
        - in: query
          name: formParam
          schema:
            type: array
            items:
              type: string
          style: form
          explode: true
          description: Form-style query parameter
          example: [ "value1", "value2", "value3" ]
        - in: query
          name: spaceDelimitedParam
          schema:
            type: array
            items:
              type: string
          style: spaceDelimited
          explode: false
          description: Space-delimited query parameter
          example: [ "value1", "value2", "value3" ]
        - in: query
          name: pipeDelimitedParam
          schema:
            type: array
            items:
              type: string
          style: pipeDelimited
          explode: false
          description: Pipe-delimited query parameter
          example: [ "value1", "value2", "value3" ]
        - in: query
          name: deepObjectParam
          schema:
            type: object
            properties:
              prop1:
                type: string
              prop2:
                type: string
          style: deepObject
          description: DeepObject-style query parameter
          example:
            prop1: "value1"
            prop2: "value2"
      responses:
        '200':
          description: Successful response
          content:
            application/json:
              schema:
                type: object
                properties:
                  message:
                    type: string

In Postman:

Using overwrites, will always be per Postman query param key, like this example:

{
  "version": 1,
  "globals": {
    "stripResponseExamples": true
  },
  "overwrites": [
    {
      "openApiOperation": "*::*",
      "overwriteRequestQueryParams": [
        {
          "key": "formParam",
          "value": "newFormVal1"
        },
        {
          "key": "formParam",
          "value": "newFormVal2"
        },
        {
          "key": "deepObjectParam[prop1]",
          "value": "{{prop1NewValue}}"
        },
        {
          "key": "deepObjectParam[prop2]",
          "value": "{{prop2NewValue}}"
        },
        {
          "key": "spaceDelimitedParam",
          "value": "newSpaceValue1%20newSpaceValue2"
        },
        {
          "key": "pipeDelimitedParam",
          "value": "newPipeValue1|newPipeValue2"
        }
      ]
    }
  ]
}

Resulting in: