[Enabler] [Research] [zos_job_submit, zos_job_query, zos_job_output] support Ansible async controls

[ddimatos]

Is there an existing issue for this?

• There are no existing issues.

Enabler description

The goal of this issue to is to review the epic #1249 notes and research what will be required to enable the modules to support async. Then review the changes and see how impactful this is to the modules and if its going to be feasible to develop and without interface impact.

Ansible module

No response

[rexemin]

To support asynchronous execution in zos_job_submit, we would need to do the following:

1. Add self._supports_async = True to zos_job_submit and zos_copy, alongside adding wrap_async=self._task.async_val when calling execute_module on both modules.
2. Move the cleanup from the zos_job_submit module to its action plugin.
3. Add conditionals to the cleanup code of both zos_job_submit and zos_copy, so the modules only remove temp files when not run in async mode.
4. Remove the use of the community module tempfile from zos_job_submit, as it doesn't support running in async mode.
5. Implement the use of the temp path defined in Ansible's configuration in zos_job_submit, just like zos_copy does, so the async wrapper from Ansible can do the cleanup after executing a task.

zos_copy needs to be modified as well since it's used by zos_job_submit's action plugin when submitting jobs from a local file. However, zos_copy would not really be affected by being run in async mode, since everything it does will still happen synchronously. Ansible deems a zos_job_submit task dispatched as soon as zos_copy gets executed inside of its action plugin, but everything inside zos_job_submit still happens sequentially, so it doesn't create two subprocesses that could then result in issues, such as zos_job_submit trying to use a file that is still in the process of being copied to the managed node.

The current architecture supports the use of async, as I have tested the changes I outlined and seen positive results so far. Multiple jobs can be submitted asynchronously by using a loop, and then querying their results with another loop in a async_status task, like this:

    - name: Submit jobs.
      ibm.ibm_zos_core.zos_job_submit:
        src: "{{ item }}.jcl"
        location: local
      loop:
        - job_1
        - job_2
      async: 45
      poll: 0
      register: job_submit

    - name: Query async task.
      async_status:
        jid: "{{ item.ansible_job_id }}"
      register: current_job
      until: current_job.finished
      retries: 10
      loop: "{{ job_submit.results }}"