Is there any performance benefit when reading in Deferred mode a single step?

[stgeke]

Aggregation etc.

[pnorbert]

TL;DR: Yes, if you are using BP5, you have several variables to read, and a single read request cannot utilize the multi-threaded reading enough.

BP5 reads multiple blocks of data in threads when possible. The granularity of a single read operation is one block of a variable on disk (a global array written by multiple writers will consist of multiple blocks of data on disk, where each block is logically contiguous block in a file). If your single read request (Get() call) is for a variable + selection that requires reading only a single block of data from disk than BP5 is not going to use multiple threads to read it, even if it is a single 1GB block on disk.

If you read a large array produced by many writers, BP5 will use threads to read them faster than if it was a single block. You can make sure that more stuff is being read at once, if you use Deferred read mode for multiple variables and call PerformGets() to get them (or EndStep() if all variables can be read at once).

The gain is not too much though. The number of threads is set to the number of logical cores of a compute node divided by the number of your MPI ranks of the reading application on that node, and is limited up to 16 by default (because we couldn't ever see an improvement using more threads even for a local NVMe drive). You can manually set the parameter Threads if you know better https://adios2.readthedocs.io/en/latest/engines/engines.html#bp5.