Total memory limitation for PCA metrics ?

[b-grimaud]

Whenever I try to calculate PCA metrics on HD-MEA data (lots of units, lots of channels), I always run out of memory before the computation can finish, on a machine with ~100 GB of available RAM.

I know that the job kwargs have a "total_memory" argument, but as far as I can see that only applies to operations performed on recordings (I'm assuming filtering, spike sorting...).

Would it be feasible to progressively calculate metrics then only keep the output in memory ? Or maybe temporarily write the arrays to disk at the expense of speed ?