[DOCS] docs enhancements from Austin

[zhao-cy]

Austin @asmacdo is thinking about writing up an outline for some docs he'd like to potentially add. It's after he plays with BABS using some weird env (fake Slurm in a container, so 'conda activate' is not available without additional prep).

I'm adding links to some docs that I think relevant to the items listed by Austin (below). Austin, please feel free to add more details/explanations (in a pull request or in any way you'd prefer)! Thank you very much in advance!

• a high level overview of the nested dataset structure
  • I have something in the example walkthrough (see drop-down window "What's inside the created BABS project") but probably not in the main doc like "Step II: Create a BABS project". So feel free to expand/add!
• explicit environment instructions for the HPC env (does it need conda and or datalad?)
  • I think I've covered some here. Not sure if this is enough to you or you'd like to add more?
• where the logs should go (analysis/logs)
  • This is mentioned in the last sentence in this section. But it might be vague/not explicit.
• document the files that are generated
  • This is a good point! I don't think we have a comprehensive description on this
• document all config files
  • Hmmm could you please say a bit more on this, Austin? Are you referring to config YAML file or?

[asmacdo]

@zhao-cy Thank you!

a high level overview of the nested dataset structure

I am imagining a graphic in the Overview, which is referenced by the example walkthrough and babs-init. As a user, I'd like a high level understanding of what I'm going to create ahead of time.

explicit environment instructions for the HPC env (does it need conda and or datalad?)

This was the result of a misunderstanding on my end. Based on your response I think the entire project is to be done in an HPC environment? I had imagined that you would install BABS onto your local machine, scaffold out a project, and then move it to an HPC environment, which I was assuming would only require singularity + workload manager. But it seems that the HPC env requires conda and BABS as well. I think this just needs a little more clarity in the overview. Long term though, I think it would be cool to support users developing the project locally and executing it on cluster. Our cluster doesn't provide many resources outside of the compute nodes.

where the logs should go (analysis/logs)

I think a reference section should explain this with links in the walkthrough and the primary docs. Specifically, babs-check-setup was failing for me, so this should be mentioned prior to babs-status

document all config files
document all files generated

I think these could be combined in another reference section. It would be nice to have a single place in the docs find a list of all the "knobs". I think this is just the config yaml, which is documented in the "create a yaml file" section, but as a user I don't know if thats the only configuration. My specific problem was related to my earlier misunderstanding, I was surprised that conda activate was needed in the HPC environment, because I thought installation was referring to local setup.

@zhao-cy I think I'd like to start with an Overview PR, and a separate PR to add a reference section and cross-links. WDYT?

[zhao-cy]

Hi @asmacdo ! These sounds great! Thank you so much for pointing these out - they are very important and there are things that I overlooked. I like your plans of PRs, too! Let me know what I can help (p.s. I will graduate in Dec this year, so after then I will leave maintainence of BABS to others. Plans on that are still to be determined but we can let you know in the slack channel who you may reach out to).

explicit environment instructions for the HPC env (does it need conda and or datalad?)

Oh you're right - currently all steps should be done on clusters. Last week I updated the installation docs to add the system requirements, but it seems this installation page and/or the overview needs more clarifications.

where the logs should go (analysis/logs)
document all config files
document all files generated

Oh, that's a good point that people may refer to logs when running babs-check-setup! And it sounds good to add reference sections!

Thanks a lot,
Chenying