Allow for ALL steps to be ran in one submission.

[selvaggi]

This enables the possibility of running the 3 steps together in one single submission by specifying the --datTier ALL option.

[clelange]

Thanks, this looks good to me, I just have two questions/comments in-line, and will have to test this myself (tomorrow/Wednesday).

[clelange]

Ah, OK, I didn't realise that only the NTUP part is written out. What about adding options to keep (i.e. copy) the intermediate steps as well? We could also address this in a separate PR later.

My current use case is actually to only run up to the RECO step, and I'd like to keep both GSD and RECO (the only reason for GSD being that we do not have a re-RECO step at the moment), so the ALL step doesn't address my use case, and I was wondering whether one could introduce a "allInOne" option instead of the ALL data tier, and allow for the data tier a comma-separated list that would determine the output to be stored. For example, --datTier NTUP --allInOne would run up to NTUP and only store NTUP, while --datTier GSD,RECO --allInOne would run up to RECO and also keep GSD. It's more work to implement this though, and we can first merge this PR.

Let me know what you prefer.

[selvaggi]

Yes, it is a good idea. But I would prefer we merge this first (after you have tested it), and then upgrade to what you suggest later. I have some others small items that I would like to add first.
Maybe you can add your suggestion to the PR, or create an issue so that we keep that in mind.

[franzoni]

hello @selvaggi @clelange
when this PR Was done, I could update and complete #87
perhaps you have an eta ?

[franzoni]

closed accidentally! Sorry.

[selvaggi]

Trying to re-open.

[franzoni]

thanks @selvaggi

[clelange]

@franzoni I'm done in terms of code and documentation review, but haven't taken the time to actually take this branch and run a few test jobs.
I think an easy test would be to produce 2-3 events using the ALL option, and producing the same events running each step individually. In the end, this should give the same ntuple, while the step-by-step approach should also have intermediate output files. Would you like to try this and report back?