init commit for working ancil suite emulator, but without ants_src

[matlipson]

This branch successfully emulates the offline/RAS suite ancillary generation process for land mask and land cover fractions. However, I have not yet included the ants_src files that it draws from, as I'm still unsure of the licensing of ANTS.

If you have access to rp23 it should work out of the box.

Otherwise if you have access to the MOSRS ants repository, you can check out the branch used by MartinBests/Siyuan's offline version with:

svn co https://code.metoffice.gov.uk/svn/ancil/ants/branches/dev/martinbest/r13910_ancil_ants_JULES_vn1.1.0/

Then update run_ancil_lct.sh variable ANTS_SRC_PATH= to point to your checkout src r13910_ancil_ants_JULES_vn1.1.0/bin. Then qsub run_ancil_lct.sh should work.

[Whyborn]

Great to see that this is working so quickly. I think given that this repository is likely to be a temporary measure, I'm happy to merge this into the main branch primarily for visibility.

[ccarouge]

@matlipson A small comment from me, if you could add some explanations of the steps in the run_ancil_lct.sh file that would be great. I'm guessing checking the documentation for each of the ANTS code used would give the information but it would be great to have a bit more info directly in the script.

In details, I have no idea what ancil_lct.py does. ancil_general_regrid.py, I can guess. ancil_lct_postproc_c4.py, my only guess is that it has to do with C4 grasses but not sure why a post-processing step is needed and what it does. I'm guessing "lct" stands for land cover transformation. I'm guessing all these Python codes are from the official ANTS package and not in the contributions from the community, is that correct?

[matlipson]

Fair points, thanks for this feedback, I can include short descriptions of what each "app" does.

But first it would be good to get your guidance on the following.

Since I did this pull request I've gone ahead and created scripts for the entire JULES ancil generation process, i.e. not just land cover (lct), but also things like canopy heights, lai, albedo, dust, soil parameters etc.

I have it working as seperate scripts (as with the lct file here), but also in a single large script. This second option is much easier for users as the dependencies are worked out for them, just need to run one bash script.

Should this large script supersede this current branch which only does one task? Perhaps this is worth discussing at the next WG meeting (but I won't be there on the 4th).

[Whyborn]

I don't think we ever got around to discussing this at the LWG meeting, but my preference would be to include both. I'm not sure exactly what your scripts look like, but I'm imagining the single large script calls smaller scripts for each separate step? Or are all the minutiae in one big script? If it's the former, then we can just include it all, and users can call the master script to get everything done at once, or call the respective bits and pieces they want/need. If it's the latter, either one big script OR a series of smaller scripts, then I'm not sure, would need to see the details to form an opinion.

[matlipson]

I don't think we ever got around to discussing this at the LWG meeting

I plan to present this at the next appropriate LWG meeting for discussion.

I'm imagining the single large script calls smaller scripts for each separate step?

Correct, one large bash script calls smaller ANTS-based scripts in the same way that they are called in the rose/cylc JULES ancillary generation suite. We can pick and choose what we want for the production of CABLE ancillaries. Obviously some of the scripts will have to change to be appropriate for CABLE, but that is something we will need to work through in the LWG. This initial script just gives us a starting point.

[ccarouge]

@matlipson We'll can discuss at the LWG meeting. For my part, I think having one script per type of ancillary file and a script to orchestrate these together is the best way to go. For example the land cover script in this PR is long enough, I wouldn't want to have it in a longer script. But I would want the ability to have 1 command to produce all the ancil files at once when needed. In short, it's the same view as Lachlan's and not quite either of the options you are proposing 😄

[ccarouge]

Not sure if you are still around but we should get this in before Christmas if you are around and get individual scripts later on.

Althoug, it would be better if ants_lct_inputs/ was renamed ants_inputs/ or even ants_app_conf/ because that doesn't include the input datasets for example but configuration information.