Updates for component NoahMP land model

[uturuncoglu]

Commit Queue Requirements:

• Fill out all sections of this template.
• All sub component pull requests have been reviewed by their code managers.
• Run the full Intel+GNU RT suite (compared to current baselines) on either Hera/Derecho/Hercules
• Commit 'test_changes.list' from previous step

---

Description:

The PR aims to bring recent updates from component land model. The major changes are,

• Add new decomposition algorithm to NoahMP component model NUOPC cap to distribute land and ocean points evenly to all the processor. The new decomposition algorithm might be more efficient especially for high resolution cases. The new decomposition algorithm can be activated by adding decomp_type = custom option to LND_attributes:: section of ufs.configure.
• A new flux calculation method is implemented in CCPP/Physics sfc_land scheme to overcome initialization issues. We also adjust run sequences of land related configurations.
• New RTs based on S2S configurations are introduced and they include extra component as land. The new tests called as cpld_control_p8_lnd and control_restart_p8_atmlnd.
• The atm-lnd side-by-side RT (control_p8_atmlnd is removed since CCPP/Physics version of the code diverged from the component model. Some of the information provided by the CCPP/Physics can not be provided in the component model. For example, update for land IAU in the ccpp-physics NoahMP. The side-by-side run can be still performed by running control_p8_atmlnd or cpld_control_p8_lnd by setting cpllnd2atm = .false.. This will turn off feedback from land component to atmospheric model and allow atmospheric model receive land fluxes from CCPP/Physics version of NoahMP.
• The issues in atm-lnd configuration related with the inconsistent input files are solved by using input files form coupled configuration.

Commit Message:

* UFSWM - new RT tests based on S2S configurations: cpld_control_p8_lnd and control_restart_p8_atmlnd.
  * CMEPS - flexible way to use same grid in atm-lnd coupling
  * FV3 - 
    * ccpp-physics - new flux calculation method is implemented in CCPP/Physics sfc_land scheme to overcome initialization issues. 
  * NOAHMP - add new decomposition algorithm to NoahMP component model NUOPC cap to distribute land and ocean points evenly to all the processor. 

Priority:

• Critical Bugfix: Reason
• High: Reason
• Normal

Git Tracking

UFSWM:

• Closes Atm-Lnd tests are not using V2 surface files #2267

Sub component Pull Requests:

• AQM:
• CDEPS:
• CICE:
• CMEPS: use redist if atm and lnd has same grid NOAA-EMC/CMEPS#126
• CMakeModules:
• FV3: Update for NoahMP land component NOAA-EMC/fv3atm#886
  • ccpp-physics: Skip using fluxes provided by land component for first time step ccpp-physics#234
  • atmos_cubed_sphere:
• GOCART:
• HYCOM:
• MOM6:
• NOAHMP: Enable new decomposition algorithm NOAA-EMC/noahmp#13
• WW3:
• stochastic_physics:
• None

UFSWM Blocking Dependencies:

• Blocked by #
• None

---

Changes

Regression Test Changes (Please commit test_changes.list):

• PR Adds New Tests/Baselines.
  Two new RTs are added: cpld_control_p8_lnd and control_restart_p8_atmlnd
  One RT is removed: control_p8_atmlnd_sbs
• PR Updates/Changes Baselines.
  All land related RTs have answer changes due to the changes in the input files and also change the way of model initialization.

Input data Changes:

• None.

Library Changes/Upgrades:

• Required
  • Library names w/versions:
  • Git Stack Issue (JCSDA/spack-stack#)
• No Updates

---

Testing Log:

• RDHPCS
  • Hera
  • Orion
  • Hercules
  • Jet
  • GaeaC5
  • GaeaC6
  • Derecho
• WCOSS2
  • Dogwood/Cactus
  • Acorn
• CI
• opnReqTest (complete task if unnecessary)

[uturuncoglu]

@junwang-noaa I created a draft PR to bring recent updates from land side and I'll keep working on it. Once it is ready, I'll change it's status. Probably, this would be the last update from land JTTI project since we are plaining to finalize it around end of August.

[DeniseWorthen]

@uturuncoglu Does this PR require an ESMF update for the "stripes" fix?

[uturuncoglu]

@DeniseWorthen No, I put a fix in CMEPS side, now it uses redist rather then interpolation (see the CMEPS PR for more detail). Since we are using same grid in both land and atmosphere that works for the RTs. CMEPS is assuming they are using same grid by default. Once we want to run land in a tiger resolution under S2S configuration, then we need newer version of ESMF that has fix.

[DeniseWorthen]

Thanks for the explanation. What about LM4?

[uturuncoglu]

@DeniseWorthen LM4 has no fully coupled configuration yet and it is only coupled with DATM. So, it won't affect it. But, I still need to add samegrid_atmlnd=.false. to also those tests since mediator mainly assumes that atmosphere and land grids are same which is not correct for data atmosphere could configurations.

[BrianCurtis-NOAA]

PET200 (lnd_comp_io): (read_tiled_file) adding land_frac to FB
PET200 ESMCI_PIO_Handler.C:1404 ESMCI::PIO_Handler::openOneTileF  Unable to open existing file: INPUT/oro_data.tile1.nc, (PIO/PNetCDF error = NetCDF: Attempt to use feature that was not turned on when netCDF was built.)
PET200 ESMCI_PIO_Handler.C:1404 ESMCI::PIO_Handler::openOneTileF  Unable to open existing file: INPUT/oro_data.tile2.nc, (PIO/PNetCDF error = NetCDF: Attempt to use feature that was not turned on when netCDF was built.)
PET200 ESMCI_PIO_Handler.C:1404 ESMCI::PIO_Handler::openOneTileF  Unable to open existing file: INPUT/oro_data.tile3.nc, (PIO/PNetCDF error = NetCDF: Attempt to use feature that was not turned on when netCDF was built.)
PET200 ESMCI_PIO_Handler.C:1404 ESMCI::PIO_Handler::openOneTileF  Unable to open existing file: INPUT/oro_data.tile4.nc, (PIO/PNetCDF error = NetCDF: Attempt to use feature that was not turned on when netCDF was built.)
PET200 ESMCI_PIO_Handler.C:1404 ESMCI::PIO_Handler::openOneTileF  Unable to open existing file: INPUT/oro_data.tile5.nc, (PIO/PNetCDF error = NetCDF: Attempt to use feature that was not turned on when netCDF was built.)
PET200 ESMCI_PIO_Handler.C:1404 ESMCI::PIO_Handler::openOneTileF  Unable to open existing file: INPUT/oro_data.tile6.nc, (PIO/PNetCDF error = NetCDF: Attempt to use feature that was not turned on when netCDF was built.)
PET200.ESMF_LogFile:20250127 170001.779 ERROR            PET200 ESMCI_PIO_Handler.C:617 ESMCI::PIO_Handler::arrayReadOne Unable to read from file  - file not open
PET200.ESMF_LogFile:20250127 170001.779 ERROR            PET200 ESMCI_IO_Handler.C:405 ESMCI::IO_Handler::arrayRead() Unable to read from file  - Internal subroutine call returned Error
PET200.ESMF_LogFile:20250127 170001.779 ERROR            PET200 ESMCI_IO.C:382 ESMCI::IO::read() Unable to read from file  - Internal subroutine call returned Error
PET200.ESMF_LogFile:20250127 170001.779 ERROR            PET200 ESMCI_IO.C:282 ESMCI::IO::read() Unable to read from file  - Internal subroutine call returned Error
PET200.ESMF_LogFile:20250127 170001.779 ERROR            PET200 ESMCI_IO_F.C:210 c_esmc_ioread() Unable to read from file  - Internal subroutine call returned Error
PET200.ESMF_LogFile:20250127 170001.779 ERROR            PET200 ESMF_IO.F90:397 ESMF_IOAddArray() Unable to read from file  - Internal subroutine call returned Error
PET200.ESMF_LogFile:20250127 170001.779 ERROR            PET200 ESMF_FieldBundle.F90:14436 ESMF_FieldBundleRead() Unable to read from file  - Internal subroutine call returned Error
PET200.ESMF_LogFile:20250127 170001.780 ERROR            PET200 lnd_comp_io.F90:869 Unable to read from file  - Passing error in return code
PET200.ESMF_LogFile:20250127 170001.780 ERROR            PET200 lnd_comp_domain.F90:220 Unable to read from file  - Passing error in return code
PET200.ESMF_LogFile:20250127 170001.780 ERROR            PET200 lnd_comp_nuopc.F90:425 Unable to read from file  - Passing error in return code
PET200.ESMF_LogFile:20250127 170001.780 ERROR            PET200 UFS Driver Grid Comp:src/addon/NUOPC/src/NUOPC_Driver.F90:2901 Unable to read from file  - Phase 'IPDv01p3' Initialize for modelComp 7: LND did not return ESMF_SUCCESS
PET200.ESMF_LogFile:20250127 170001.780 ERROR            PET200 UFS Driver Grid Comp:src/addon/NUOPC/src/NUOPC_Driver.F90:1985 Unable to read from file  - Passing error in return code
PET200.ESMF_LogFile:20250127 170001.780 ERROR            PET200 UFS Driver Grid Comp:src/addon/NUOPC/src/NUOPC_Driver.F90:489 Unable to read from file  - Passing error in return code
PET200.ESMF_LogFile:20250127 170001.780 ERROR            PET200 UFS.F90:397 Unable to read from file  - Aborting UFS

PET"s 200 to 343 out of 383 same error

[uturuncoglu]

@BrianCurtis-NOAA Okay. It seems that some files are missing in WCOSS2. It seems it has trouble with INPUT/oro_data.tile1.nc file.

[uturuncoglu]

@BrianCurtis-NOAA That tests (tests/tests/cpld_control_p8_lnd) basally sourcing source tests/cpld_control_p8 and it is using same input files with cpld_control_p8. It has no additional input requirement. So, if tests/fv3_conf/cpld_control_run.IN works for cpld_control_p8, it must work for cpld_control_p8_lnd too. I wonder if you have any error in rt.sh output?

[uturuncoglu]

@BrianCurtis-NOAA I am also seeing (PIO/PNetCDF error = NetCDF: Attempt to use feature that was not turned on when netCDF was built.). So, ESMF installation has some issue and I think it is not using PIO. If I remember correctly, we had similar issue with WCOSS2 in the past and it was related with the build script. I could not remember issue number but I could find it if you want.

[uturuncoglu]

Here are some similar issues:
#2232 (it is closed)
#2319

[DeniseWorthen]

I think in the end we turned off the lnd tests on WCOSS2 because it couldn't be resolved?

[uturuncoglu]

@DeniseWorthen I think we could disable these tests on WCOSS2 since I think we are not running any land tests over there. Let me modify rt.conf. BTW, #2232 says that it is fixed but maybe I missed something.

[uturuncoglu]

@DeniseWorthen @BrianCurtis-NOAA Okay. I have just disabled them on WCOSS2 like other land tests.

[jkbk2004]

We can start merging process.

[BrianCurtis-NOAA]

Noting that Acorn is being skipped due to batch submission issues

[uturuncoglu]

@BrianCurtis-NOAA JFYI, i am out of office now but back to in a hour and start to point official repositories and branches for merged ones.

[uturuncoglu]

@BrianCurtis-NOAA I am back. Waiting for FV3 PR to be merged to update submodule pointers.

[BrianCurtis-NOAA]

FV3 hash: NOAA-EMC/fv3atm@c2eb454

[uturuncoglu]

@BrianCurtis-NOAA Okay. I have just updated CMEPS, NoahMP and FV3.

[uturuncoglu]

@jkbk2004 @BrianCurtis-NOAA BTW, thanks to all for your help through the PR process.