Nesting

A multi-scale nesting system is available since rXXXX. It currently undergoes major testing and revisions. A detailed description will follow after the official release.

Job preparation

The following example provide a step-by-step tutorial on how to set up a nesting system with one parent (root) domain and two child domains.

1. Prepare parameter files for all domains, e.g. for run with topography and separate topography data for each domain:

   testrun_p3d (root/parent domain)
   testrun_01_p3d (child domain 1)
   testrun_02_p3d (child domain 2)
   testrun_topo
   testrun_01_topo
   testrun_02_topo
   

2. The nesting system is activated by specifying a &nestpar NAMELIST in the _p3d file of the root domain, e.g.:

    &nestpar domain_layouts = 'root domain',      1,  -1,  24,    0.0,    0.0,
                              'child domain 1',   2,   1,  24,  100.0,  100.0,
                              'child domain 2',   3,   1,  24,  200.0,  200.0,
             nesting_datatransfer_mode = 'mixed',
             nesting_mode = 'two-way',
   /
   

For details on the parameters, see nestpar.

3. Add file connection statements for all child domains to the .mrun.config file,

   #----------------------------------------------------------------------------
   # List of input-files for nested model runs
   #----------------------------------------------------------------------------
   PARIN_01                  in:job        nest#      $base_data/$fname/INPUT   _01_p3d
   PARIN_02                  in:job        nest#      $base_data/$fname/INPUT   _02_p3d
   TOPOGRAPHY_DATA_01        in:locopt     nest#      $base_data/$fname/INPUT   _01_topo
   TOPOGRAPHY_DATA_02        in:locopt     nest#      $base_data/$fname/INPUT   _02_topo
   #----------------------------------------------------------------------------
   # List of output-files for nested domains
   #----------------------------------------------------------------------------
   RUN_CONTROL_01     out:loc:tr   nest#     $base_data/$fname/MONITORING  _1_rc
   RUN_CONTROL_01     out:loc:tra  nestf     $base_data/$fname/MONITORING  _1_rc
   HEADER_01          out:loc:tr   nest#     $base_data/$fname/MONITORING  _1_header
   HEADER_01          out:loc:tra  nestf     $base_data/$fname/MONITORING  _1_header
   CPU_MEASURES_01    out:loc:tr   nest#     $base_data/$fname/MONITORING  _1_cpu
   CPU_MEASURES_01    out:loc:tra  nestf     $base_data/$fname/MONITORING  _1_cpu
   LIST_PROFIL_01     out:loc:tr   nest#     $base_data/$fname/MONITORING  _1_list_pr
   BINOUT_01          out:loc:flpe restart   $base_data/$fname/RESTART     _1_d3d
   #
   RUN_CONTROL_02     out:loc:tr   nest#     $base_data/$fname/MONITORING  _2_rc
   RUN_CONTROL_02     out:loc:tra  nestf     $base_data/$fname/MONITORING  _2_rc
   HEADER_02          out:loc:tr   nest#     $base_data/$fname/MONITORING  _2_header
   HEADER_02          out:loc:tra  nestf     $base_data/$fname/MONITORING  _2_header
   CPU_MEASURES_02    out:loc:tr   nest#     $base_data/$fname/MONITORING  _2_cpu
   CPU_MEASURES_02    out:loc:tra  nestf     $base_data/$fname/MONITORING  _2_cpu
   LIST_PROFIL_02     out:loc:tr   nest#     $base_data/$fname/MONITORING  _2_list_pr
   BINOUT_02          out:loc:flpe restart   $base_data/$fname/RESTART     _2_d3d
   #
   DATA_1D_PR_NETCDF_1    out:loc:tr prn#:prnf   $base_data/$fname/OUTPUT _1_pr  nc
   DATA_1D_SP_NETCDF_1    out:loc:tr spn#:spnf   $base_data/$fname/OUTPUT _1_sp  nc
   DATA_1D_TS_NETCDF_1    out:loc:tr tsn#:tsnf   $base_data/$fname/OUTPUT _1_ts  nc
   DATA_1D_PTS_NETCDF_1   out:loc:tr ptsn#:ptsnf $base_data/$fname/OUTPUT _1_pts nc
   DATA_2D_XY_NETCDF_1    out:loc:tr xyn#:xynf   $base_data/$fname/OUTPUT _1_xy  nc
   DATA_2D_XY_AV_NETCDF_1 out:loc:tr xyn#:xynf   $base_data/$fname/OUTPUT _1_xy_av  nc
   DATA_2D_XZ_NETCDF_1    out:loc:tr xzn#:xznf   $base_data/$fname/OUTPUT _1_xz  nc
   DATA_2D_XZ_AV_NETCDF_1 out:loc:tr xzn#:xznf   $base_data/$fname/OUTPUT _1_xz_av  nc
   DATA_2D_YZ_NETCDF_1    out:loc:tr yzn#:yznf   $base_data/$fname/OUTPUT _1_yz  nc
   DATA_2D_YZ_AV_NETCDF_1 out:loc:tr yzn#:yznf   $base_data/$fname/OUTPUT _1_yz_av  nc
   DATA_3D_NETCDF_1       out:loc:tr 3dn#:3dnf   $base_data/$fname/OUTPUT _1_3d  nc
   DATA_3D_AV_NETCDF_1    out:loc:tr 3dn#:3dnf   $base_data/$fname/OUTPUT _1_3d_av nc
   #
   DATA_1D_PR_NETCDF_2    out:loc:tr prn#:prnf   $base_data/$fname/OUTPUT _2_pr  nc
   DATA_1D_SP_NETCDF_2    out:loc:tr spn#:spnf   $base_data/$fname/OUTPUT _2_sp  nc
   DATA_1D_TS_NETCDF_2    out:loc:tr tsn#:tsnf   $base_data/$fname/OUTPUT _2_ts  nc
   DATA_1D_PTS_NETCDF_2   out:loc:tr ptsn#:ptsnf $base_data/$fname/OUTPUT _2_pts nc
   DATA_2D_XY_NETCDF_2    out:loc:tr xyn#:xynf   $base_data/$fname/OUTPUT _2_xy  nc
   DATA_2D_XY_AV_NETCDF_2 out:loc:tr xyn#:xynf   $base_data/$fname/OUTPUT _2_xy_av  nc
   DATA_2D_XZ_NETCDF_2    out:loc:tr xzn#:xznf   $base_data/$fname/OUTPUT _2_xz  nc
   DATA_2D_XZ_AV_NETCDF_2 out:loc:tr xzn#:xznf   $base_data/$fname/OUTPUT _2_xz_av  nc
   DATA_2D_YZ_NETCDF_2    out:loc:tr yzn#:yznf   $base_data/$fname/OUTPUT _2_yz  nc
   DATA_2D_YZ_AV_NETCDF_2 out:loc:tr yzn#:yznf   $base_data/$fname/OUTPUT _2_yz_av  nc
   DATA_3D_NETCDF_2       out:loc:tr 3dn#:3dnf   $base_data/$fname/OUTPUT _2_3d  nc
   DATA_3D_AV_NETCDF_2    out:loc:tr 3dn#:3dnf   $base_data/$fname/OUTPUT _2_3d_av nc
   

4. Execute mrun and specify the nesting input files, i.e.,

   mrun -d testrun -r "d3# ts# pr# ... nest# tsn# prn# ..." -X "72" ...
   

Make sure that the total number of PEs (here 72) the sum of the PEs allocated for the individual domains (3 x 24).