Coupled model runs
Starting from version 3.4 PALM allows coupled atmosphere-ocean model runs. If MPI-2 support is available, mrun starts the coupled model as two concurrent executables, the atmosphere version and the ocean version in analogy with the modular structure of PALM.
The coupler establishes a one-way interaction between the atmosphere and the ocean. The atmosphere sends its bottom surface fluxes (temperature, humidity, momentum) to the ocean. The ocean sends its top surface temperature and top horizontal velocity to the atmosphere. The atmosphere assumes saturation of humidity at its bottom surface. For calculations with humidity = .T. the atmospheric evaporation leads to a salinity flux in the ocean (see e.g. Steinhorn 1991, JPO 21, p. 1681).
If the full MPI-2 standard is available, it can be activated by adding the preprocessor directive -D__mpi2 to cpp_options in the .mrun.config configuration file. Otherwise, PALM will use a coupling via MPI1. Please note that coupling with different horizontal grid size and different number of processors in ocean and atmosphere is only possible with MPI-1. To start a coupled model run, this must be requested with the mrun option -Y “#1 #2”, where #1 is the number of processors for the atmospheric and #2 the number of processors for the oceanic version of PALM. This tells mrun to start two PALM executables. Coupled runs are only possible in parallel mode, which means that the mrun option -K parallel must also be set. The mrun call for coupled runs has to include the following coupling-related options:
mrun ... -K parallel -r “d3# d3o# ...” -X ... -Y “#1 #2”.
To minimize idle of processors in ocean or atmosphere the ratio of the number of processors in atmosphere (#1) and in ocean (#2) should be close to
TracMath macro processor has detected an error. Please fix the problem before continuing.
The command:
'/usr/bin/pdflatex -interaction=nonstopmode 069046bc66ae0665a67625d5eb5b0765abdc1aef.tex'
failed with the following output:
"This is pdfTeX, Version 3.1415926-2.5-1.40.14 (TeX Live 2013)\n restricted \\write18 enabled.\nentering extended mode\n(./069046bc66ae0665a67625d5eb5b0765abdc1aef.tex\nLaTeX2e <2011/06/27>\nBabel <3.9h> and hyphenation patterns for 78 languages loaded.\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/article.cls\nDocument Class: article 2007/10/19 v1.4h Standard LaTeX document class\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/size10.clo))\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/inputenc.sty\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/utf8.def\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/t1enc.dfu)\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/ot1enc.dfu)\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/omsenc.dfu)))\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/cmap/cmap.sty)\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/cm-super/type1ec.sty\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/t1cmr.fd))\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/fontenc.sty\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/t1enc.def)<<t1.cmap\n>>) (/localdata/software/texlive/2013/texmf-dist/tex/latex/amsmath/amsmath.sty\nFor additional information on amsmath, use the `?' option.\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amsmath/amstext.sty\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amsmath/amsgen.sty))\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amsmath/amsbsy.sty)\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amsmath/amsopn.sty))\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amscls/amsthm.sty)\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amsfonts/amssymb.sty\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amsfonts/amsfonts.sty))\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/tools/bm.sty)\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/preview/preview.sty\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/preview/prtightpage.def)\n) (./069046bc66ae0665a67625d5eb5b0765abdc1aef.aux)\nPreview: Fontsize 10pt\nPreview: PDFoutput 1\n! Missing $ inserted.\n<inserted text> \n $\nl.14 \\frac{\\#1}{\\#2}\n \\approx\\frac{(\\Delta t)_{\\mbox{O}}}{(\\Delta t)_{\\mbox{A...\n\n<<ot1.cmap>><<oml.cmap>><<oms.cmap>><<omx.cmap>>\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amsfonts/umsa.fd)\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amsfonts/umsb.fd)\n! Extra }, or forgotten $.\n\\frac #1#2->{\\begingroup #1\\endgroup \\@@over #2}\n \nl.14 \\frac{\\#1}{\\#2}\n \\approx\\frac{(\\Delta t)_{\\mbox{O}}}{(\\Delta t)_{\\mbox{A...\n\n! Missing $ inserted.\n<inserted text> \n $\nl.15 \\end{preview}\n \n! Missing } inserted.\n<inserted text> \n }\nl.15 \\end{preview}\n \nPreview: Tightpage -32891 -32891 32891 32891\n[1{/localdata/software/texlive/2013/texmf-var/fonts/map/pdftex/updmap/pdftex.ma\np}] (./069046bc66ae0665a67625d5eb5b0765abdc1aef.aux) )\n(see the transcript file for additional information){/localdata/software/texliv\ne/2013/texmf-dist/fonts/enc/dvips/cm-super/cm-super-t1.enc}</localdata/software\n/texlive/2013/texmf-dist/fonts/type1/public/amsfonts/cm/cmmi5.pfb></localdata/s\noftware/texlive/2013/texmf-dist/fonts/type1/public/amsfonts/cm/cmmi7.pfb></loca\nldata/software/texlive/2013/texmf-dist/fonts/type1/public/amsfonts/cm/cmr5.pfb>\n</localdata/software/texlive/2013/texmf-dist/fonts/type1/public/amsfonts/cm/cmr\n7.pfb></localdata/software/texlive/2013/texmf-dist/fonts/type1/public/amsfonts/\ncm/cmsy7.pfb></localdata/software/texlive/2013/texmf-dist/fonts/type1/public/cm\n-super/sfrm1000.pfb>\nOutput written on 069046bc66ae0665a67625d5eb5b0765abdc1aef.pdf (1 page, 46439 b\nytes).\nTranscript written on 069046bc66ae0665a67625d5eb5b0765abdc1aef.log.\n"
with
TracMath macro processor has detected an error. Please fix the problem before continuing.
The command:
'/usr/bin/pdflatex -interaction=nonstopmode 9bfe8e1dff7ab482001982395023c0559175a474.tex'
failed with the following output:
"This is pdfTeX, Version 3.1415926-2.5-1.40.14 (TeX Live 2013)\n restricted \\write18 enabled.\nentering extended mode\n(./9bfe8e1dff7ab482001982395023c0559175a474.tex\nLaTeX2e <2011/06/27>\nBabel <3.9h> and hyphenation patterns for 78 languages loaded.\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/article.cls\nDocument Class: article 2007/10/19 v1.4h Standard LaTeX document class\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/size10.clo))\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/inputenc.sty\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/utf8.def\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/t1enc.dfu)\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/ot1enc.dfu)\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/omsenc.dfu)))\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/cmap/cmap.sty)\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/cm-super/type1ec.sty\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/t1cmr.fd))\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/fontenc.sty\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/t1enc.def)<<t1.cmap\n>>) (/localdata/software/texlive/2013/texmf-dist/tex/latex/amsmath/amsmath.sty\nFor additional information on amsmath, use the `?' option.\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amsmath/amstext.sty\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amsmath/amsgen.sty))\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amsmath/amsbsy.sty)\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amsmath/amsopn.sty))\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amscls/amsthm.sty)\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amsfonts/amssymb.sty\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amsfonts/amsfonts.sty))\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/tools/bm.sty)\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/preview/preview.sty\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/preview/prtightpage.def)\n) (./9bfe8e1dff7ab482001982395023c0559175a474.aux)\nPreview: Fontsize 10pt\nPreview: PDFoutput 1\n! Missing $ inserted.\n<inserted text> \n $\nl.14 (\\Delta\n t)_{\\mbox{O}},\\, (\\Delta t)_{\\mbox{A}}\n<<ot1.cmap>><<oml.cmap>><<oms.cmap>><<omx.cmap>>\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amsfonts/umsa.fd)\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amsfonts/umsb.fd)\n! Missing $ inserted.\n<inserted text> \n $\nl.15 \\end{preview}\n \nPreview: Tightpage -32891 -32891 32891 32891\n[1{/localdata/software/texlive/2013/texmf-var/fonts/map/pdftex/updmap/pdftex.ma\np}] (./9bfe8e1dff7ab482001982395023c0559175a474.aux) )\n(see the transcript file for additional information){/localdata/software/texliv\ne/2013/texmf-dist/fonts/enc/dvips/cm-super/cm-super-t1.enc}</localdata/software\n/texlive/2013/texmf-dist/fonts/type1/public/amsfonts/cm/cmmi10.pfb></localdata/\nsoftware/texlive/2013/texmf-dist/fonts/type1/public/amsfonts/cm/cmr10.pfb></loc\naldata/software/texlive/2013/texmf-dist/fonts/type1/public/cm-super/sfrm1000.pf\nb>\nOutput written on 9bfe8e1dff7ab482001982395023c0559175a474.pdf (1 page, 29538 b\nytes).\nTranscript written on 9bfe8e1dff7ab482001982395023c0559175a474.log.\n"
the expected averaged time step and
TracMath macro processor has detected an error. Please fix the problem before continuing.
The command:
'/usr/bin/pdflatex -interaction=nonstopmode 650a2ecfd5baca810da57ebb6d37f58842b73463.tex'
failed with the following output:
"This is pdfTeX, Version 3.1415926-2.5-1.40.14 (TeX Live 2013)\n restricted \\write18 enabled.\nentering extended mode\n(./650a2ecfd5baca810da57ebb6d37f58842b73463.tex\nLaTeX2e <2011/06/27>\nBabel <3.9h> and hyphenation patterns for 78 languages loaded.\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/article.cls\nDocument Class: article 2007/10/19 v1.4h Standard LaTeX document class\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/size10.clo))\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/inputenc.sty\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/utf8.def\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/t1enc.dfu)\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/ot1enc.dfu)\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/omsenc.dfu)))\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/cmap/cmap.sty)\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/cm-super/type1ec.sty\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/t1cmr.fd))\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/fontenc.sty\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/base/t1enc.def)<<t1.cmap\n>>) (/localdata/software/texlive/2013/texmf-dist/tex/latex/amsmath/amsmath.sty\nFor additional information on amsmath, use the `?' option.\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amsmath/amstext.sty\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amsmath/amsgen.sty))\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amsmath/amsbsy.sty)\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amsmath/amsopn.sty))\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amscls/amsthm.sty)\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amsfonts/amssymb.sty\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amsfonts/amsfonts.sty))\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/tools/bm.sty)\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/preview/preview.sty\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/preview/prtightpage.def)\n) (./650a2ecfd5baca810da57ebb6d37f58842b73463.aux)\nPreview: Fontsize 10pt\nPreview: PDFoutput 1\n! Missing $ inserted.\n<inserted text> \n $\nl.14 N_\n {\\mbox{O}},\\, N_{\\mbox{A}}\n<<ot1.cmap>><<oml.cmap>><<oms.cmap>><<omx.cmap>>\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amsfonts/umsa.fd)\n(/localdata/software/texlive/2013/texmf-dist/tex/latex/amsfonts/umsb.fd)\n! Missing $ inserted.\n<inserted text> \n $\nl.15 \\end{preview}\n \nPreview: Tightpage -32891 -32891 32891 32891\n[1{/localdata/software/texlive/2013/texmf-var/fonts/map/pdftex/updmap/pdftex.ma\np}] (./650a2ecfd5baca810da57ebb6d37f58842b73463.aux) )\n(see the transcript file for additional information){/localdata/software/texliv\ne/2013/texmf-dist/fonts/enc/dvips/cm-super/cm-super-t1.enc}</localdata/software\n/texlive/2013/texmf-dist/fonts/type1/public/amsfonts/cm/cmmi10.pfb></localdata/\nsoftware/texlive/2013/texmf-dist/fonts/type1/public/cm-super/sfrm1000.pfb>\nOutput written on 650a2ecfd5baca810da57ebb6d37f58842b73463.pdf (1 page, 16566 b\nytes).\nTranscript written on 650a2ecfd5baca810da57ebb6d37f58842b73463.log.\n"
the total number of grid points in ocean and atmosphere.
Each coupled executable has its own, unique set of I/O filenames. Information on file name conventions of coupled runs are given here. The configuration file .mrun.config has to be extended for coupled runs. It is recommended to duplicate existing file connection identifiers such as "d3#", "pr#" etc. using the coupled ocean filenames accordingly. For example, the example of the previous chapters could be duplicated as follows:
PARIN in:job:npe d3# ~/palm/current_version/JOBS/$fname/INPUT _p3d
PARIN in:job:npe d3f ~/palm/current_version/JOBS/$fname/INPUT _p3df
BININ in:loc d3f ~/palm/current_version/JOBS/$fname/OUTPUT _d3d
#
BINOUT out:loc restart ~/palm/current_version/JOBS/$fname/OUTPUT _d3d
#
RUN_CONTROL out:loc:tr d3# ~/palm/current_version/JOBS/$fname/MONITORING _rc
HEADER out:loc:tr d3# ~/palm/current_version/JOBS/$fname/MONITORING _header
PLOT1D_PAR out:loc:tr pr# ~/palm/current_version/JOBS/$fname/OUTPUT _pr_par
PLOT1D_DATA out:loc:tr pr# ~/palm/current_version/JOBS/$fname/OUTPUT _pr_in
#
PARIN_O in:job:npe d3o# ~/palm/current_version/JOBS/$fname/INPUT _o_p3d
PARIN_O in:job:npe d3of ~/palm/current_version/JOBS/$fname/INPUT _o_p3df
BININ_O in:loc d3of ~/palm/current_version/JOBS/$fname/OUTPUT _o_d3d
#
BINOUT_O out:loc restart ~/palm/current_version/JOBS/$fname/OUTPUT _o_d3d
#
RUN_CONTROL_O out:loc:tr d3o# ~/palm/current_version/JOBS/$fname/MONITORING _o_rc
HEADER_O out:loc:tr d3o# ~/palm/current_version/JOBS/$fname/MONITORING _o_header
PLOT1D_PAR_O out:loc:tr pro# ~/palm/current_version/JOBS/$fname/OUTPUT _o_pr_par
PLOT1D_DATA_O out:loc:tr pro# ~/palm/current_version/JOBS/$fname/OUTPUT _o_pr_in
The coupled ocean model filenames in the first column (e.g. PARIN_O) must be specified as given here; the file connection identifiers (e.g. d3o#) and the file name extension (e.g. _o_p3d) may be changed at the user's discretion.
The coupler requires the following parameters to be equal in both PARIN and PARIN_O: dt_coupling, end_time, restart_time, dt_restart. Furthermore the model domain length in x- and y-direction in ocean and atmosphere has to be same. In the coupled atmosphere executable, bc_pt_b is internally set and does not need to be prescribed; in the coupled ocean executable, bc_uv_t is internally set ( 'neumann' ) and does not need to be prescribed. The coupled ocean parameter file PARIN_O should include dummy REAL value assignments to both top_momentumflux_u and top_momentumflux_v (e.g. top_momentumflux_u = 0.0, top_momentumflux_v = 0.0) to enable the momentum flux coupling.
The coupling interval dt_coupling must be explicitly set. In order to ensure synchronous coupling throughout the simulation, dt_coupling should be chosen larger than dt_max.
Precursor runs
It's also possible to perform precursor runs (one atmospheric and one oceanic), followed by a coupled restart run to save computational time owing to different spin-up times in atmosphere and ocean. To use the precursor run control, the parameter coupling_start_time must be set in both, the precursor runs as well as in the coupled restart run.
Configuration (.mrun.config)
For the precursor runs the following relation must be valid:
end_time - coupling_start_time = 0.
Here, coupling_start_time refers to the time period, in which the precursor runs shall run uncoupled. In the output data, the time will show negative values during this period. In the restart run, coupling_start_time must be set equal to the settings in the precursor runs.
For the coupled restart run the following relation must be valid:
end_time - coupling_start_time = X,
where X is equal in the atmospheric and the oceanic model.
Example
Precursor run (atmosphere):
end_time = 3600.0
coupling_start_time = 3600.0
Precursor run (ocean):
end_time = 40000.0
coupling_start_time = 40000.0
Coupled restart run (atmosphere):
end_time = 23600.0
coupling_start_time = 3600.0
Coupled restart run (ocean):
end_time = 60000.0
coupling_start_time = 40000.0
Starting (mrun)
The atmospheric precursor run can be started like a usual atmospheric run:
mrun -d examplerun ... -r “d3# ... restart”,
while the oceanic precursor run requires the -y option:
mrun -d examplerun ... -r “d3o# ... restart” -y
The coupled restart then is carried out by:
mrun -d examplerun ... -r “d3f d3of ... restart” -Y "...".
During the precursor runs, the time exhibits negative values such that both, atmosphere and ocean, restart at time zero in the coupled run.
The parameter restart_time? will not take effect during the precursor runs and its value is related to the time in the coupled run.
