== Coupled model runs ==

Starting from version 3.4 PALM allows coupled atmosphere-ocean model runs. If MPI-2 support is available, '''mrun''' starts the coupled model as two concurrent executables, the atmosphere version and the ocean version in analogy with the modular structure of PALM.\\\\
The coupler establishes a one-way interaction between the atmosphere and the ocean. The atmosphere sends its bottom surface fluxes (temperature, humidity, momentum) to the ocean. The ocean sends its top surface temperature and top horizontal velocity to the atmosphere. The atmosphere assumes saturation of humidity at its bottom surface. For calculations with [../../initialization_parameters#humidity humidity] = ''.T.'' the atmospheric evaporation leads to a salinity flux in the ocean (see e.g. Steinhorn 1991, JPO 21, p. 1681).\\
\\
If the full MPI-2 standard is available, it can be activated by adding the preprocessor directive {{{-D__mpi2}}} to {{{cpp_options}}} in the {{{.mrun.config}}} configuration file. Otherwise, PALM will use a coupling via MPI1. Please note that coupling with different horizontal grid size and different number of processors in ocean and atmosphere is only possible with MPI-1. To start a coupled model run, this must be requested with the '''mrun''' option {{{-Y}}} {{{“#1 #2”}}}, where {{{#1}}} is the number of processors for the atmospheric and {{{#2}}} the number of processors for the oceanic version of PALM. This tells '''mrun''' to start two PALM executables. Coupled runs are only possible in parallel mode, which means that the '''mrun''' option {{{-K parallel}}} must also be set. The '''mrun''' call for coupled runs has to include the following coupling-related options:
{{{
      mrun ... -K parallel -r “d3# d3o# ...” -X ... -Y “#1 #2”.
}}}
To minimize idle of processors in ocean or atmosphere the ratio of the number of processors in atmosphere ({{{#1}}}) and in ocean ({{{#2}}}) should be close to
{{{
#!Latex
\begin{align*} 
\frac{\#1}{\#2} \approx\frac{(\Delta t)_{\mbox{O}}}{(\Delta t)_{\mbox{A}}} \frac{N_{\mbox{A}}}{N_{\mbox{O}}} ,
\end{align*}
}}}
with 
{{{
#!Latex
\begin{align*} 
(\Delta t)_{\mbox{O}},\, (\Delta t)_{\mbox{A}}
\end{align*}
}}}
the expected averaged time step and  
{{{
#!Latex
\begin{align*}
N_{\mbox{O}},\, N_{\mbox{A}}
\end{align*}
}}}
the total number of grid points in ocean and atmosphere.

Each coupled executable has its own, unique set of I/O filenames. Information on file name conventions of coupled runs are given [../../iofiles here]. The configuration file {{{.mrun.config}}} has to be extended for coupled runs. It is recommended to duplicate existing file connection identifiers such as {{{"d3#"}}}, {{{"pr#"}}} etc. using the coupled ocean filenames accordingly. For example, the example of the previous chapters could be duplicated as follows:
{{{
PARIN in:job:npe d3# ~/palm/current_version/JOBS/$run_identifier/INPUT _p3d
PARIN in:job:npe d3r ~/palm/current_version/JOBS/$run_identifier/INPUT _p3dr
BININ in:loc d3r ~/palm/current_version/JOBS/$run_identifier/OUTPUT _d3d
#
BINOUT out:loc restart ~/palm/current_version/JOBS/$run_identifier/OUTPUT _d3d
#
RUN_CONTROL out:loc:tr d3# ~/palm/current_version/JOBS/$run_identifier/MONITORING _rc
HEADER out:loc:tr d3# ~/palm/current_version/JOBS/$run_identifier/MONITORING _header
PLOT1D_PAR out:loc:tr pr# ~/palm/current_version/JOBS/$run_identifier/OUTPUT _pr_par
PLOT1D_DATA out:loc:tr pr# ~/palm/current_version/JOBS/$run_identifier/OUTPUT _pr_in
#
PARIN_O in:job:npe d3o# ~/palm/current_version/JOBS/$run_identifier/INPUT _o_p3d
PARIN_O in:job:npe d3or ~/palm/current_version/JOBS/$run_identifier/INPUT _o_p3dr
BININ_O in:loc d3or ~/palm/current_version/JOBS/$run_identifier/OUTPUT _o_d3d
#
BINOUT_O out:loc restart ~/palm/current_version/JOBS/$run_identifier/OUTPUT _o_d3d
#
RUN_CONTROL_O out:loc:tr d3o# ~/palm/current_version/JOBS/$run_identifier/MONITORING _o_rc
HEADER_O out:loc:tr d3o# ~/palm/current_version/JOBS/$run_identifier/MONITORING _o_header
PLOT1D_PAR_O out:loc:tr pro# ~/palm/current_version/JOBS/$run_identifier/OUTPUT _o_pr_par
PLOT1D_DATA_O out:loc:tr pro# ~/palm/current_version/JOBS/$run_identifier/OUTPUT _o_pr_in
}}}
The coupled ocean model filenames in the first column (e.g. [../../iofiles#PARIN_O PARIN_O]) must be specified as given [../../iofiles here]; the file connection identifiers (e.g.  {{{d3o#}}}) and the file name extension (e.g.  {{{_o_p3d}}}) may be changed at the user's discretion.\\\\
The coupler requires the following parameters to be equal in both [../../iofiles#PARIN PARIN] and [../../iofiles#PARIN_O PARIN_O]: [../../d3par#dt_coupling dt_coupling], [../../d3par#end_time end_time], [../../d3par#restart_time restart_time], [../../d3par#dt_restart dt_restart]. Furthermore the model domain length in x- and y-direction in ocean and atmosphere has to be same. In the coupled atmosphere executable, [../../initialization_parameters#bc_pt_b bc_pt_b] is internally set and does not need to be prescribed; in the coupled ocean executable, [../../initialization_parameters#bc_uv_t bc_uv_t] is internally set ('' 'neumann' '') and does not need to be prescribed. The coupled ocean parameter file [../../iofiles#PARIN_O PARIN_O] should include dummy REAL value assignments to both [../../initialization_parameters#top_momentumflux_u top_momentumflux_u] and [../../initialization_parameters#top_momentumflux_v top_momentumflux_v] (e.g. top_momentumflux_u = 0.0, top_momentumflux_v = 0.0) to enable the momentum flux coupling.\\\\
The coupling interval [../../d3par#dt_coupling dt_coupling] must be explicitly set. In order to ensure synchronous coupling throughout the simulation, dt_coupling should be chosen larger than [../../d3par#dt_max dt_max].\\
\\
=== [=#precursor_runs Precursor runs] === 

It's also possible to perform precursor runs (one atmospheric and one oceanic), followed by a coupled restart run to save computational time owing to different spin-up times in atmosphere and ocean. To use the precursor run control, the parameter [../../initialization_parameters#coupling_start_time coupling_start_time] must be set in both, the precursor runs as well as in the coupled restart run.\\
\\
==== Configuration of the parameter files ====
For the precursor runs the following relation must be valid:\\
\\
{{{ end_time - coupling_start_time = 0}}}.\\
\\
Here, {{{coupling_start_time}}} refers to the time period, in which the precursor runs shall run uncoupled. In the output data, the time will show negative values during this period. In the restart run, [../../initialization_parameters#coupling_start_time coupling_start_time] must be set equal to the settings in the precursor runs. 
For the coupled restart run the following relation must be valid:\\
\\
{{{ 
end_time - coupling_start_time = X,
}}}
where {{{X}}} is equal in the atmospheric and the oceanic model.
\\
===== Example =====
Precursor run (atmosphere):\\
{{{
end_time            = 3600.0
coupling_start_time = 3600.0
}}}
\\
\\
Precursor run (ocean):\\
{{{
end_time            = 40000.0
coupling_start_time = 40000.0
}}}
\\
Coupled restart run (atmosphere):\\
{{{
end_time            = 23600.0
coupling_start_time = 3600.0
}}}
\\
Coupled restart run (ocean):\\
{{{
end_time            = 60000.0
coupling_start_time = 40000.0
}}}
\\

==== Starting ({{{mrun}}})====
The atmospheric precursor run can be started like a usual atmospheric run:\\
\\
{{{
mrun -d examplerun ... -r “d3# ... restart”,
}}}
\\
while the oceanic precursor run requires the {{{-y}}} option:\\
\\
{{{
mrun -d examplerun ... -r “d3o# ... restart” -y
}}}
\\
The coupled restart then is carried out by:\\
\\
{{{
mrun -d examplerun ... -r “d3r d3or ... restart” -Y "...".
}}}
\\

==== Remarks ====
During the precursor runs, the time exhibits negative values such that both, atmosphere and ocean, restart at time zero in the coupled run.
The parameter [../../d3par#restart_time restart_time] will not take effect during the precursor runs and its value is related to the time in the coupled run.