Changes between Version 22 and Version 23 of doc/app/palmrun

Timestamp:

Apr 30, 2018 12:44:01 PM (7 years ago)

Author:

raasch

Comment:

--

doc/app/palmrun

@@ -201 +201 @@
 == How to run PALM in batch mode
 
-{{{palmrun}}} supports two different ways to run PALM in batch mode. It creates a batch job (a file containing directives for a queuing-system plus commands to run PALM) which is either be submitted to your local computer or to a remote computer. Running PALM in batch mode requires you to manually modify and extend your configuration file {{{.palm.config....}}}, and that a batch system (e.g. PBS or ...) is installed on the respective computer.
+Large simulation set-ups usually cannot be run interactively, since the large amount of required resources (memory as well as cpu-time) are only provided through batch environments. {{{palmrun}}} supports two different ways to run PALM in batch mode. It creates a batch job (a file containing directives for a queuing-system plus commands to run PALM) which is either submitted to your local computer or to a remote computer. Running PALM in batch mode requires you to manually modify and extend your configuration file {{{.palm.config....}}}, and that a batch system (e.g. PBS or ...) is installed on the respective computer.
 
 === Running PALM in batch on a local computer
@@ -207 +207 @@
 The local computer is the one where the commands that you enter in a terminal sessions are executed. This might be your local PC/workstation, or a login-node of a cluster-system / computer center where you are logged in via ssh. Regardless of the computer, it is assumed that PALM has been successfully installed on that machine, either using the automatic installer or via manual installation.
 
-As first step add information to your configuration file. You may edit an existing file (.e.g. {{{.palm.config.default}}}) or create a new one (e.g. by copying the default file to e.g. {{{.palm.config.batch}}} and then editing the new file. It is not necessarily required to create a new file, since you can use the same configuration file for running interactive jobs and batch jobs as well. Let's assume here that you have created a new file {{.palm.config.batch}}}. Edit this file and add ...
+For running PALM in batch mode you need to include additional options in the {{{palmrun}}} command to specify the system resources requested by the job, and to modify your configuration file. A minimum set of additional {{{palmrun}}} options is
+{{{
+   palmrun  .... -t <cputime>  -X <total number of cores>  -T <MPI tasks per node>  -q <queue>  -m <memory per core>
+}}}
+where
+ * {{{<cputime>}}} is the maximum CPU time (wall clock time) in seconds requested by the batch job
+ * {{{<total number of cores>}}} is the total number of CPU cores (not CPUs!) that shall be used for your run
+ * {{{<MPI tasks per node>}}} is the number of MPI tasks to be started on one node of the computer system. Typically, {{{<MPI tasks per node>}}} is chosen as the total number of CPU cores available on one node, e.g. if a node has two CPUs with 12 cores each, then {{{<MPI tasks per node> = 24}}}.
+ * {{{<queue>>> is the name of the batch job queue that you like to use. See your batch system documentation about available queues and keep in mind that usually each queue has special limits for requested resources.
+ * {{{<memory per core>}}} is the memory in MByte requested by each core
+
+As first step add information to your configuration file. You may edit an existing file (.e.g. {{{.palm.config.default}}}) or create a new one (e.g. by copying the default file to e.g. {{{.palm.config.batch}}} and then editing the new file. It is not necessarily required to create a new file, since you can use the same configuration file for running interactive jobs and batch jobs as well. Let's assume here that you have created a new file {{.palm.config.batch}}}. Edit this file and add those batch directives required by your batch system.  Keep in mind that there is a wide variety of batch systems and that many computer centers introduce their own special settings for these systems. Please read the documentation of your respective batch system carefully in order to figure out the required settings for your system (e.g. to run an MPI job on multiple cores). The following lines give a minimum example for the portable batch system (PBS).
+{{{
+BD:#!/bin/bash
+BD:#PBS -N {{job_id}}
+BD:#PBS -l walltime={{cpu_hours}}:{{cpu_minutes}}:{{cpu_seconds}}
+BD:#PBS -l nodes={{nodes}}:ppn={{tasks_per_node}}
+BD:#PBS -o {{job_protocol_file}}
+BD:#PBS -j oe
+BD:#PBS -q {{queue}}
+}}}
+Batch directive lines in the configuration file must start in the first column with string {{{BD:}}}, immediately followed by the directive of the respective batch system (the PBS directives must e.g. start with {{{#PBS}}} followed by a {{{blank}}}). Strings parenthesized by double curly brackets {{{ {{...}} }}} are variables used in {{{palmrun}}} and will be replaced by their respective values while {{{palmrun}}} creates the batch job file. A complete list of {{{palmrun}}} variables that can be used in batch directives is given in section [wiki:doc/app/batch_directives batch_directives].
+
+