Changes between Version 25 and Version 26 of doc/app/palm_config


Ignore:
Timestamp:
Nov 21, 2018 1:40:04 PM (6 years ago)
Author:
kanani
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • doc/app/palm_config

    v25 v26  
    1 = '''This page is under construction! ''' =
    2 \\\\\\
    3 
    4 = The PALM configuration file
    5 
    6 Running PALM with the {{{palmrun}}} script or compiling PALM with {{{palmbuild}}} requires a configuration file in the working directory from where the scripts are called. The configuration file contains information about
    7 
    8  * the compiler and compiler options to be used
    9  * paths to required libraries (e.g. MPI, netCDF, or fftw)
    10  * preprocessor directives to switch on/off special features of PALM
    11  * paths to be used for storing PALM's input and output files
    12  * paths where user-interface routines and job protocol files can be found
    13  * the execute command to be used for starting the PALM executable
    14  * settings of other UNIX environment variable
    15  * batch job directives
     1= The PALM configuration file =
     2
     3Running PALM with the {{{palmrun}}} script or compiling PALM with {{{palmbuild}}} requires a '''configuration file''' in the working directory from where the scripts are called. The configuration file contains information about
     4
     5 * the compiler and compiler options to be used\\\\
     6 * paths to required libraries (e.g. MPI, NetCDF, or fftw)\\\\
     7 * pre-processor directives to switch on/off special features of PALM\\\\
     8 * paths to be used for storing PALM's input and output files\\\\
     9 * paths where user-interface routines and job protocol files can be found\\\\
     10 * the execute command to be used for starting the PALM executable\\\\
     11 * settings of other UNIX environment variable\\\\
     12 * batch job directives\\\\
    1613 * unix commands to be executed before / after the PALM code is started, or that should be carried out in case of errors that appeared during the run
    1714
    18 If PALM has been successfully installed by the automatic installer, the installer creates a configuration file named {{{.palm.config.default}}} in the working directory of the user (which by default is assumed to be {{{$HOME/palm/current_version}}}). This file should contain all relevant information to run PALM on the respective computer.
    19 
    20 The installer may fail for various reasons, e.g. because of mismatches in the user's system configuration. Problems may appear if the user has installed more than one FORTRAN compiler, and/or if the netCDF libraries have been generated with a different compiler than is used for compiling PALM. Furthermore, the automatic installer cannot be used on many larger computer systems where software (compilers, libraries) is managed via the {{{module}}} environment, or if you like to run batch jobs an a remote host. In all these cases, the user has to setup the configuration file manually.
    21 
    22 
    23 == How to generate a configuration file manually
     15If PALM has been successfully installed by the automatic installer, the installer creates a configuration file named {{{.palm.config.default}}} in the working directory of the user, by default {{{$HOME/palm/current_version}}}. This file should contain all relevant information to run PALM on the respective computer in [wiki:doc/app/palmrun#interactive interactive mode]. If you encounter any problems, please check the [wiki:doc/install/automatic automatic installer] or [wiki:Help/FAQ FAQ] page. If you don't find a solution there, contact us via the [/newticket ticket system].\\
     16
     17For [wiki:doc/app/palmrun#batch batch jobs] and in [wiki:doc/app/palmrun#batch_remote remote mode], the user has to modify the configuration file manually.
     18
     19
     20== How to generate a configuration file manually ==
    2421
    2522As a first step, copy the default template of the configuration file (which is part of the PALM download) to your working directory:
     
    2926}}}
    3027
    31 === Configuration file format
    32 
    33 The configuration file is an ASCII file which may be opened with any editor. The format of the file is simple. Lines need to start either with {{{#}}}, {{{%}}}, {{{IC:}}}, {{{OC:}}}, {{{EC:}}}, {{{BD:}}}, or {{{BDT:}}}. Empty line are allowed too.
    34 
    35 * lines starting with a {{{#}}} in the first column are comment lines
     28=== Configuration file format ===
     29
     30The configuration file is an ASCII file which may be opened with any editor. Lines need to start with either\\
     31{{{#}}}, {{{%}}}, {{{IC:}}}, {{{OC:}}}, {{{EC:}}}, {{{BD:}}}, or {{{BDT:}}}. Empty lines are allowed.
     32
     33* {{{#}}} starts comment lines
    3634{{{
    3735# this is a comment line
    3836}}}
    3937
    40 * lines starting with a {{{%}}} in the first column are defining unix environment variables used in {{{palmrun}}} and {{{palmbuild}}}.
     38* {{{%}}} in the first column defines unix environment variables used in {{{palmrun}}} and {{{palmbuild}}}:
    4139{{{
    4240%var value of var
    4341}}}
    44   means that a variable named {{{var}}} with value {{{value of var}}} is created. There must be at least one blank between the variable name and its value. The value may contain an arbitrary number of blanks. The value can contain already defined variables:
     42  A variable named {{{var}}} with value {{{value of var}}} is created. There must be at least one blank between the variable name and its value. The value may contain an arbitrary number of blanks. The value can contain already defined variables:
    4543{{{
    4644%default_folder  /work/abcd
    47 %subfolder1  $default_folder/efgh
     45%subfolder1      $default_folder/efgh
    4846}}}
    49   which means that {{{subfolder1}}} has the value {{{/work/abcd/efgh}}}. You can also use any variables that are already defined within the {{{palmrun}}}/{{{palmbuild}}} scripts. The most important one is {{{jobname}}}, which value is given with {{{palmrun}}} option {{{-r}}}, and which defines the so-called ''jobname''. This variable is mentioned here because it is used in the default configuration files for naming I/O files and to better sort I/O files from different PALM runs in an organized folder structure. Beside the value replacement using the UNIX shell syntax (i.e. {{{$abcd}}} gives the value of variable {{{abcd}}}), another way to insert values of environment variables is to write them in double curly brackets, i.e. {{{ {{abcd}} }}} will be replaced by the value of {{{abce}}}. This way of variable replacement is required for some special variables and in the job directives.
    50 
    51 * lines starting with {{{IC:}}} define any unix commands that are executed by {{{palmrun}}} just before the PALM code is started. A typical input command might be
     47  which means that {{{subfolder1}}} has the value {{{/work/abcd/efgh}}}. You can also use any variables that are already defined within the {{{palmrun}}}/{{{palmbuild}}} scripts. The most important one is {{{run_identifier}}}, given by {{{palmrun}}} option {{{-r}}}, and which defines the so-called ''run identifier''. This variable is mentioned here because it is used in the default configuration files for naming I/O files and to better sort I/O files from different PALM runs in an organized folder structure. Beside the value replacement using the UNIX shell syntax (i.e. {{{$abcd}}} gives the value of variable {{{abcd}}}), another way to insert values of environment variables is to write them in double curly brackets, i.e. {{{ {{abcd}} }}} will be replaced by the value of {{{abce}}}. This way of variable replacement is required for some special variables and in the job directives.
     48
     49* {{{IC:}}} defines any unix commands that are executed by {{{palmrun}}} just before the PALM code is started. A typical input command might be
    5250{{{
    5351IC:ulimit -s unlimited
     
    5553  to increase the system's stack size (if this is not unlimited by default), which is required for PALM runs with larger memory demands.
    5654
    57 * lines starting with {{{OC:}}} define unix commands that are executed by {{{palmrun}}} just after the PALM code has stopped. For example, you may inform yourself about termination of the program by sending an email:
    58 {{{
    59 OC:echo "PALM simulation $jobname has finished" | mailx  username@email-address
    60 }}}
    61 
    62 * lines starting with {{{EC:}}} define unix commands that shall be executed in case that the PALM code or the {{{palmrun}}} script terminated because of any kind of error. You can restrict execution of error commands to specific kinds of error, e.g. errors that appear during PALM execution:
     55* {{{OC:}}} defines unix commands that are executed by {{{palmrun}}} just after the PALM code has stopped. For example, you may inform yourself about termination of the program by sending an email:
     56{{{
     57OC:echo "PALM simulation $run_identifier has finished" | mail  username@email-address
     58}}}
     59
     60* {{{EC:}}} defines unix commands that shall be executed in case that the PALM code or the {{{palmrun}}} script terminated because of any kind of error. You can restrict execution of error commands to specific kinds of error, e.g. errors that appear during PALM execution:
    6361{{{
    6462EC:[[ \$locat = execution ]]  &&  error-command
    6563}}}
    66 See the {{{palmrun}}} source code for other specific locations that are used in this script.
    67 
    68 * lines starting with {{{BD:}}} define directives that are required for batch jobs, i.e. if PALM shall be run in batch mode. Explanations for batch directives are given further below.
    69 
    70 * lines starting with {{{BDT:}}} define directives for an additional batch job that is required in case of running PALM in batch mode on a remote host. This additional job transfers the job protocol file of the main PALM job back to the local host.
    71 
    72 
    73 === Environment variables and their meaning
     64  Search the {{{palmrun}}} script (.../trunk/SCRIPTS) for other specific locations that are used in this script.
     65
     66* {{{BD:}}} defines directives that are required for batch jobs, i.e. if PALM shall be run in batch mode. Explanations for batch directives are given further below.
     67
     68* {{{BDT:}}} defines directives for an additional batch job that is required in case of running PALM in batch mode on a remote host. This additional job transfers the job's log file of the main PALM job back to the local host.
     69
     70
     71=== Environment variables and their meaning ===
    7472
    7573The [source:palm/trunk/SCRIPTS/.palm.config.default default template] contains settings for the minimum set of variables that always need to be defined, plus suggestions for other variables that you may need to uncomment, in case that you like/need to use them. The following table lists all relevant variables and their meaning. No default values are assigned for these variables in scripts {{{palmrun}}} and {{{palmbuild}}}. Values given in the template file are for a Linux system with Intel Fortran compiler, mpich-, fftw-, and NetCDF4-library that is locally used by the PALM group.
     
    7876|-----------
    7977|| base_data         || Directory where the PALM I/O-files are stored. This variable is used in the file configuration file (see [source:palm/trunk/SCRIPTS/.palm.iofiles .palm.iofiles], and the [wiki:doc/app/palm_iofiles description] of this file). Since this variable is also used to determine file locations on remote hosts, it must use the {{{~}}} instead of {{{$HOME}}}.
    80 || [=#ba_di base_directory || Working directory from where {{{palmrun}}} or {{{palmbuild}}} is called. This folder must contain the configuration file {{{.palm.config....}}}.
     78|| [=#ba_di base_directory] || Working directory from where {{{palmrun}}} or {{{palmbuild}}} is called. This folder must contain the configuration file {{{.palm.config....}}}.
    8179|| compiler_name     || Name of the FORTRAN compiler to be used to create the PALM executable. Typically, this is the name of a wrapper script like ''mpif90'' or e.g. ''ftn'' on Cray machines, which automatically invokes the required MPI library and MPI include file. If you don't have a wrapper script, you may need to explicitly give compiler options (see {{{compiler_options}}}) to provide paths to the library / include file. If you like to run PALM without MPI (serial mode, or OpenMP parallelization), you should not use a wrapper script and give the original compiler name instead.
    8280|| compiler_name_ser || FORTRAN compiler name to create non-MPI executables. This name is required, because {{{palmbuild}}} generates several helper programs for pre-/post-processing, which run in serial mode on just one code. Here you give the original compiler name, like ''ifort'', ''pgfortran'', ''gfortran'', or ''xlf95''. 
     
    8583|| defaultqueue      || Batch job queue to be used if no queue is explicitly given with {{{palmrun}}} option {{{-q}}}. 
    8684|| execute_command   || MPI command to start the PALM executable. \\  Please see your local MPI documentation about which command needs to be used on your system. The name of the PALM executable, usually the last argument of the execute command, must be {{{palm}}}. Typically, the command requires to give several further options like the number of MPI processes to be started, or the number of compute nodes to be used. Values of these options may change from run to run. Don't give specific values here and use variables (written in double curly brackets) instead which will be automatically replaced by {{{palmrun}}} with values that you have specified with respective {{{palmrun}}} options. As an example {{{aprun  -n {{mpi_tasks}}  -N {{tasks_per_node}}  palm}}} will be interpreted as {{{aprun  -n 240  -N 24  palm}}} if you call {{{palmrun ... -X240 -T24 ...}}}. See the batch job section below about further variables that are recognized by {{{palmrun}}}. 
    87 || fast_io_catalog   || Path to a file system with fast discs (if available). This folder is used so store the temporary catalog generated by {{{palmrun}}} during each run. It should also be used to store large I/O files (e.g. restart data or 3D-output) in order to reduce I/O time. This variable is used in the default {{{.palm.iofiles}}} for the restart data files. The folder must be accessible from all compute nodes, i.e. it must reside in a global file system. WARNING: {{{/tmp}}} will only work on single node systems! In case of batch jobs on remote hosts, the variable refers to a folder on the remote host. The variable has no default value and must be set by the user.
     85|| [=#fa_io fast_io_catalog]   || Path to a file system with fast discs (if available). This folder is used so store the temporary catalog generated by {{{palmrun}}} during each run. It should also be used to store large I/O files (e.g. restart data or 3D-output) in order to reduce I/O time. This variable is used in the default {{{.palm.iofiles}}} for the restart data files. The folder must be accessible from all compute nodes, i.e. it must reside in a global file system. WARNING: {{{/tmp}}} will only work on single node systems! In case of batch jobs on remote hosts, the variable refers to a folder on the remote host. The variable has no default value and must be set by the user.
    8886|| hostfile          || Name of the hostfile that is used by MPI to determine the nodes on which the MPI processes are started. \\\\ {{{palmrun}}} automatically generates the hostfile if you set {{{auto}}}. All MPI processes will then be started on the node on which {{{palmrun}}} is executed. The real name of the hostfile will then be set to {{{hostfile}}} (instead of {{{auto}}}) and, depending on your local MPI implementation, you may have to give this name in the {{{execute_command}}}. MPI implementations on large computer centers often do not require to explicitly specify a hostfile (in such a case you can remove this line from the configuration file), or the batch systems provides a hostfile which name you may access via environment variables (e.g. {{{$PBS_NODEFILE}}}) and which needs to be given in the {{{execute_command}}}. Please see your local system / batch system documentation about the hostfile policy on your system. 
    8987|| linker_options    || Compiler options to be used to link the PALM executable. Typically, these are paths to libraries used by PALM, e.g. NetCDF, FFTW, MPI, etc. You may repeat the options that you have given with {{{compiler_options}}} here. See your local system documentation / software manuals for required path settings. Requirements differ from system to system and also depend on the respective libraries that you are using. See [wiki:doc/app/recommended_compiler_options] for specific path settings that we, the PALM group, are using on our computers.  Be aware, that these settings probably will not work on your computer system. 
     
    102100|| ssh_key           || Name of the file from which the identity (private key) for public key authentication is read. This file is assumed to be in folder {{{$HOME/.ssh}}}. By default (if you omit this variable), file {{{id_dsa}}} or {{{id_rsa}}} is used.
    103101|| submit_command    || Full path to the command that has to be used to submit batch jobs on your system (either on the local, or on the remote host), including required option. See documentation of your batch system / computing center to find out which command has to be used. An example for a {{{moab}}} batch system could be {{{/opt/moab/default/bin/msub -E}}}. If you only know the command name (e.g. ''msub''), entering {{{which msub}}} on the local/remote host will give you the full path. 
    104 || user_source_path  || Path to the [wiki:doc/app/userint user interface routines]. The variable {{{jobname}}} that may be used in the default path is replaced by the argument given with {{{palmrun}}}-option {{{-r}}}.
     102|| user_source_path  || Path to the [wiki:doc/app/userint user interface routines]. The variable {{{run_identifier}}} that may be used in the default path is replaced by the argument given with {{{palmrun}}}-option {{{-r}}}.
    105103
    106104
     
    197195Output commands are carried out immediately after the PALM executable has finished, and before the output files are copied. A typical example would be to send a message as a reminder that execution has finished:
    198196{{{
    199 OC:echo "PALM simulation $jobname has finished" | mailx  username@email-address
     197OC:echo "PALM simulation $run_identifier has finished" | mailx  username@email-address
    200198}}}
    201199