palmbuild technical description

Introduction

palmbuild is the shell script to compile PALM and further utilities required for pre- and postprocessing the PALM data. It is invoked by the automatic installer as part of the installation process, or it can be called manually, and puts the created binaries in a folder named MAKE_DEPOSITORY_<ci>, where <ci> is the configuration identifier. It is also automatically invoked by the script palmrun in two different ways, but only if palmrun is called manually. If palmrun cannot find pre-compiled PALM sources, it calls palmbuild to create the MAKE_DEPOSITORY. Furthermore, manual calls of palmrun always call palmbuild to generate a specific folder (names SOURCES_FOR_RUN_...) that contains all binaries for the specific run, which may be re-used by automatic restart runs (see automatic calls further below).

Manual calls of palmbuild are required in case that a new PALM code version has been checked out from the repository, or if a modified working copy of the code shall be used for PALM runs.

palmbuild is steered by options as well as settings in the configuration file. It compiles code based on the unix makefile mechanism.

How to call palmbuild manually

Manual calls of palmbuild are used to generate or update the compiled PALM sources (binaries). Specific informations e.g. about the compiler and the compiler options to be used need to be specified in the configuration file. Relevant variables to be specified are source_path, user_source_path, compiler_name, compiler_name_ser, cpp_options, make_options, compiler_options and linker_options. See configuration file for a description of these variables. Since you can have different configuration files (for different compilers, compiler options, or hosts), you need to specify the configuration file to be used by palmbuild. Assuming a configuration file .palm.config.abcde, you need to enter

   palmbuild -c abcde

abcde is the so-called configuration identifier. After entering the command, informative messages will appear in the terminal:

#------------------------------------------------------------------------# 
| palmbuild  1.0  Rev: 3210 $              Thu Aug 30 09:29:56 CEST 2018 | 
| PALM code       Rev: 3220                                              | 
|                                                                        | 
| called on:   bora (IP:130.75.105.103)                                  | 
| config file: /home/raasch/palm/current_version/.palm.config.abcde      | 
| makefile:    /home/raasch/palm/current_version/trunk/SOURCE/Makefile   | 
| source path: /home/raasch/palm/current_version/trunk/SOURCE            | 
|                                                                        | 
| config. identifier: abcde                                              | 
| local depository:   /home/raasch/palm/current_version/MAKE_DEPOSITORY_ | 
|                     abcde                                              | 
| username:           raasch                                             | 
| address:            130.75.105.103                                     | 
| compiler:           mpif90                                             | 
| serial compiler:    ifort                                              | 
| make options:       -j 8                                               | 
| cpp options:        -cpp -D__parallel ......                           | 
| compiler options:   -fpe0 -O3 ......                                   | 
| linker options:     -fpe0 -O3 ......                                   | 
#------------------------------------------------------------------------# 

 >>> continue (y(es)/c(ontinue)/a(bort)) ?  

If y is entered, compilation will be started and compiler messages will appear. After successful completion, you will find the binaries in the directory given in the configuration file by base_directory under the folder name MAKE_DEPOSITORY_abcde.

In case of a configuration file for PALM runs on a remote host, sources are copied via scp to the remote host and the compiler is called via ssh on the remote host too. Folder MAKE_DEPOSITORY_abcde is created under base_directory on the remote host.

In case you have updated the PALM code, you need to call palmbuild again:

   cd ~/palm/current_version
   svn update trunk
   palmbuild -c abcde

If you have changed the compiler options in the configuration file and like to re-compile the PALM code with the new options, you need to touch the source code files in your working copy of the repository first,

   touch trunk/SOURCE/*.f90
   palmbuild -c abcde

because otherwise the make mechanism would see no changes in the source code and would not compile at all (message make: Nothing to be done). Alternatively, you may delete the MAKE_DEPOSITORY folder:

   rm -rf MAKE_DEPOSITORY_abcde
   palmbuild -c abcde

If you have more than one configuration file, you need to call palmbuild separately for each configuration.

Automatic calls of palmbuild from palmrun

palmbuild is automatically called by palmrun, but only if palmrun has been manually called (i.e. if the palmrun call has been entered by the user in a terminal). There are two different calls for two different purposes:

1. If palmrun does not find a MAKE_DEPOSITORY folder, it calls palmbuild with the respective configuration identifier (e.g. palmbuild -c abcde in order to pre-compile PALM.

2. palmbuild is called to compile sources for the specific PALM run. This may be user-interface code (if there is any), or modified code from the working copy (i.e. from .../trunk/SOURCE) that has been specified by palmrun option -s. palmrun has put the respective source files (plus all other files from the MAKE_DEPOSITORY) into a newly created folder named SOURCES_FOR_RUN_<ci>_<ri>, where <ci> is the configuration identifier and <ri> is the run identifier that have been specified with palmrun options -c and -r. This SOURCES_FOR_RUN-folder is created in the directory that has been specified by variable base_directory in the configuration file. palmbuild copies this folder to directory $fast_io_catalog/SOURCES_FOR_RUN_<ci>_<ri> and compiles the user-interface or/and modified sources (if there are any). Variable fast_io_catalog is also defined in the configuration file. In case of remote jobs, the SOURCES_FOR_RUN folder is copied via scp and the compiler is invoked via ssh. The internal palmbuild call generally looks like

      palmbuild -v -c <ci> -r <ri>
   

   where <ci and ri are as given by the respective palmrun options. Option -V is added if the corresponding palmrun option -V has been set. For explanations of the internal options see the table below.

palmbuild is never called by palmrun calls from within a batch job, or by automatic calls of palmrun to generate restart jobs. Reasons are to avoid compilation within batch jobs (which may be very inefficient if the job runs on many cores), and because automatic restart runs should and can re-use the binaries that have been created by the initial run.

palmbuild options

Only option -c and -v should be used for manual calls of palmbuild. Other options are internal options that only work if palmbuild is automatically called from within palmrun. --- in the default value column means that the respective option has no argument.

option	default value	meaning
-c	default	configuration identifier. Tells palmbuild which configuration file is to be used.
-r	" "	run identifier, as specified with palmrun option -r
-v	---	to run in silent mode. Switches off most messages and queries
-V	---	do not compile sources for the SOURCES_FOR_RUN folder

How does palmbuild operate?

A detailed list of consecutive steps that are carried out will follow soon ...