| | 34 | === [=#package_configuration]Package configuration === |
| | 35 | |
| | 36 | Compilation and execution of PALM is mainly controlled by two shell scripts named [wiki:doc/app/palmbuild {{{palmbuild}}}] and [wiki:doc/app/palmrun {{{palmrun}] that are part of the download and reside in folder [[[.../trunk/SCRIPTS]]]. To use these scripts, you need to extend your {{{PATH}}}-variable, by adding a line |
| | 37 | {{{ |
| | 38 | export PATH=$HOME/palm/current_version/trunk/SCRIPTS:$PATH |
| | 39 | |
| | 40 | }}} |
| | 41 | in your {{{.bashrc}}} file. The change will be active in any newly opened terminal. You need to enter |
| | 42 | {{{ |
| | 43 | source $HOME/.bashrc |
| | 44 | }}} |
| | 45 | in order to activate it in your existing terminal session. |
| | 46 | |
| | 47 | {{{palmbuild}}} as well as {{{palmrun}}} are controlled by options as well as settings in the main configuration file that is expected to be in your working directory. You can get a default version of this file from the repository: |
| | 48 | {{{ |
| | 49 | cd ~/palm/current_version |
| | 50 | cp trunk/SCRIPTS/.palm.config.default .palm.config.default |
| | 51 | }}} |
| | 52 | The string after the last dot (here {{{default}}}) is the so-called ''configuration identifier'' which you can freely choose. As default, the scripts expect the configuration identifier {{{default}}}. |
| | 53 | |
| | 54 | The default configuration file that you have just copied contains settings for the Intel compiler, an FFTW-library, and a NetCDF4 library, which you need to adjust for your environment. See section [wiki:doc/app/palm_config PALM configuration file] about the format and meaning of entries in this file. The default settings are for running PALM in interactive mode. Replace strings in angular brackets {{{<...>}}} with values to be used on your system. You need to uncomment lines (by removing the {{{#}}} in the first column) to use PALM in batch mode. Default batch directives contained in the file are for a Torque/MOAB batch system on a Cray-XC40. You may also need to adjust these settings to your batch system. |
| | 55 | |
| | 56 | '''Warning: When editing the configuration file, please NEVER use the {{{TAB}}} key. Otherwise, the scripts may run into very confusing errors.''' |
| | 57 | |
| | 58 | Beside the default configuration files {{{.palm.config.default}}}, folder {{{trunk/SCRIPTS}}} contains additional configuration files which are already adjusted for specific environments, e.g. {{{.palm.config.crayh}}} can be used for a Cray-System at [https://www.hlrn.de HLRN], etc.. These files have to be edited in the same way as described above. \\\\ |
| | 59 | |
| | 60 | |
| | 61 | === [=#package_configuration]Compiling the PALM sources === |
| | 62 | |
| | 63 | |
| | 64 | After the configuration file has been modified, the PALM sources and helper routines can be compiled by entering |
| | 65 | {{{ |
| | 66 | cd ~/palm/current_version |
| | 67 | palmbuild -h default |
| | 68 | }}} |
| | 69 | Option {{{-h default}}} can be omitted, because {{{default}}} is the default. |
| | 70 | |
| | 71 | {{{palmbuild}}} will prompt some queries, which must all be answered "y". Depending on which compiler optimization options have been set in the configuration file, compilations may take up to 10 minutes. {{{palmbuild}}} is using the ''make'' mechanism. Dependencies are described in a [source:palm/trunk/SOURCE/Makefile Makefile]. {{{palmbuild}}} creates a folder {{{MAKE_DEPOSITORY_<configuration identifier>}}} in your working directory (in the default case it is {{{MAKE_DEPOSITORY_default}}}.\\\\ |
| | 72 | |
| | 73 | |
| | 74 | |
| | 75 | === [=#verification]Installation verification === |
| | 76 | |
| | 77 | As a last step, after the compilation has been finished, the PALM installation has to be verified. For this purpose a simple test run is carried out. This once again requires the '''mrun''' [[wiki:doc/app/configexample|configuration file]], as well as the [[wiki:doc/app/par|parameter file]]. The parameter file must be copied from the PALM working copy by |
| | 78 | {{{ |
| | 79 | mkdir -p JOBS/example_cbl/INPUT |
| | 80 | cp trunk/INSTALL/example_cbl_p3d JOBS/example_cbl/INPUT/example_cbl_p3d |
| | 81 | }}} |
| | 82 | The test run can now be started by executing the command |
| | 83 | {{{ |
| | 84 | mrun -d example_cbl -h lccrayh -K parallel -X 8 -T 8 -t 500 -q mpp1testq -r "d3# pr#" |
| | 85 | }}} |
| | 86 | This specific run will be carried out on 8 PEs and is allowed to use up to 500 seconds CPU time. After pressing <return>, the most important settings of the job are displayed at the terminal window and the user is prompted for o.k. ("{{{y}}}"). Next, a message of the queuing system like "''Request … Submitted to queue… by…''" should be displayed. Now the job is queued and either started immediately or at a later time, depending on the current workload of the remote host. Provided that it is executed immediately and that all things work as designed, the job protocol of this run will appear under the file name {{{~/job_queue/lccrayh_example}}} no more than a few minutes later. The content of this file should be carefully examined for any error messages.\\\\ |
| | 87 | Beside the job protocol and according to the configuration file and arguments given for 'mrun' options {{{-d}}} and {{{-r}}}, further files should be found in the directories |
| | 88 | {{{ |
| | 89 | ~/palm/current_version/JOBS/example_cbl/MONITORING |
| | 90 | }}} |
| | 91 | and |
| | 92 | {{{ |
| | 93 | ~/palm/current_version/JOBS/example_cbl/OUTPUT |
| | 94 | }}} |
| | 95 | Please compare the contents of file |
| | 96 | {{{ |
| | 97 | ~/palm/current_version/JOBS/example_cbl/MONITORING/lccrayh_example_rc |
| | 98 | }}} |
| | 99 | with those of the example result file which can be found under {{{trunk/INSTALL/example_cbl_rc}}}, e.g. by using the standard {{{diff}}} command |
| | 100 | {{{ |
| | 101 | diff JOBS/example_cbl/MONITORING/lccrayh_example_cbl_rc trunk/INSTALL/example_cbl_rc |
| | 102 | }}} |
| | 103 | where it is assumed that your working directory is {{{~/palm/current_version}}}.\\\\ |
| | 104 | '''You should not find any difference between these two files''', except for the run date and time displayed at the top of the file header. If the file contents are identical, the installation is successfully completed.\\\\ |
| | 105 | |
| 53 | | |
| 54 | | |
| 55 | | === [=#package_configuration]Package configuration === |
| 56 | | |
| 57 | | To use the PALM scripts, the {{{PATH}}}-variable has to be extended and the environment variable {{{PALM_BIN}}} has to be set (on local and remote host) in the respective profile of the users default shell (e.g. in {{{.profile}}}, if {{{ksh}}} is used): |
| 58 | | {{{ |
| 59 | | export PATH=$HOME/palm/current_version/trunk/SCRIPTS:$PATH |
| 60 | | export PALM_BIN=$HOME/palm/current_version/trunk/SCRIPTS |
| 61 | | }}} |
| 62 | | You may have to login again in order to activate these settings.\\\\ |
| 63 | | On the local and on the remote host, some small helper/utility programs have to be installed, which are later used by '''mrun''' e.g. for PALM data postprocessing. The installation is done by '''mbuild'''. This script requires a configuration file {{{.mrun.config}}}, which will be also used by '''mrun''' in the following. A copy has to be put into the working directory under the name {{{.mrun.config}}} by |
| 64 | | {{{ |
| 65 | | cp trunk/SCRIPTS/.mrun.config.<compiler> .mrun.config |
| 66 | | }}} |
| 67 | | Please replace {{{<compiler>}}} with either {{{gfortran}}} or {{{ifort}}} in order to get the default configuration file for the respective compiler. Beside many other things, this file contains typical installation parameters like compiler name, compiler options, etc. for a set of different (remote) hosts. Please edit this file, uncomment lines like |
| 68 | | {{{ |
| 69 | | #%remote_username <replace by your ... username> <host identifier> |
| 70 | | }}} |
| 71 | | by removing the first hash (#) character and replace the string "{{{<replace by ...>}}}" by your username on the respective host given in the {{{<host identifier>}}}. You only have to uncomment lines for those hosts on which you intend to use PALM.\\\\ |
| 72 | | '''Warning: When editing the configuration file, please NEVER use the TAB key. Otherwise, very confusing errors may occur when mrun is executing.'''\\\\ |
| 73 | | Beside the default configuration files {{{.mrun.config.<compiler>}}}, the directory {{{trunk/SCRIPTS}}} contains additional configuration files which are already adjusted for special hosts, e.g. {{{.mrun.config.imuk}}} can be used at Hannover University, etc.. These files have to be edited in the same way as described above.\\\\ |
| 74 | | After modifying the configuration file, the respective executables are generated by executing |
| 75 | | {{{ |
| 76 | | mbuild -u -h lcmuk |
| 77 | | mbuild -u -h lccrayh |
| 78 | | }}} |
| 79 | | The second call also copies the PALM scripts (like '''mrun''' and '''mbuild''') to the remote host.\\\\ |
| 80 | | |
| 81 | | === [=#pre_compilation]Pre-compilation of PALM code === |
| 82 | | |
| 83 | | To avoid the re-compilation of the complete source code for each model run, PALM willl be pre-compiled once on the remote host by again using the script '''mbuild'''. Due to the use of FORTRAN modules in the source code, the subroutines must be compiled in a certain order. Therefore the so-called ''make'' mechanism is used (see the respective man-page of the Unix operating system), requiring a {{{Makefile}}}, in which the dependencies are described. This file is found in subdirectory {{{trunk/SOURCE}}}, where also the PALM code is stored. The compiled sources (object files) are stored on the remote computer in the default directory {{{~/palm/current_version/MAKE_DEPOSITORY_<block_descriptor>}}}, where {{{<block_descriptor>}}} is composed of the third (and fourth, if existing) column of the respective block in the configuration file (e.g. {{{lccrayh_parallel}}} for HLRN). |
| 84 | | |
| 85 | | The pre-compilation for the remote host (here the Cray XC30/40 system of HLRN) is done by |
| 86 | | {{{ |
| 87 | | mbuild -h lccrayh |
| 88 | | }}} |
| 89 | | 'mbuild' will prompt some queries, which must all be answered "y" by the user. The compiling process will take some time. '''mbuild''' transfers the respective compiler calls to the remote host where they are carried out interactively. You can follow the progress at the terminal window, where also error messages are displayed (hopefully not for this standard installation). By just entering |
| 90 | | {{{ |
| 91 | | mbuild |
| 92 | | }}} |
| 93 | | PALM will be (consecutively) pre-compiled for all remote hosts listed in the configuration file. If you want to compile for the local host only, please enter |
| 94 | | {{{ |
| 95 | | mbuild -h lcmuk |
| 96 | | }}} |
| 97 | | \\ |
| 98 | | |
| 99 | | === [=#verification]Installation verification === |
| 100 | | |
| 101 | | As a last step, after the compilation has been finished, the PALM installation has to be verified. For this purpose a simple test run is carried out. This once again requires the '''mrun''' [[wiki:doc/app/configexample|configuration file]], as well as the [[wiki:doc/app/par|parameter file]]. The parameter file must be copied from the PALM working copy by |
| 102 | | {{{ |
| 103 | | mkdir -p JOBS/example_cbl/INPUT |
| 104 | | cp trunk/INSTALL/example_cbl_p3d JOBS/example_cbl/INPUT/example_cbl_p3d |
| 105 | | }}} |
| 106 | | The test run can now be started by executing the command |
| 107 | | {{{ |
| 108 | | mrun -d example_cbl -h lccrayh -K parallel -X 8 -T 8 -t 500 -q mpp1testq -r "d3# pr#" |
| 109 | | }}} |
| 110 | | This specific run will be carried out on 8 PEs and is allowed to use up to 500 seconds CPU time. After pressing <return>, the most important settings of the job are displayed at the terminal window and the user is prompted for o.k. ("{{{y}}}"). Next, a message of the queuing system like "''Request … Submitted to queue… by…''" should be displayed. Now the job is queued and either started immediately or at a later time, depending on the current workload of the remote host. Provided that it is executed immediately and that all things work as designed, the job protocol of this run will appear under the file name {{{~/job_queue/lccrayh_example}}} no more than a few minutes later. The content of this file should be carefully examined for any error messages.\\\\ |
| 111 | | Beside the job protocol and according to the configuration file and arguments given for 'mrun' options {{{-d}}} and {{{-r}}}, further files should be found in the directories |
| 112 | | {{{ |
| 113 | | ~/palm/current_version/JOBS/example_cbl/MONITORING |
| 114 | | }}} |
| 115 | | and |
| 116 | | {{{ |
| 117 | | ~/palm/current_version/JOBS/example_cbl/OUTPUT |
| 118 | | }}} |
| 119 | | Please compare the contents of file |
| 120 | | {{{ |
| 121 | | ~/palm/current_version/JOBS/example_cbl/MONITORING/lccrayh_example_rc |
| 122 | | }}} |
| 123 | | with those of the example result file which can be found under {{{trunk/INSTALL/example_cbl_rc}}}, e.g. by using the standard {{{diff}}} command |
| 124 | | {{{ |
| 125 | | diff JOBS/example_cbl/MONITORING/lccrayh_example_cbl_rc trunk/INSTALL/example_cbl_rc |
| 126 | | }}} |
| 127 | | where it is assumed that your working directory is {{{~/palm/current_version}}}.\\\\ |
| 128 | | '''You should not find any difference between these two files''', except for the run date and time displayed at the top of the file header. If the file contents are identical, the installation is successfully completed.\\\\ |
