= Full manual installation - step by step instructions [[TracNav(doc/install/toc)]] [[NoteBox(warn,This page is currently under construction.)]] '''Before you start, please check if you have fulfilled all [wiki:doc/install installation requirements!]''' === [=#package_installation]First step: Package installation === The '''first installation step''' requires creating a set of directories on the local and, for the advanced method, on the remote host. These are: {{{ ~/job_queue ~/palm ~/palm/current_version ~/palm/current_version/JOBS }}} The names of these directories can be freely selected by adjusting the [wiki:doc/app/palm_config configuration file], however new users should choose them as suggested, since many examples in this documentation as well as all example files are based on these settings. The directory {{{~/palm/current_version}}} on the local host will be called the working directory from now on.\\\\ In the '''second step''' a working copy of the recent PALM revision, including the source code, scripts, helper routines, etc. must be copied to the working directory by entering the following commands. Replace {{{}}} by the name that you have chosen in the [https://palm.muk.uni-hannover.de/trac/register register form]. {{{ cd ~/palm/current_version svn checkout --username https://palm.muk.uni-hannover.de/svn/palm/trunk trunk }}} You will then be prompted for your password that you have set in the register form too. After completion, a subdirectory {{{trunk}}} should appear in your working directory. It contains a number of further subfolders, which contain e.g. the PALM source code ({{{SOURCE}}}) and the scripts for running PALM ({{{SCRIPTS}}}). '''Please never touch any file in your working copy of PALM, unless you know what you are doing.''' In case of any problems with the {{{svn checkout}}} command, the most probable reason are firewall settings on your system. Please check or ask your system administrator, if https-ports are open. \\\\ The above {{{svn checkout}}} command provides you the most recent revision (developer version) of PALM. In case that you like to use a fixed release, please enter {{{ svn checkout --username https://palm.muk.uni-hannover.de/svn/palm/tags/release-<#> trunk }}} where {{{<#>}}} can be any of the available PALM releases, e.g. "''5.0''". See the [wiki:doc/tec/releasenotes release notes] for a list of available releases. New releases will be announced via the PALM mailing list. === [=#package_configuration]Package configuration === Compilation and execution of PALM is mainly controlled by two shell scripts named [wiki:doc/app/palmbuild {{{palmbuild}}}] and [wiki:doc/app/palmrun {{{palmrun}] that are part of the download and reside in folder [[[.../trunk/SCRIPTS]]]. To use these scripts, you need to extend your {{{PATH}}}-variable, by adding a line {{{ export PATH=$HOME/palm/current_version/trunk/SCRIPTS:$PATH }}} in your {{{.bashrc}}} file. The change will be active in any newly opened terminal. You need to enter {{{ source $HOME/.bashrc }}} in order to activate it in your existing terminal session. {{{palmbuild}}} as well as {{{palmrun}}} are controlled by options as well as settings in the main configuration file that is expected to be in your working directory. You can get a default version of this file from the repository: {{{ cd ~/palm/current_version cp trunk/SCRIPTS/.palm.config.default .palm.config.default }}} The string after the last dot (here {{{default}}}) is the so-called ''configuration identifier'' which you can freely choose. As default, the scripts expect the configuration identifier {{{default}}}. The default configuration file that you have just copied contains settings for the Intel compiler, an FFTW-library, and a NetCDF4 library, which you need to adjust for your environment. See section [wiki:doc/app/palm_config PALM configuration file] about the format and meaning of entries in this file. The default settings are for running PALM in interactive mode. Replace strings in angular brackets {{{<...>}}} with values to be used on your system. You need to uncomment lines (by removing the {{{#}}} in the first column) to use PALM in batch mode. Default batch directives contained in the file are for a Torque/MOAB batch system on a Cray-XC40. You may also need to adjust these settings to your batch system. '''Warning: When editing the configuration file, please NEVER use the {{{TAB}}} key. Otherwise, the scripts may run into very confusing errors.''' Beside the default configuration files {{{.palm.config.default}}}, folder {{{trunk/SCRIPTS}}} contains additional configuration files which are already adjusted for specific environments, e.g. {{{.palm.config.crayh}}} can be used for a Cray-System at [https://www.hlrn.de HLRN], etc.. These files have to be edited in the same way as described above. \\\\ === [=#package_configuration]Compiling the PALM sources === After the configuration file has been modified, the PALM sources and helper routines can be compiled by entering {{{ cd ~/palm/current_version palmbuild -h default }}} Option {{{-h default}}} can be omitted, because {{{default}}} is the default. {{{palmbuild}}} will prompt some queries, which must all be answered "y". Depending on which compiler optimization options have been set in the configuration file, compilations may take up to 10 minutes. {{{palmbuild}}} is using the ''make'' mechanism. Dependencies are described in a [source:palm/trunk/SOURCE/Makefile Makefile]. {{{palmbuild}}} creates a folder {{{MAKE_DEPOSITORY_}}} in your working directory (in the default case it is {{{MAKE_DEPOSITORY_default}}}.\\\\ === [=#verification]Installation verification === As a last step, after the compilation has been finished, the PALM installation has to be verified. For this purpose a simple test run is carried out. This once again requires the '''mrun''' [[wiki:doc/app/configexample|configuration file]], as well as the [[wiki:doc/app/par|parameter file]]. The parameter file must be copied from the PALM working copy by {{{ mkdir -p JOBS/example_cbl/INPUT cp trunk/INSTALL/example_cbl_p3d JOBS/example_cbl/INPUT/example_cbl_p3d }}} The test run can now be started by executing the command {{{ mrun -d example_cbl -h lccrayh -K parallel -X 8 -T 8 -t 500 -q mpp1testq -r "d3# pr#" }}} This specific run will be carried out on 8 PEs and is allowed to use up to 500 seconds CPU time. After pressing , the most important settings of the job are displayed at the terminal window and the user is prompted for o.k. ("{{{y}}}"). Next, a message of the queuing system like "''Request … Submitted to queue… by…''" should be displayed. Now the job is queued and either started immediately or at a later time, depending on the current workload of the remote host. Provided that it is executed immediately and that all things work as designed, the job protocol of this run will appear under the file name {{{~/job_queue/lccrayh_example}}} no more than a few minutes later. The content of this file should be carefully examined for any error messages.\\\\ Beside the job protocol and according to the configuration file and arguments given for 'mrun' options {{{-d}}} and {{{-r}}}, further files should be found in the directories {{{ ~/palm/current_version/JOBS/example_cbl/MONITORING }}} and {{{ ~/palm/current_version/JOBS/example_cbl/OUTPUT }}} Please compare the contents of file {{{ ~/palm/current_version/JOBS/example_cbl/MONITORING/lccrayh_example_rc }}} with those of the example result file which can be found under {{{trunk/INSTALL/example_cbl_rc}}}, e.g. by using the standard {{{diff}}} command {{{ diff JOBS/example_cbl/MONITORING/lccrayh_example_cbl_rc trunk/INSTALL/example_cbl_rc }}} where it is assumed that your working directory is {{{~/palm/current_version}}}.\\\\ '''You should not find any difference between these two files''', except for the run date and time displayed at the top of the file header. If the file contents are identical, the installation is successfully completed.\\\\ === Passwordless login via ssh === All hosts (local as well as remote) are accessed via the secure shell (ssh). The user must establish passwordless login using the [[wiki:/doc/install/passwordless|private/public-key mechanism]] (HLRNIII users please see [[wiki:/doc/app/machine/hlrnIII|hints]]). '''To ensure proper function of mrun, passwordless login must be established in both directions, from the local to the remote host as well as from the remote to the local host! '''Test this by carrying out e.g. on the local host: {{{ ssh @ }}} and on the remote host: {{{ ssh @ }}} In both cases you should not be prompted for a password. '''Before continuing the further installation process, this must be absolutely guaranteed! '''It must also be guaranteed for '''all''' other remote hosts, on which PALM shall run.\\\\ Please note that on many remote hosts, passwordless login must also work '''within the remote host''', i.e. for ssh connections from the remote host to itself. Test this by executing on the remote host: {{{ ssh @ }}} You should not be prompted for a password.\\\\ === [=#other_machines]Configuration for other machines === Starting from version 3.2a, beside the default hosts (HLRN, etc.), PALM can also be installed and run on other Linux-Cluster-, IBM-AIX, or NEC-SX-systems. To configure PALM for a non-default host only requires to add some lines to the configuration file {{{.mrun.config}}}.\\\\ First, you have to define the host identifier (a string of arbitrary length) under which your local host shall be identified by adding a line {{{ %host_identifier }}} to the configuration file (best to do this in the section where the other default host identifiers are defined). Here {{{}}} must be the name of your local host as provided by the unix-command "{{{hostname}}}". The first characters of {{{}}} have to be "{{{lc}}}", if your system is (part of) a linux-cluster, "{{{ibm}}}", or "{{{nec}}}" in case of an IBM-AIX- or NEC-SX-system, respectively. For example, if you want to install on a linux-cluster, the line may read as {{{ %host_identifier foo lc_bar }}} In the second step, you have to give all informations neccessary to compile and run PALM on your local host by adding an additional section to the configuration file: {{{ %remote_username <1> parallel %tmp_user_catalog <2> parallel %compiler_name <3> parallel %compiler_name_ser <4> parallel %cpp_options <5> parallel %netcdf_inc <6> parallel %netcdf_lib <7> parallel %fopts <8> parallel %lopts <9> parallel }}} The section consists of four columns each separated by one or more blanks. The first column gives the name of the respective environment variable used by '''mrun''' and '''mbuild''', while the second column defines its value. The third column has to be the host identifier as defined above, and the last column in each line must contain the string "{{{parallel}}}". Otherwise, the respective line(s) will be interpreted as belonging to the setup for compiling and running a serial (non-parallel) version of PALM.\\\\ All brackets have to be replaced by the appropriate settings for your local host: * {{{<1>}}} is the username on your LOCAL host * {{{<2>}}} is the temporary directory in which PALM runs will be carried out * {{{<3>}}} is the compiler name which generates parallel code * {{{<4>}}} is the compiler name for generating serial code * {{{<5>}}} are the preprocessor options to be invoked. In most of the cases, it will be neccessary to adjust the MPI data types to double precision by giving {{{-DMPI_REAL=MPI_DOUBLE_PRECISION -DMPI_2REAL=MPI_2DOUBLE_PRECISION}}}. To switch on the netCDF support, you also have to give {{{-D__netcdf}}} and {{{-D__netcdf4}}} (if you like to have netCDF4/HDF5 data format; this requires a netCDF4 library!). * {{{<6>}}} is the compiler option for specifying the include path to search for the netCDF module/include files * {{{<7>}}} are the linker options to search for the netCDF library * {{{<8>}}} are the general compiler options to be used. You should allways switch on double precision (e.g. {{{-r8}}}) and code optimization (e.g. {{{-O2}}}). * {{{<9>}}} are the linker options * {{{}}} is the host identifier as defined before A typical example may be: {{{ %remote_username raasch lc_bar parallel %tmp_user_catalog /tmp lc_bar parallel %compiler_name mpif90 lc_bar parallel %compiler_name_ser ifort lc_bar parallel %cpp_options -DMPI_REAL=MPI_DOUBLE_PRECISION:-DMPI_2REAL=MPI_2DOUBLE_PRECISION:-D__netcdf lc_bar parallel %netcdf_inc -I:/usr/local/netcdf/include lc_bar parallel %netcdf_lib -L/usr/local/netcdf/lib:-lnetcdf lc_bar parallel %fopts -axW:-cpp:-openmp:-r8:-nbs lc_bar parallel %lopts -axW:-cpp:-openmp:-r8:-nbs:-Vaxlib lc_bar parallel }}} Currently (version 3.7a), depending on the MPI version which is running on your local host, the options for the execution command (which may be {{{mpirun}}} or {{{mpiexec}}}) may have to be adjusted manually in the '''mrun'''-script. A future version will allow to give the respective settings in the configuration file.\\\\ If you have any problems with the PALM installation, the members of the PALM working group are pleased to help you.\\\\\\ = [=#update]Installation of new / other versions, version update = The PALM group announces code revisions by emails send to the PALM mailing list. If you like to be put on this list, just send an email to raasch@muk.uni-hannover.de. Details about new releases can be found in the [../tec/changelog PALM change log].\\\\ Generally, there are two ways of installing new / other versions. You can install a version from the list of available PALM releases or you can update your current installation with the newest developer version of PALM.\\\\ If you have previously checked out the most recent (at that time) PALM developer version by using {{{ svn checkout ...../palm/trunk trunk }}} you can easily make an update to the newest version by changing into the working directory {{{~/palm/current_version}}} and executing {{{ svn update trunk }}} This updates all files in the PALM working copy in subdirectory {{{trunk}}}. The update may fail due the '''subversion''' rules, if you have modified the contents of trunk. In case of any conflicts with the repository, please refer to the '''subversion''' documentation on how to remove them. In order to avoid such conflicts, modifications of the default PALM code should be omitted and be restricted to the user-interface only (see [../app/userint here]).\\\\ Alternatively, you can install new or other releases in a different directory, eg. {{{ mkdir ~/palm/release-3.1c cd ~/palm/release-3.1c svn checkout --username https://palm.muk.uni-hannover.de/svn/palm/tags/release-3.1c trunk }}} However, this would require to carry out again the complete installation process described above. So far, different versions of PALM cannot be used at the same time. The PALM releases from {{{palm/tags}}} never have to be updated with "{{{svn update}}}", since these releases are frozen! \\\\ After updating the working copy, please check for any differences between your current configuration file ({{{.mrun.config}}}) and the default configuration files under {{{trunk/SCRIPTS/.mrun.config.}}} and adjust your current file, if neccessary.\\\\ The scripts and the pre-compiled code must then be updated via {{{ mbuild -u -h lcmuk mbuild -u -h ibmh mbuild -h ibmh }}} or via {{{ mbuild -u mbuild }}} on all remote hosts listed in the configuration file {{{.mrun.config}}}.\\\\ You can use '''subversion''' for code comparison between the different versions. Also, modified code can be committed to the repository, but this is restricted to PALM developers.\\\\ If you want to recompile PALM via {{{mbuild}}} after you have modified the configuration file {{{.mrun.config}}} (e.g. if you switch to a newer compiler or NetCDF version), you will have to perform the touch command on all source files: {{{ touch trunk/SOURCE/* . }}} because otherwise the {{{make}}} mechanism will not be able to recompile the code. As a last step, a suitable test run should be carried out. It should be carefully examined whether and how the results created by the new version differ from those of the old version. Possible discrepancies which go beyond the ones announced in the [../tec/changelog PALM change log] should be communicated as soon as possible to the PALM group.