= Full manual installation - step by step instructions [[TracNav(doc/install/toc)]] [[NoteBox(warn,This page is currently under construction.)]] '''Before you start, please check if you have fulfilled all [wiki:doc/install installation requirements!]''' The [wiki:doc/install/automatic automatic installer] normally cares for steps 1-5 that are described below. Failure of the automatic installation process is usually caused by inconsistencies in your software environment (e.g. mismatches between your compiler, NetCDF- and MPI- libraries) which will also cause failure of the manual installation. Anyhow, at least parts of the installation steps may be required to be carried out manually. For example, if your system has a very strict firewall and does not allow downloads from our repository, you may carry out the download (second step below) on a different system and copy the {{trunk}}} folder to your target system, before carrying on with the [wiki:doc/install/automatic automatic installer]. Installation and configuration for batch jobs cannot be done by the [wiki:doc/install/automatic automatic installer] and requires manual work in any case, as described further below. == [=#package_installation]First step: Package installation == The '''first installation step''' requires creating a set of directories on the local and, for the advanced method, on the remote host. These are: {{{ ~/job_queue ~/palm ~/palm/current_version ~/palm/current_version/JOBS }}} The names of these directories can be freely selected by adjusting the [wiki:doc/app/palm_config configuration file], however new users should choose them as suggested, since many examples in this documentation as well as all example files are based on these settings. The directory {{{~/palm/current_version}}} on the local host will be called the working directory from now on.\\\\ In the '''second step''' a working copy of the recent PALM revision, including the source code, scripts, helper routines, etc. must be copied to the working directory by entering the following commands. Replace {{{}}} by the name that you have chosen in the [https://palm.muk.uni-hannover.de/trac/register register form]. {{{ cd ~/palm/current_version svn checkout --username https://palm.muk.uni-hannover.de/svn/palm/trunk trunk }}} You will then be prompted for your password that you have set in the register form too. After completion, a subdirectory {{{trunk}}} should appear in your working directory. It contains a number of further subfolders, which contain e.g. the PALM source code ({{{SOURCE}}}) and the scripts for running PALM ({{{SCRIPTS}}}). '''Please never touch any file in your working copy of PALM, unless you know what you are doing.''' In case of any problems with the {{{svn checkout}}} command, the most probable reason are firewall settings on your system. Please check or ask your system administrator, if https-ports are open. \\\\ The above {{{svn checkout}}} command provides you the most recent revision (developer version) of PALM. In case that you like to use a fixed release, please enter {{{ svn checkout --username https://palm.muk.uni-hannover.de/svn/palm/tags/release-<#> trunk }}} where {{{<#>}}} can be any of the available PALM releases, e.g. "''5.0''". See the [wiki:doc/tec/releasenotes release notes] for a list of available releases. New releases will be announced via the PALM mailing list. == [=#package_configuration]Third step: Package configuration == Compilation and execution of PALM is mainly controlled by two shell scripts named [wiki:doc/app/palmbuild {{{palmbuild}}}] and [wiki:doc/app/palmrun {{{palmrun}] that are part of the download and reside in folder [[[.../trunk/SCRIPTS]]]. To use these scripts, you need to extend your {{{PATH}}}-variable, by adding a line {{{ export PATH=$HOME/palm/current_version/trunk/SCRIPTS:$PATH }}} in your {{{.bashrc}}} file. The change will be active in any newly opened terminal. You need to enter {{{ source $HOME/.bashrc }}} in order to activate it in your existing terminal session. {{{palmbuild}}} as well as {{{palmrun}}} are controlled by options as well as settings in the main configuration file that is expected to be in your working directory. You can get a default version of this file from the repository: {{{ cd ~/palm/current_version cp trunk/SCRIPTS/.palm.config.default .palm.config.default }}} The string after the last dot (here {{{default}}}) is the so-called ''configuration identifier'' which you can freely choose. As default, the scripts expect the configuration identifier {{{default}}}. The default configuration file that you have just copied contains settings for the Intel compiler, an FFTW-library, and a NetCDF4 library, which you need to adjust for your environment. See section [wiki:doc/app/palm_config PALM configuration file] about the format and meaning of entries in this file. The default settings are for running PALM in interactive mode. Replace strings in angular brackets {{{<...>}}} with values to be used on your system. You need to uncomment lines (by removing the {{{#}}} in the first column) to use PALM in batch mode. Default batch directives contained in the file are for a Torque/MOAB batch system on a Cray-XC40. You may also need to adjust these settings to your batch system. '''Warning: When editing the configuration file, please NEVER use the {{{TAB}}} key. Otherwise, the scripts may run into very confusing errors.''' Beside the default configuration files {{{.palm.config.default}}}, folder {{{trunk/SCRIPTS}}} contains additional configuration files which are already adjusted for specific environments, e.g. {{{.palm.config.crayh}}} can be used for a Cray-System at [https://www.hlrn.de HLRN], etc.. These files have to be edited in the same way as described above. \\\\ == [=#package_configuration]Fourth step: Compiling the PALM sources == After the configuration file has been modified, the PALM sources and helper routines can be compiled by entering {{{ cd ~/palm/current_version palmbuild -c default }}} Option {{{-c default}}} can be omitted, because {{{default}}} is the default. {{{palmbuild}}} will prompt some queries, which must all be answered "y". Depending on which compiler optimization options have been set in the configuration file, compilations may take up to 10 minutes. {{{palmbuild}}} is using the ''make'' mechanism. Dependencies are described in a [source:palm/trunk/SOURCE/Makefile Makefile]. {{{palmbuild}}} creates a folder {{{MAKE_DEPOSITORY_}}} in your working directory (in the default case it is {{{MAKE_DEPOSITORY_default}}}.\\\\ == [=#verification]Fifth step: Installation verification == As a last step, after the compilation has been finished, the PALM installation has to be verified. For this purpose a simple test run needs to be started using the script {{{palmrun}}}. In addition to the configuration file, {{{palmrun}}} requires a [wiki:doc/app/par parameter file] as well. The parameter file for the test case is provided as part of the download and needs to be copied first: {{{ cd ~/palm/current_version mkdir -p JOBS/example_cbl/INPUT cp trunk/TESTS/cases/example_cbl/INPUT/example_cbl_p3d JOBS/example_cbl/INPUT/example_cbl_p3d }}} Here, the string {{{example_cbl}}} acts as the so-called ''run identifier''. The test run can now be started by entering {{{ palmrun -r example_cbl -c default -X4 -a "d3#" }}} See the [wiki:doc/app/palmrun palmrun description] for detailed explanations of available options. This specific run will be carried out on 4 cores (if available on your system, others you may need to adjust the {{{-X}}} option). Most important settings of this run are displayed at the terminal window and you are prompted for o.k. ("{{{y}}}") to continue. Informations about the progress of the simulation will be output to the terminal. After {{{palmrun}}} has finished, you should find some result files in folder {{{JOBS/example_cbl/MONITORING}}}. Please compare the contents of file {{{ ~/palm/current_version/JOBS/example_cbl/MONITORING/example_cbl_rc }}} with those of the example result file that is provided under {{{trunk/TESTS/cases/example_cbl/MONITORING/example_cbl_rc}}}, e.g. by using the standard {{{diff}}} command {{{ cd ~/palm/current_version diff JOBS/example_cbl/MONITORING/example_cbl_rc trunk/TESTS/cases/example_cbl/MONITORING//example_cbl_rc }}} '''You should not find any difference between these two files if you are using the latest PALM revision''', except for the run date and time displayed at the top of the file header. If the file contents are identical, the installation is successfully completed.\\\\ == Installation for running PALM in batch mode == === Installation for batch jobs on the local machine === Running PALM in batch mode on your local computer (requires that the computer where you are logged in has a batch system running) requires to add appropriate batch directives to the configuration file as well as settings for variables like {{{local_jobcatalog}}}, {{{defaultqueue}}}, {{{memory}}}, and {{{submit_command}}}. Settings for {{{module_commands}}} and {{{login_init_cmd}}} may be needed too. See the [wiki:doc/app/palm_config configuration file description] for further details. In order to run PALM in batch mode, the installation process is the same as described above, but {{{palmrun}}} requires additional options and may look like this {{{ palmrun -r example_cbl -c default -X4 -T4 -t200 -m1000 -a "d3#" -q testqueue -b }}} The {{{-b}}} option is essential to tell {{{palmrun}}} to generate and submit a batch job. Otherwise, it will try to execute PALM interactively in your terminal session. Again, result files for verifying the installation can be found in folder {{{JOBS/example_cbl/MONITORING}}}, after the batch job has been executed. The protocol file of the batch job, which is typically created by every batch system, can be found in the folder that has been set by {{{local_jobcatalog}}} under the name {{{_}}}, which is {{{default_example_cbl}}} in the given example. Further informations about running PALM in batch mode on local machines can be found in the [wiki:doc/app/palmrun_quickstart palmrun quickstart guide]. === Batch jobs on a remote machine === Follow the installation steps described above. In addition to the settings for a local batch job, installation of PALM for running batch jobs on remote machines requires further additional entries in the configuration file, at least variables {{{remote_ip}}}, {{{remote_username}}}, and {{{remote_jobcatalog}}} need to be set. For further informations see the [wiki:doc/app/palmrun_quickstart palmrun quickstart guide] and the [wiki:doc/app/palmrun palmrun documentation]. Assuming a configuration file {{{.palm.config.remote_system}}}, compiling the PALM sources via {{{ palmbuild -c remote_system }}} will copy the PALM sources by {{{scp}}} from your local computer to the remote system and invokes the remote compiler using {{{ssh}}}. The binaries will be put in folder {{{$HOME/palm/current_version/MAKE_DEPOSITORY_remote_system}}} on the remote system. For using {{{palmrun}}}, additional batch directives have to be added to the configuration file in order to transfer back the job protocol file (see the [wiki:doc/app/palmrun_quickstart palmrun quickstart guide] for further details). The {{{palmrun}}} command for generating the test run then reads {{{ palmrun -r example_cbl -c remote_system -X4 -T4 -t200 -m1000 -a "d3#" -q testqueue }}} {{{palmrun}}} transfers back the result file via {{{scp}}} and you should find it on your local system in folder {{{JOBS/example_cbl/MONITORING}}} under the name {{{remote_system_example_cbl_rc}}} after the job on the remote system has finished. The job protocol file will also be copied to the folder that has been set by {{{local_jobcatalog}}}. Using {{{palmbuild}}} and {{{palmrun}}} for installing and running PALM on remote machines requires passwordless login via {{{scp}}} and {{{ssh}}}, as descrobed in the next section. === Passwordless login via ssh === All hosts (local as well as remote) are accessed via the secure shell (ssh). The user must establish passwordless login using the [[wiki:/doc/install/passwordless|private/public-key mechanism]] (HLRNIII users please see [[wiki:/doc/app/machine/hlrnIII|hints]]). '''To ensure proper function of {{{palmbuild}}} and {{{palmrun}}}, passwordless login must be established in '''both directions''', from the local to the remote host as well as from the remote to the local host! '''Test this by carrying out e.g. on the local host: {{{ ssh @ }}} and on the remote host: {{{ ssh @ }}} In both cases you should not be prompted for a password. '''Before starting with the installation process, this should be absolutely guaranteed! '''It must also be guaranteed for '''all''' other remote hosts, on which PALM shall run.\\\\ Please note that on many remote hosts, passwordless login must also work '''within the remote host''', i.e. for {{{ssh}}} connections from the remote host to itself (e.g. for connections from compute nodes to login nodes). Test this by executing on the remote host: {{{ ssh @ }}} You should not be prompted for a password.\\\\ == [=#update]Installation of new / other revisions, revision update == All code revisions are documented under [wiki:doc/tec/changelog]. The PALM group announces major code revisions via the PALM mailing list. You will be automatically set on the list by creating an account using the [[//trac/register|register form]].\\\\ Generally, there are two ways of installing new / other versions. You can install a version from the [wiki:doc/tec/releasenotes list of available PALM releases] or you can update your current installation with the newest developer revision of PALM.\\\\ If you have previously checked out the PALM developer revison by using {{{ svn checkout ...../palm/trunk trunk }}} you can easily make an update to the newest revision by {{{ cd ~/palm/current_version svn update trunk }}} This updates all files in the working copy in folder {{{trunk}}} (which is your working copy of the PALM repository). The update may fail due the '''subversion''' rules, if you have modified the contents of trunk. In case of any conflicts with the repository, please refer to the '''subversion''' documentation on how to remove them. In order to avoid such conflicts, modifications of the default PALM code should be omitted and be restricted to the user-interface only (see [../app/userint here]), except you are a PALM developer.\\\\ Alternatively, you can install new or other releases in a different directory, eg. {{{ mkdir ~/palm/release-4.0 cd ~/palm/release-4.0 svn checkout --username https://palm.muk.uni-hannover.de/svn/palm/tags/release-4.0 trunk }}} However, this requires to carry out again the complete installation process described above. So far, different versions of PALM cannot be used at the same time. The PALM releases from {{{palm/tags}}} never have to be updated with "{{{svn update}}}", since these versions are frozen! \\\\ The compiled PALM code and helper routines must then be updated via {{{ palmbuild -c default }}} or for any other configuration files that you are using.\\\\ You can use '''subversion''' for code comparison between the different revisions. Also, modified code can be committed to the repository, but this is restricted to PALM developers.\\\\ If you want to recompile PALM via {{{palmbuild}}} after you have modified the configuration file (e.g. if you changed compiler options or switched to other libraries), you need to apply the {{{touch}}} command on all source files in advance: {{{ touch trunk/SOURCE/* }}} because otherwise the {{{make}}} mechanism will not detect any source file that needs to be compiled. As an alternative, instead of ''touching'' the files, you may delete the {{{MAKE_DEPOSITORY}}} folder before calling {{{palmbuild}}}, but then the complete code will be re-compiled. As a last step, a suitable test run should be carried out. It should be carefully examined whether and how the results created by the new revision differ from those of the old version. Possible discrepancies which go beyond the ones announced in the [wiki:doc/tec/changelog PALM change log] should be communicated as soon as possible via our [/newticket ticket system].\\\\ == [=#multiple]Central installation for multiple users == For a working group with multiple users, who only apply the PALM code and have no need to modify the working copy of the repository, PALM needs to be installed only once. The system administrator (or any other person) has to do the installation as described above. Assuming that the working copy of the repository (folder {{{trunk}}}) is in folder {{{/path_to_the_central_installation}}}, and that a configuration file {{{.palm.config.default}}} has been created in that folder, each single user has to create his/her own working directory: {{{ mkdir $HOME/palm/current_version/JOBS cd $HOME/palm/current_version cp /path_to_the_central_installation/.palm.config.default . }}} Then edit the copied configuration file and modify folder paths as follows: {{{ %base_directory /path_to_the_central_installation %source_path /path_to_the_central_installation/trunk/SOURCE %base_data ~/palm/current_version/JOBS %user_source_path $HOME/palm/current_version/JOBS/$run_identifier/USER_CODE %local_username %fast_io_catalog }}} Every user will then use the pre-compiled version of PALM that has been created for the central installation under the configuration identifier {{{default}}}. {{{palmrun}}} can only be called with this configuration identifier ({{{palmrun -c default ....}}}). If other configurations are required, they have to be created ''by the administrator'' under the central installation {{{ cp /path_to_the_central_installation/trunk/SCRIPTS/.palm.config.default /path_to_the_central_installation/.palm.config.config_new # edit .palm.config.config_new cd /path_to_the_central_installation palmbuild -c config_new }}} After that, users need to copy and modify this configuration file as described above, and can use this configuration via {{{palmrun -c config_new ....}}}.