source: palm/trunk/DOC/app/chapter_5.0.html @ 41

Last change on this file since 41 was 41, checked in by raasch, 17 years ago

small changes to documentation

File size: 22.9 KB
1<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
3  <meta http-equiv="CONTENT-TYPE" content="text/html; charset=windows-1252">
5  <title>PALM chapter 5.0</title><meta name="GENERATOR" content="StarOffice 7  (Win32)">
6  <meta name="AUTHOR" content="Siegfried Raasch">
7  <meta name="CREATED" content="20041103;12380720">
8  <meta name="CHANGED" content="20041123;15532356">
9  <meta name="KEYWORDS" content="parallel LES model">
10  <style>
11        <!--
12                @page { size: 21cm 29.7cm }
13        -->
14        </style></head>
15<body dir="ltr" lang="en-US">
16<h2 style="line-height: 100%;"><font size="4">5.0 Installation of the
18<font color="#000000">This chapter
19describes the installation of PALM on a Linux workstation (local host) and a suitable remote computer, on which the
20model runs are to be carried out. The local host is used to
21start batch jobs with <b>mrun</b> and to analyze the results
22which are produced by the model on the remote host and send back to the local host. Alternatively, <span style="font-weight: bold;">mrun</span> can also be used to start PALM on the local host in interactive mode or as a batch job.</font><p style="line-height: 100%;"><font color="#000000"></font></p><p style="line-height: 100%;"><font color="#000000"><span style="font-weight: bold; text-decoration: underline;">Requirements</span></font></p><p style="line-height: 100%;"><font color="#000000">The installation and operation of PALM requires at mimimum (on both, the local and the remote host, unless stated otherwise):</font></p><ol><li><font color="#000000">The AT&amp;T Korn-shell (ksh) must be available under<span style="font-family: Courier New,Courier,monospace;"> </span></font><font style="font-family: Courier New,Courier,monospace;" color="#000000">/bin/ksh</font><font color="#000000">. Most of the current Linux distributions are using the public domain Korn-shell (pdksh). The PALM scripts <span style="font-weight: bold;">mrun</span> and <span style="font-weight: bold;">mbuild</span> do not work with the pdksh! Please replace it by the AT&amp;T ksh. If the AT&amp;T ksh path is different from<span style="font-family: Courier New,Courier,monospace;"> </span></font><font style="font-family: Courier New,Courier,monospace;" color="#000000">/bin/ksh</font><font color="#000000">, please modify the first lines in <span style="font-weight: bold;">mrun</span> and <span style="font-weight: bold;">mbuild</span>, respectively.</font></li><li><font color="#000000">The NetCDF-library with version number not earlier than 3.6.0-p1 (for NetCDF, see under <a href=""></a>).</font></li><li><font color="#000000">A FORTRAN90/95 compiler.</font></li><li><font color="#000000">The Message Passing Interface (MPI), at least on the remote host, if the parallel version of PALM shall be used.</font></li><li><font color="#000000">On the local host, the revision control system <span style="font-weight: bold;">subversion</span> (see <a href=""></a>).
23This is already included in many Linux distributions (e.g. SuSe). <span style="font-weight: bold;">subversion requires port 3690 to be open for tcp/udp.</span> <span style="font-weight: bold;">If there are firewall restrictions concerning this port, the PALM code cannot be accessed.</span> The
24user needs a permit to access the PALM repository. For getting a permit
25please contact the PALM group (<a href=""></a>)
26and define a username under which you like to access the repository.
27You will then receive a password which allows the access under this
28name.</font></li><li><font color="#000000">A job queueing system must be available on the remote host. Currently, <span style="font-weight: bold;">mrun</span> can handle LoadLeveler (IBM-AIX) and NQS/PBS (Linux-Clusters, NEC-SX).</font></li><li><font color="#000000">ssh/scp-connections to and from the remote host must not be blocked by a firewall.<br></font></li></ol><font color="#000000"><span style="font-weight: bold;"></span></font><font color="#000000"> Currently, <span style="font-weight: bold;">mrun</span> is configured to be used on a limited number of selected machines. These are IBM-Regatta at computing center HLRN in Hannover (<span style="font-style: italic;">ibmh</span>), Berlin (<span style="font-style: italic;">ibmb</span>), at KISTI, Korea (<span style="font-style: italic;">ibms</span>), at Yonsei University, Seoul (<span style="font-style: italic;">ibms</span>), on NEC-SX6/8 systems at DKRZ, Hamburg (<span style="font-style: italic;">nech</span>) and RIAM, Kyushu University, Fukuoka (<span style="font-style: italic;">neck</span>), as well as on the Linux cluster of IMUK (<span style="font-style: italic;">lcmuk</span>) and Tokyo Institute of Technology (<span style="font-style: italic;">lctit</span>). The strings given in brackets are the systems names under which <span style="font-weight: bold;">mrun</span> indentifies the different hosts. mrun should also work on other systems, but this requires some modifications in the scripts <span style="font-weight: bold;">mrun</span>, <span style="font-weight: bold;">mbuild</span>, and <span style="font-weight: bold;">subjob</span>. Future PALM versions will allow to configure the scripts for different kinds of systems by just using a configuration file.</font><p style="line-height: 100%;"><font color="#000000">The examples given in this chapter refer to an
29installation of PALM on an IMUK Linux workstation and the IBM-Regatta
30system of
31the HLRN used as&nbsp;remote host. They are just called local and remote host from now on. </font></p><p style="line-height: 100%;">
33<p style="line-height: 100%;">The installation process requires a valid
34account on the local and on the remote host as well.<br>
35&nbsp; </p>
37  All hosts (local as well as remote) are
38accessed via the secure shell (ssh). The user must establish
39passwordless login using the private/public-key mechanism (see e.g. the
40  <a href="">HLRN documentation</a>).
41  <b>To ensure proper function of mrun, passwordless login must be
42established in both directions, from the local to the remote host as
43well as from the remote to the local host!</b> Test this by carrying
44out e.g. on the local host:
45  <p style="line-height: 100%;"><span style="font-family: monospace;">&nbsp;&nbsp;&nbsp;
46</span><span style="font-family: Courier New,Courier,monospace;">ssh&nbsp; &lt;username on remote host&gt;@&lt;remote IP-address&gt;</span><br>
47  </p>
48  <p style="line-height: 100%;">and on the remote host:<br>
49  </p>
50  <p style="line-height: 100%;"><span style="font-family: monospace;">&nbsp;&nbsp;&nbsp;
51</span><span style="font-family: Courier New,Courier,monospace;">ssh&nbsp; &lt;username on local host&gt;@&lt;local IP-adddress&gt;</span><br>
52  </p>
53  In both cases you should&nbsp;not be
54prompted for a password. <b>Before continuing the further installation
55process, this must be absolutely guaranteed!</b> This must also be
56guaranteed for <b>all</b> other remote hosts, on which PALM shall run.
58<p style="line-height: 100%;"></p><p style="line-height: 100%;"><span style="font-weight: bold; text-decoration: underline;">Package Installation</span></p><p style="line-height: 100%;">In the <b>first installation step</b> a
59set of directories must be created both on the local and on the
60remote host. These directories are:</p><div style="margin-left: 40px; font-family: Courier New,Courier,monospace;">~/job_queue<br></div><div style="margin-left: 40px; font-family: Courier New,Courier,monospace;">~/palm<br></div><div style="margin-left: 40px; font-family: Courier New,Courier,monospace;">~/palm/current_version<br></div><div style="margin-left: 40px;"><span style="font-family: Courier New,Courier,monospace;">~/palm/current_version/JOBS</span><br></div><p style="line-height: 100%;">The names of these directories are
61freely selectable (except <tt><font size="2">~/job_queue</font></tt>),
62however new users should use them as suggested, since many
63examples in this documentation as well as all example files are
64assuming these settings. The directory <span style="font-family: monospace;">~/palm/current_version</span> on the local host will be called the working directory from now on. </p>
65<p style="line-height: 100%;"><font color="#000000">In the <b>second
67a working copy of the recent version of the PALM software package,
68including the source code, scripts, documentation, etc.&nbsp; must be
69copied to the working directory (</font><font color="#000000">local host!) by executing the following
70commands. Replace &lt;your username&gt; by the name that you choosed to access the repository.</font></p><div style="margin-left: 40px;"><span style="font-family: Courier New,Courier,monospace;">cd ~/palm/current_version</span><br style="font-family: Courier New,Courier,monospace;"><span style="font-family: Courier New,Courier,monospace;">svn checkout --username &lt;your username&gt; svn://</span><br></div><p style="line-height: 100%;"><font color="#000000">You will then be prompted for your password. After finishing, the subdirectory</font><font style="font-family: Courier New,Courier,monospace;" color="#000000"> trunk </font><font color="#000000">should
71appear in your working directory. It contains a number of further
72subdirectories which contain e.g. the PALM source code (</font><font style="font-family: Courier New,Courier,monospace;" color="#000000">SOURCE</font><font color="#000000">) and the scripts for running PALM (</font><font style="font-family: Courier New,Courier,monospace;" color="#000000">SCRIPTS</font><font color="#000000">).</font></p><p style="line-height: 100%;">Alternatively,&nbsp;executing</p><pre style="line-height: 100%; margin-left: 40px; font-family: Courier New,Courier,monospace;">svn checkout --username &lt;your username&gt; svn:// abcde</pre><p style="line-height: 100%;">will place your working copy in a directory named<span style="font-family: Courier New,Courier,monospace;"> abcde </span>instead of a directory named<span style="font-family: Courier New,Courier,monospace;"> trunk</span>. But keep in mind that you will have to adjust several paths given below, if you do not use the default directory<span style="font-family: Courier New,Courier,monospace;"> trunk</span>.</p><p style="line-height: 100%;">Please never touch any file in your working copy of PALM, unless you&nbsp;know exactly what you are doing.</p><p style="line-height: 100%; font-weight: bold;">In future PALM releases, repository access to the most recent PALM version will
73probably be restricted because this is a test version. However, access
74to the last (and earlier) stable version(s) will still be possible with
75(e.g.)</p><pre style="line-height: 100%; margin-left: 40px; font-family: Courier New,Courier,monospace;">svn checkout --username &lt;your username&gt; svn:// trunk</pre><p style="line-height: 100%;"><font color="#000000"></font></p><p style="line-height: 100%;"><font color="#000000"><span style="font-weight: bold; text-decoration: underline;">Package Configuration</span></font></p><p style="line-height: 100%;"><font color="#000000">To
76use the PALM scripts, the PATH variable has to be extended and the
77environment variable PALM_BIN has to be set (on local and remote host)
78in the respective profile of the users default shell (e.g. in<span style="font-family: Courier New,Courier,monospace;"> .profile</span>, if<span style="font-family: Courier New,Courier,monospace;"> ksh </span>is used):</font></p><p style="line-height: 100%; margin-left: 40px;"><font style="font-family: Courier New,Courier,monospace;" color="#000000">export PATH=$HOME/palm/current_version/trunk/SCRIPTS:$PATH</font><font color="#000000"><br><span style="font-family: Courier New,Courier,monospace;">export PALM_BIN=</span></font><font style="font-family: Courier New,Courier,monospace;" color="#000000">$HOME/palm/current_version/trunk/SCRIPTS</font></p><p style="line-height: 100%;"><font color="#000000">You may habe to login again in order to activate these settings.</font></p><font color="#000000">On the local and on the remote host, some small helper/utility programs have to be installed, which are e.g. later used by <span style="font-weight: bold;">mrun</span> or for PALM data postprocessing. The installation is done by <span style="font-weight: bold;">mbuild</span>. This script requires a configuration file<span style="font-family: Courier New,Courier,monospace;"> .mrun.config</span>, which will be also used by <span style="font-weight: bold;">mrun</span> in the following. A copy has to be put into the working directory under the name<span style="font-family: Courier New,Courier,monospace;"> .mrun.config </span>by</font><p style="line-height: 100%; margin-left: 40px;"><font color="#000000"><span style="font-family: Courier New,Courier,monospace;">cp trunk/SCRIPTS/.mrun.config.default .mrun.config</span></font></p><font color="#000000">Beside many other things, this file contains typical installation parameters
79like compiler name, compiler options, etc.
80for a set of different remote hosts. Please edit this file, uncomment lines like</font><div style="margin-left: 40px; font-family: Courier New,Courier,monospace;">#%remote_username &nbsp;&lt;replace by your ... username&gt; &nbsp; &lt;remote-computer-shortcut&gt;</div><p style="line-height: 100%;"><font color="#000000">by removing the first hash (<span style="font-family: Courier New,Courier,monospace;">#</span>) character and replace the string "<span style="font-family: Courier New,Courier,monospace;">&lt;replace by ...&gt;</span>" by your username on the respective host given in the <span style="font-family: Courier New,Courier,monospace;">&lt;remote-computer-shortcut&gt;</span>. You only have to uncomment lines for those hosts on which you intend to use PALM.</font></p><p style="line-height: 100%;"><font color="#000000">After modifying the configuration file, the respective executables are generated by executing</font></p><p style="line-height: 100%; margin-left: 40px; font-family: Courier New,Courier,monospace;"><font color="#000000">mbuild -u -h lcmuk<br>mbuild -u -h ibmh</font></p><p style="line-height: 100%;"><font color="#000000">The second call also copies the PALM scripts like <span style="font-weight: bold;">mrun</span> and <span style="font-weight: bold;">mbuild</span> to the remote host.</font></p><p style="line-height: 100%;"><font color="#000000"></font></p><p style="line-height: 100%;"><font color="#000000"><span style="font-weight: bold; text-decoration: underline;">Pre-Compilation of PALM Code</span></font></p><font color="#000000">
82<br><font color="#000000">To avoid the
83re-compilation of the complete source code for each model run, PALM
84willl be pre-compiled once on the remote host by again using the script <span style="font-weight: bold;">mbuild</span>. Due to the use of
85FORTRAN modules in the source code, the subroutines must be compiled
86in a certain order. Therefore the so-called <i>make</i> mechanism
87is used (see the respective man-page of the Unix operating system),
88requiring a<span style="font-family: Courier New,Courier,monospace;"> </span></font><font style="font-family: Courier New,Courier,monospace;" color="#000000">Makefile</font><font color="#000000">,
89in which the dependencies are described. This file is found in subdirectory <span style="font-family: Courier New,Courier,monospace;">trunk/SOURCE<span style="font-family: Times New Roman,Times,serif;">, where also the PALM code is stored</span></span>. The compiled sources (object
90files) are
91stored&nbsp;on the remote computer in the default directory
92</font><font style="font-family: Courier New,Courier,monospace;" color="#000000">~/palm/current_version/MAKE_DEPOSITORY</font>.<font color="#000000"><span style="font-weight: bold;"></span></font><p style="line-height: 100%;"><font color="#000000">The
93pre-compilation for the remote host (here the IBM-Regatta of HLRN) is done by</font><span style="font-family: monospace;"></span></p><div style="margin-left: 40px;"><span style="font-family: Courier New,Courier,monospace;">mbuild -h ibmh&nbsp;&nbsp;</span></div><p style="line-height: 100%;"><span style="font-family: Courier New,Courier,monospace;">mbuild
94</span>will prompt some queries<font color="#000000">, which must all be
95answered "y" by the user. The compiling process </font><font color="#000000">will take some time.<span style="font-family: Courier New,Courier,monospace;"> </span></font><font style="font-family: Courier New,Courier,monospace;" color="#000000">mbuild</font><font color="#000000"><span style="font-family: Courier New,Courier,monospace;"> </span>transfers the respective compiler calls to the remote
96host where they are carried out interactively. You can follow the
97progress at the terminal window, where also error messages
98are displayed (hopefully not for this standard installation). By just
99entering </font></p><p style="line-height: 100%;">
101<blockquote style="line-height: 100%; font-family: Courier New,Courier,monospace;"><font color="#000000">mbuild</font></blockquote>
102<p style="line-height: 100%;"><font color="#000000">PALM will
103be (consecutively) pre-compiled for all remote hosts listed in
104the configuration file. If you want to compile for the local host only, please enter</font></p><p style="line-height: 100%; margin-left: 40px;"><font color="#000000"><span style="font-family: Courier New,Courier,monospace;">mbuild -h lcmuk</span> </font>
106<p style="line-height: 100%;"><font color="#000000"></font></p><p style="line-height: 100%; font-weight: bold; text-decoration: underline;"><font color="#000000">Installation Verification</font></p><font color="#000000">As a last step,
107after the compilation has been finished, the PALM installation has to
108be verified. For this
109purpose a simple test run is carried out. This once again requires the <b>mrun
110</b>configuration file (described in </font><a href="chapter_3.2.html"><font color="#000080">chapter
1113.2</font></a><font color="#000000">), as well as the </font><a href=""><font color="#000080">parameter
112file</font></a><font color="#000000"> (described in </font><a href="chapter_4.4.html"><font color="#000080">chapter
1134.4</font></a>)<font color="#000000">. The parameter file must be
114copied from the PALM working copy by<br>
116<div style="margin-left: 40px;"><span style="font-family: monospace;"></span><font style="font-family: Courier New,Courier,monospace;" color="#000000">mkdir -p JOBS/example/INPUT<br>cp trunk/INSTALL/example_p3d&nbsp;JOBS/example/INPUT/example_p3d</font><font color="#000000"><span style="font-family: Courier New,Courier,monospace;"></span></font></div>
117<p style="line-height: 100%;"><font color="#000000">The test run can
118now be started by executing the command </font>
120<pre style="margin-left: 1cm; margin-right: 1cm; margin-bottom: 0.5cm; line-height: 100%; font-family: Courier New,Courier,monospace;"><font color="#000000">mrun -d example -h ibmh -K parallel -X 8 -T 8 -t 500 -q cdev -r &ldquo;d3# pr#&rdquo;</font></pre>
121<font color="#000000">This specific run
122will be carried out on 8 PEs and is allowed to use up to 500 seconds
123CPU time. After pressing &lt;return&gt;, the most important settings of
124the job are displayed at the terminal window
125and the user is prompted for o.k. (&ldquo;</font><font style="font-family: Courier New,Courier,monospace;" color="#000000">y</font><font color="#000000">&rdquo;). Next, a message of the queuing system like &ldquo;<i>Request</i> &hellip;
126<i>Submitted to queue&hellip; by&hellip;&rdquo; </i>should be displayed. Now the job is
127queued and either started immediately or at a later time, depending on the
128current workload of the remote host. Provided that it is executed
129immediately and that all things work as designed, the job protocol of
130this run will appear under the file name<span style="font-family: Courier New,Courier,monospace;"> </span></font><font style="font-family: Courier New,Courier,monospace;" color="#000000">~/job_queue/ibmh_example</font><font color="#000000"><span style="font-family: Courier New,Courier,monospace;"> </span>no more than a few minutes later. The content of this
131file should be carefully examined for any error messages.</font><p style="line-height: 100%;">
135<p style="line-height: 100%;">Beside the job protocol and according to
136the configuration file and arguments given for <b>mrun</b>
137options<span style="font-family: Courier New,Courier,monospace;"> -d </span>and<span style="font-family: Courier New,Courier,monospace;"> -r</span><tt>,</tt>further files should be found in
139directories</p><div style="margin-left: 40px; font-family: Courier New,Courier,monospace;">~/palm/current_version/JOBS/example/MONITORING</div><p style="line-height: 100%;">and</p>
140<p style="line-height: 100%;"><span style="font-family: monospace;">&nbsp;&nbsp;&nbsp;
141</span><span style="font-family: Courier New,Courier,monospace;">~/palm/current_version/JOBS/example/OUTPUT</span><span style="font-family: monospace;"></span><br>
143<p style="line-height: 100%;">Please compare the contents of file</p>
144<p style="line-height: 100%;"><span style="font-family: monospace;">&nbsp;&nbsp;&nbsp;
145</span><span style="font-family: Courier New,Courier,monospace;">~/palm/current_version/JOBS/example/MONITORING/ibmh_example_rc</span><br>
147<p style="line-height: 100%;">with those of the example result file which can be found under<span style="font-family: Courier New,Courier,monospace;"> trunk/INSTALL/example_rc</span>., e.g. by using the<span style="font-family: Courier New,Courier,monospace;"><span style="font-family: Times New Roman,Times,serif;"> standard</span> diff </span>command:</p><p style="line-height: 100%; margin-left: 40px; font-family: Courier New,Courier,monospace;">diff &nbsp;JOBS/example/MONITORING/ibmh_example_rc trunk/INSTALL/example_rc</p><p style="line-height: 100%;">where it is assumed that your working directory is<span style="font-family: Courier New,Courier,monospace;"> ~/palm/current_version</span>.</p><span style="font-weight: bold;">You should not find any difference between these two files</span>, except of the run date and time displayed at the top of the file header. <font color="#000000">If
148the file contents are identical, the installation is successfully
150<p style="line-height: 100%;">If you have any problems with the PALM
151installation, the members of the PALM working group are pleased to
152help you. &nbsp; </p>
154<p style="line-height: 100%;"><font color="#000080"><font color="#000080"><a href="chapter_4.6.html"><font color="#000080"><img src="left.gif" name="Grafik1" align="bottom" border="2" height="32" width="32"></font></a><a href="Inhaltsverzeichnis.html"><font color="#000080"><img src="up.gif" name="Grafik2" align="bottom" border="2" height="32" width="32"></font></a><a href="chapter_5.1.html"><font color="#000080"><img src="right.gif" name="Grafik3" align="bottom" border="2" height="32" width="32"></font></a></font></font></p>
155<p style="line-height: 100%;"><i>Last change:&nbsp;</i> 02/03/07 (SR) </p>
Note: See TracBrowser for help on using the repository browser.