source: palm/trunk/DOC/app/chapter_5.0.html @ 62

Last change on this file since 62 was 62, checked in by raasch, 15 years ago

Id string added to all html files

  • Property svn:keywords set to Id
File size: 23.0 KB
1<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
3<meta http-equiv="CONTENT-TYPE" content="text/html; charset=windows-1252"><title>PALM
4chapter 5.0</title> <meta name="GENERATOR" content="StarOffice 7 (Win32)"> <meta name="AUTHOR" content="Siegfried Raasch"> <meta name="CREATED" content="20041103;12380720"> <meta name="CHANGED" content="20041123;15532356"> <meta name="KEYWORDS" content="parallel LES model"> <style>
6@page { size: 21cm 29.7cm }
9<body style="direction: ltr;" lang="en-US"><h2 style="line-height: 100%;"><font size="4">5.0
10Installation of the
12<font color="#000000">This chapter
13describes the installation of PALM on a Linux workstation (local host)
14and a suitable remote computer, on which the
15model runs are to be carried out. The local host is used to
16start batch jobs with <b>mrun</b> and to analyze the
18which are produced by the model on the remote host and send back to the
19local host. Alternatively, <span style="font-weight: bold;">mrun</span>
20can also be used to start PALM on the local host in interactive mode or
21as a batch job.</font><p style="line-height: 100%;"></p><p style="line-height: 100%;"><font color="#000000"><span style="font-weight: bold; text-decoration: underline;">Requirements</span></font></p><p style="line-height: 100%;"><font color="#000000">The
22installation and operation of PALM requires at mimimum (on both, the
23local and the remote host, unless stated otherwise):</font></p><ol><li><font color="#000000">The AT&amp;T Korn-shell (ksh) must be
24available under<span style="font-family: Courier New,Courier,monospace;"> </span></font><font style="font-family: Courier New,Courier,monospace;" color="#000000">/bin/ksh</font><font color="#000000">.
25Most of the current Linux distributions are using the public domain
26Korn-shell (pdksh). The PALM scripts <span style="font-weight: bold;">mrun</span> and <span style="font-weight: bold;">mbuild</span> do not work
27with the pdksh! Please replace it by the AT&amp;T ksh. If the
28AT&amp;T ksh path is different from<span style="font-family: Courier New,Courier,monospace;"> </span></font><font style="font-family: Courier New,Courier,monospace;" color="#000000">/bin/ksh</font><font color="#000000">,
29please modify the first lines in <span style="font-weight: bold;">mrun</span>
30and <span style="font-weight: bold;">mbuild</span>,
31respectively.</font></li><li><font color="#000000">The
32NetCDF-library with version number not earlier than 3.6.0-p1 (for
33NetCDF, see under <a href=""></a>).</font></li><li><font color="#000000">A FORTRAN90/95 compiler.</font></li><li><font color="#000000">The Message Passing Interface (MPI), at
34least on the remote host, if the parallel version of PALM shall be used.</font></li><li><font color="#000000">On the local host, the revision control
35system <span style="font-weight: bold;">subversion</span>
36(see <a href=""></a>).
37This is already included in many Linux distributions (e.g. SuSe). <span style="font-weight: bold;">subversion requires port 3690 to
38be open for tcp/udp.</span> <span style="font-weight: bold;">If
39there are firewall restrictions concerning this port, the PALM code
40cannot be accessed.</span> The
41user needs a permit to access the PALM repository. For getting a permit
42please contact the PALM group (<a href=""></a>)
43and define a username under which you like to access the repository.
44You will then receive a password which allows the access under this
45name.</font></li><li><font color="#000000">A
46job queueing system must be available on the remote host. Currently, <span style="font-weight: bold;">mrun</span> can handle
47LoadLeveler (IBM-AIX) and NQS/PBS (Linux-Clusters, NEC-SX).</font></li><li><font color="#000000">ssh/scp-connections to and from the remote
48host must not be blocked by a firewall.<br></font></li></ol><font color="#000000"><span style="font-weight: bold;"></span></font><font color="#000000"> Currently, <span style="font-weight: bold;">mrun</span> is configured
49to be used on a limited number of selected machines. These are
50IBM-Regatta at computing center HLRN in Hannover (<span style="font-style: italic;">ibmh</span>), Berlin (<span style="font-style: italic;">ibmb</span>), at KISTI,
51Korea (<span style="font-style: italic;">ibms</span>),
52at Yonsei University, Seoul (<span style="font-style: italic;">ibms</span>),
53on NEC-SX6/8 systems at DKRZ, Hamburg (<span style="font-style: italic;">nech</span>) and RIAM,
54Kyushu University, Fukuoka (<span style="font-style: italic;">neck</span>),
55as well as on the Linux cluster of IMUK (<span style="font-style: italic;">lcmuk</span>) and Tokyo
56Institute of Technology (<span style="font-style: italic;">lctit</span>).
57The strings given in brackets are the systems names under which <span style="font-weight: bold;">mrun</span> indentifies the
58different hosts. mrun should also work on other systems, but this
59requires some modifications in the scripts <span style="font-weight: bold;">mrun</span>, <span style="font-weight: bold;">mbuild</span>, and <span style="font-weight: bold;">subjob</span>. Future PALM
60versions will allow to configure the scripts for different kinds of
61systems by just using a configuration file.</font><p style="line-height: 100%;"><font color="#000000">The
62examples given in this chapter refer to an
63installation of PALM on an IMUK Linux workstation and the IBM-Regatta
64system of
65the HLRN used as&nbsp;remote host. They are just called local and
66remote host from now on. </font></p><p style="line-height: 100%;">
67</p><p style="line-height: 100%;">The installation
68process requires a valid
69account on the local and on the remote host as well.<br>
70&nbsp; </p>
71<ul> All hosts (local as well as remote) are
72accessed via the secure shell (ssh). The user must establish
73passwordless login using the private/public-key mechanism (see e.g. the
74<a href="">HLRN
75documentation</a>). <b>To ensure proper function of mrun,
76passwordless login must be
77established in both directions, from the local to the remote host as
78well as from the remote to the local host!</b> Test this by
80out e.g. on the local host: <p style="line-height: 100%;"><span style="font-family: monospace;">&nbsp;&nbsp;&nbsp;
81</span><span style="font-family: Courier New,Courier,monospace;">ssh&nbsp;
82&lt;username on remote host&gt;@&lt;remote
83IP-address&gt;</span><br> </p> <p style="line-height: 100%;">and on the remote host:<br>
84</p> <p style="line-height: 100%;"><span style="font-family: monospace;">&nbsp;&nbsp;&nbsp;
85</span><span style="font-family: Courier New,Courier,monospace;">ssh&nbsp;
86&lt;username on local host&gt;@&lt;local IP-adddress&gt;</span><br>
87</p> In both cases you should&nbsp;not be
88prompted for a password. <b>Before continuing the further
90process, this must be absolutely guaranteed!</b> This must also
92guaranteed for <b>all</b> other remote hosts, on which
93PALM shall run.
94</ul><p style="line-height: 100%;"></p><p style="line-height: 100%;"><span style="font-weight: bold; text-decoration: underline;">Package
95Installation</span></p><p style="line-height: 100%;">In
96the <b>first installation step</b> a
97set of directories must be created both on the local and on the
98remote host. These directories are:</p><div style="margin-left: 40px; font-family: Courier New,Courier,monospace;">~/job_queue<br></div><div style="margin-left: 40px; font-family: Courier New,Courier,monospace;">~/palm<br></div><div style="margin-left: 40px; font-family: Courier New,Courier,monospace;">~/palm/current_version<br></div><div style="margin-left: 40px;"><span style="font-family: Courier New,Courier,monospace;">~/palm/current_version/JOBS</span><br></div><p style="line-height: 100%;">The names of these directories
100freely selectable (except <tt><font size="2">~/job_queue</font></tt>),
101however new users should use them as suggested, since many
102examples in this documentation as well as all example files are
103assuming these settings. The directory <span style="font-family: monospace;">~/palm/current_version</span>
104on the local host will be called the working directory from now on. </p>
105<p style="line-height: 100%;"><font color="#000000">In
106the <b>second
108a working copy of the recent version of the PALM software package,
109including the source code, scripts, documentation, etc.&nbsp; must
111copied to the working directory (</font><font color="#000000">local
112host!) by executing the following
113commands. Replace &lt;your username&gt; by the name that you
114chose to
115access the repository, and &lt;#&gt; by any of the available
117releases, e.g. "<span style="font-style: italic;">3.1c</span>"
118(new releases will be anounced by email to the PALM mailing list).</font></p><div style="margin-left: 40px;"><span style="font-family: Courier New,Courier,monospace;">cd
119~/palm/current_version</span><br style="font-family: Courier New,Courier,monospace;"><span style="font-family: Courier New,Courier,monospace;">svn
120checkout --username &lt;your username&gt;
121svn://;#&gt; trunk</span><br></div><p style="line-height: 100%;"><font color="#000000">You
122will then be prompted for your password. After finishing, the
123subdirectory</font><font style="font-family: Courier New,Courier,monospace;" color="#000000"> trunk </font><font color="#000000">should
124appear in your working directory. It contains a number of further
125subdirectories which contain e.g. the PALM source code (</font><font style="font-family: Courier New,Courier,monospace;" color="#000000">SOURCE</font><font color="#000000">)
126and the scripts for running PALM (</font><font style="font-family: Courier New,Courier,monospace;" color="#000000">SCRIPTS</font><font color="#000000">).</font></p><p style="line-height: 100%;">Alternatively,&nbsp;executing</p><pre style="line-height: 100%; margin-left: 40px; font-family: Courier New,Courier,monospace;">svn checkout --username &lt;your username&gt; svn://;#&gt; abcde</pre><p style="line-height: 100%;">will place your working copy in a
127directory named<span style="font-family: Courier New,Courier,monospace;"> abcde </span>instead
128of a directory named<span style="font-family: Courier New,Courier,monospace;"> trunk</span>.
129But keep in mind that you will have to adjust several paths given
130below, if you do not use the default directory<span style="font-family: Courier New,Courier,monospace;"> trunk</span>.</p><p style="line-height: 100%;">Please never touch any file in
131your working copy of PALM, unless you&nbsp;know exactly what you
132are doing.</p><p style="line-height: 100%; font-weight: bold;"><span style="font-weight: normal;">You can also get a copy of the
133most recent code by executing</span></p><pre style="line-height: 100%; margin-left: 40px; font-family: Courier New,Courier,monospace;">svn checkout --username &lt;your username&gt; svn:// trunk</pre>However,
134this version may contain bugs and new features may not be documented. <span style="font-weight: bold;">In future PALM releases,
135repository access to this most recent&nbsp;version will
136probably be restricted to the PALM developers.</span><p style="line-height: 100%;"></p><p style="line-height: 100%;"><font color="#000000"><span style="font-weight: bold; text-decoration: underline;">Package
137Configuration</span></font></p><p style="line-height: 100%;"><font color="#000000">To
138use the PALM scripts, the PATH variable has to be extended and the
139environment variable PALM_BIN has to be set (on local and remote host)
140in the respective profile of the users default shell (e.g. in<span style="font-family: Courier New,Courier,monospace;"> .profile</span>,
141if<span style="font-family: Courier New,Courier,monospace;">
142ksh </span>is used):</font></p><p style="line-height: 100%; margin-left: 40px;"><font style="font-family: Courier New,Courier,monospace;" color="#000000">export
143PATH=$HOME/palm/current_version/trunk/SCRIPTS:$PATH</font><font color="#000000"><br><span style="font-family: Courier New,Courier,monospace;">export
144PALM_BIN=</span></font><font style="font-family: Courier New,Courier,monospace;" color="#000000">$HOME/palm/current_version/trunk/SCRIPTS</font></p><p style="line-height: 100%;"><font color="#000000">You
145may habe to login again in order to activate these settings.</font></p><font color="#000000">On the local and on the remote host, some
146small helper/utility programs have to be installed, which are e.g.
147later used by <span style="font-weight: bold;">mrun</span>
148or for PALM data postprocessing. The installation is done by <span style="font-weight: bold;">mbuild</span>. This script
149requires a configuration file<span style="font-family: Courier New,Courier,monospace;">
150.mrun.config</span>, which will be also used by <span style="font-weight: bold;">mrun</span> in the
151following. A copy has to be put into the working directory under the
152name<span style="font-family: Courier New,Courier,monospace;">
153.mrun.config </span>by</font><p style="line-height: 100%; margin-left: 40px;"><font color="#000000"><span style="font-family: Courier New,Courier,monospace;">cp
154trunk/SCRIPTS/.mrun.config.default .mrun.config</span></font></p><font color="#000000">Beside many other things, this file contains
155typical installation parameters
156like compiler name, compiler options, etc.
157for a set of different remote hosts. Please edit this file, uncomment
158lines like</font><div style="margin-left: 40px; font-family: Courier New,Courier,monospace;">#%remote_username
159&nbsp;&lt;replace by your ... username&gt; &nbsp;
160&lt;remote-computer-shortcut&gt;</div><p style="line-height: 100%;"><font color="#000000">by
161removing the first hash (<span style="font-family: Courier New,Courier,monospace;">#</span>)
162character and replace the string "<span style="font-family: Courier New,Courier,monospace;">&lt;replace
163by ...&gt;</span>" by your username on the respective host
164given in the <span style="font-family: Courier New,Courier,monospace;">&lt;remote-computer-shortcut&gt;</span>.
165You only have to uncomment lines for those hosts on which you intend to
166use PALM.</font></p><p style="line-height: 100%;"><font color="#000000">After modifying the configuration file, the
167respective executables are generated by executing</font></p><p style="line-height: 100%; margin-left: 40px; font-family: Courier New,Courier,monospace;"><font color="#000000">mbuild -u -h lcmuk<br>mbuild -u -h ibmh</font></p><p style="line-height: 100%;"><font color="#000000">The
168second call also copies the PALM scripts like <span style="font-weight: bold;">mrun</span> and <span style="font-weight: bold;">mbuild</span> to the remote
169host.</font></p><p style="line-height: 100%;"></p><p style="line-height: 100%;"><font color="#000000"><span style="font-weight: bold; text-decoration: underline;">Pre-Compilation
170of PALM Code</span></font></p><font color="#000000">
171</font><br><font color="#000000">To avoid the
172re-compilation of the complete source code for each model run, PALM
173willl be pre-compiled once on the remote host by again using the script
174<span style="font-weight: bold;">mbuild</span>. Due
175to the use of
176FORTRAN modules in the source code, the subroutines must be compiled
177in a certain order. Therefore the so-called <i>make</i>
179is used (see the respective man-page of the Unix operating system),
180requiring a<span style="font-family: Courier New,Courier,monospace;">
181</span></font><font style="font-family: Courier New,Courier,monospace;" color="#000000">Makefile</font><font color="#000000">,
182in which the dependencies are described. This file is found in
183subdirectory <span style="font-family: Courier New,Courier,monospace;">trunk/SOURCE<span style="font-family: Times New Roman,Times,serif;">, where
184also the PALM code is stored</span></span>. The compiled
185sources (object
186files) are
187stored&nbsp;on the remote computer in the default directory
188</font><font style="font-family: Courier New,Courier,monospace;" color="#000000">~/palm/current_version/MAKE_DEPOSITORY</font>.<font color="#000000"><span style="font-weight: bold;"></span></font><p style="line-height: 100%;"><font color="#000000">The
189pre-compilation for the remote host (here the IBM-Regatta of HLRN) is
190done by</font><span style="font-family: monospace;"></span></p><div style="margin-left: 40px;"><span style="font-family: Courier New,Courier,monospace;">mbuild
191-h ibmh&nbsp;&nbsp;</span></div><p style="line-height: 100%;"><span style="font-family: Courier New,Courier,monospace;">mbuild
192</span>will prompt some queries<font color="#000000">,
193which must all be
194answered "y" by the user. The compiling process </font><font color="#000000">will take some time.<span style="font-family: Courier New,Courier,monospace;"> </span></font><font style="font-family: Courier New,Courier,monospace;" color="#000000">mbuild</font><font color="#000000"><span style="font-family: Courier New,Courier,monospace;"> </span>transfers
195the respective compiler calls to the remote
196host where they are carried out interactively. You can follow the
197progress at the terminal window, where also error messages
198are displayed (hopefully not for this standard installation). By just
199entering </font></p><p style="line-height: 100%;">
200</p><blockquote style="line-height: 100%; font-family: Courier New,Courier,monospace;"><font color="#000000">mbuild</font></blockquote>
201<p style="line-height: 100%;"><font color="#000000">PALM
203be (consecutively) pre-compiled for all remote hosts listed in
204the configuration file. If you want to compile for the local host only,
205please enter</font></p><p style="line-height: 100%; margin-left: 40px;"><font color="#000000"><span style="font-family: Courier New,Courier,monospace;">mbuild
206-h lcmuk</span> </font>
207</p><p style="line-height: 100%;"></p><p style="line-height: 100%; font-weight: bold; text-decoration: underline;"><font color="#000000">Installation Verification</font></p><font color="#000000">As a last step,
208after the compilation has been finished, the PALM installation has to
209be verified. For this
210purpose a simple test run is carried out. This once again requires the <b>mrun
211</b>configuration file (described in </font><a href="chapter_3.2.html"><font color="#000080">chapter
2123.2</font></a><font color="#000000">), as well
213as the </font><a href=""><font color="#000080">parameter
214file</font></a><font color="#000000">
215(described in </font><a href="chapter_4.4.html"><font color="#000080">chapter
2164.4</font></a>)<font color="#000000">. The
217parameter file must be
218copied from the PALM working copy by<br>
219</font><div style="margin-left: 40px;"><span style="font-family: monospace;"></span><font style="font-family: Courier New,Courier,monospace;" color="#000000">mkdir -p JOBS/example/INPUT<br>cp
220trunk/INSTALL/example_p3d&nbsp;JOBS/example/INPUT/example_p3d</font><font color="#000000"><span style="font-family: Courier New,Courier,monospace;"></span></font></div>
221<p style="line-height: 100%;"><font color="#000000">The
222test run can
223now be started by executing the command </font>
224</p><pre style="margin-left: 1cm; margin-right: 1cm; margin-bottom: 0.5cm; line-height: 100%; font-family: Courier New,Courier,monospace;"><font color="#000000">mrun -d example -h ibmh -K parallel -X 8 -T 8 -t 500 -q cdev -r &ldquo;d3# pr#&rdquo;</font></pre>
225<font color="#000000">This specific run
226will be carried out on 8 PEs and is allowed to use up to 500 seconds
227CPU time. After pressing &lt;return&gt;, the most important
228settings of
229the job are displayed at the terminal window
230and the user is prompted for o.k. (&ldquo;</font><font style="font-family: Courier New,Courier,monospace;" color="#000000">y</font><font color="#000000">&rdquo;).
231Next, a message of the queuing system like &ldquo;<i>Request</i>
233<i>Submitted to queue&hellip; by&hellip;&rdquo; </i>should
234be displayed. Now the job is
235queued and either started immediately or at a later time, depending on
237current workload of the remote host. Provided that it is executed
238immediately and that all things work as designed, the job protocol of
239this run will appear under the file name<span style="font-family: Courier New,Courier,monospace;"> </span></font><font style="font-family: Courier New,Courier,monospace;" color="#000000">~/job_queue/ibmh_example</font><font color="#000000"><span style="font-family: Courier New,Courier,monospace;"> </span>no
240more than a few minutes later. The content of this
241file should be carefully examined for any error messages.</font><p style="line-height: 100%;">
242</p><p style="line-height: 100%;">Beside the job
243protocol and according to
244the configuration file and arguments given for <b>mrun</b>
245options<span style="font-family: Courier New,Courier,monospace;">
246-d </span>and<span style="font-family: Courier New,Courier,monospace;"> -r</span><tt>,</tt>further
247files should be found in
249directories</p><div style="margin-left: 40px; font-family: Courier New,Courier,monospace;">~/palm/current_version/JOBS/example/MONITORING</div><p style="line-height: 100%;">and</p>
250<p style="line-height: 100%;"><span style="font-family: monospace;">&nbsp;&nbsp;&nbsp;
251</span><span style="font-family: Courier New,Courier,monospace;">~/palm/current_version/JOBS/example/OUTPUT</span><span style="font-family: monospace;"></span><br>
252</p><p style="line-height: 100%;">Please compare the
253contents of file</p>
254<p style="line-height: 100%;"><span style="font-family: monospace;">&nbsp;&nbsp;&nbsp;
255</span><span style="font-family: Courier New,Courier,monospace;">~/palm/current_version/JOBS/example/MONITORING/ibmh_example_rc</span><br>
256</p><p style="line-height: 100%;">with those of the
257example result file which can be found under<span style="font-family: Courier New,Courier,monospace;">
258trunk/INSTALL/example_rc</span>., e.g. by using the<span style="font-family: Courier New,Courier,monospace;"><span style="font-family: Times New Roman,Times,serif;"> standard</span>
259diff </span>command:</p><p style="line-height: 100%; margin-left: 40px; font-family: Courier New,Courier,monospace;">diff
261trunk/INSTALL/example_rc</p><p style="line-height: 100%;">where
262it is assumed that your working directory is<span style="font-family: Courier New,Courier,monospace;">
263~/palm/current_version</span>.</p><span style="font-weight: bold;">You should not find any
264difference between these two files</span>, except of the run date
265and time displayed at the top of the file header. <font color="#000000">If
266the file contents are identical, the installation is successfully
268<p style="line-height: 100%;">If you have any problems
269with the PALM
270installation, the members of the PALM working group are pleased to
271help you. &nbsp; </p>
272<hr><p style="line-height: 100%;"><font color="#000080"><font color="#000080"><a href="chapter_4.6.html"><font color="#000080"><img src="left.gif" name="Grafik1" align="bottom" border="2" height="32" width="32"></font></a><a href="Inhaltsverzeichnis.html"><font color="#000080"><img src="up.gif" name="Grafik2" align="bottom" border="2" height="32" width="32"></font></a><a href="chapter_5.1.html"><font color="#000080"><img src="right.gif" name="Grafik3" align="bottom" border="2" height="32" width="32"></font></a></font></font></p>
273<p style="line-height: 100%;"><i>Last
274change:&nbsp;</i> $Id: chapter_5.0.html 62 2007-03-13 02:52:40Z raasch $</p>
Note: See TracBrowser for help on using the repository browser.