Changeset 62 for palm/trunk/DOC/app/chapter_3.3.html
- Timestamp:
- Mar 13, 2007 2:52:40 AM (18 years ago)
- File:
-
- 1 edited
Legend:
- Unmodified
- Added
- Removed
-
palm/trunk/DOC/app/chapter_3.3.html
r54 r62 1 1 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> 2 <html> 3 <head> 4 <meta http-equiv="CONTENT-TYPE" 5 content="text/html; charset=windows-1252"> 6 <title>PALM chapter 3.3</title> 7 <meta name="GENERATOR" content="StarOffice 7 (Win32)"> 8 <meta name="AUTHOR" content="Marcus Oliver Letzel"> 9 <meta name="CREATED" content="20040728;14053490"> 10 <meta name="CHANGED" content="20041112;14150257"> 11 <meta name="KEYWORDS" content="parallel LES model"> 12 <style> 13 <!-- 14 @page { size: 21cm 29.7cm } 15 --> 16 </style> 17 </head> 18 <body dir="ltr" lang="en-US"> 19 <h3 style="line-height: 100%;">3.3 Initialization and restart runs</h3> 20 <p style="line-height: 100%;">A job started by <b>mrun</b> will 2 <html><head> 3 <meta http-equiv="CONTENT-TYPE" content="text/html; charset=windows-1252"><title>PALM 4 chapter 3.3</title> <meta name="GENERATOR" content="StarOffice 7 (Win32)"> <meta name="AUTHOR" content="Marcus Oliver Letzel"> <meta name="CREATED" content="20040728;14053490"> <meta name="CHANGED" content="20041112;14150257"> <meta name="KEYWORDS" content="parallel LES model"> <style> 5 <!-- 6 @page { size: 21cm 29.7cm } 7 --> 8 </style></head> 9 10 <body style="direction: ltr;" lang="en-US"><h3 style="line-height: 100%;">3.3 Initialization and restart 11 runs</h3> 12 <p style="line-height: 100%;">A job started by <b>mrun</b> 13 will 21 14 - according to its requested computing time, its memory size 22 15 requirement and … … 35 28 very fast, jobs with higher requirements must wait longer (sometimes 36 29 several days). </p> 37 <p style="line-height: 100%;">Before the start of a model run the user 30 <p style="line-height: 100%;">Before the start of a model 31 run the user 38 32 must estimate how much CPU time the model will need for the simulation. 39 33 The necessary time in seconds has to be indicated with the mrun 40 <b>option</b> <tt><a 41 href="http://www.muk.uni-hannover.de/institut/software/mrun_beschreibung.html#Opt-t">-t</a></tt> 34 <b>option</b> <tt><a href="http://www.muk.uni-hannover.de/institut/software/mrun_beschreibung.html#Opt-t">-t</a></tt> 42 35 and has an influence on the job class into which the job is queued. Due 43 36 to the fact that the model usually uses a variable … … 49 42 which normally leads to an abort of the job as soon as the available 50 43 CPU time is consumed. In principle one could solve this problem by 51 setting a very generously estimated value for <u><font 52 style="font-size: 10pt;" size="2"><font face="Cumberland, monospace">-t</font></font></u>, 44 setting a very generously estimated value for <u><font style="font-size: 10pt;" size="2"><font face="Cumberland, monospace">-t</font></font></u>, 53 45 but this will possibly lead to the disadvantage that the queued job has 54 46 to wait longer for execution.<br> 55 </p> 56 <p style="line-height: 100%;">To avoid this problem <b>mrun </b>offers 57 the possibility of so-called <b>restart runs</b>. During the model 47 </p><p style="line-height: 100%;">To avoid this 48 problem <b>mrun </b>offers 49 the possibility of so-called <b>restart runs</b>. During 50 the model 58 51 run PALM continuously examines how much time is left for the 59 52 execution of the job. If the run is not completed and finished shortly … … 75 68 first run of this chain (model start at t=0) is called 76 69 <b>initial run</b>. </p> 77 <p style="line-height: 100%;">Working with restart runs and their 78 generation through <b>mrun</b> requires certain entries in the 70 <p style="line-height: 100%;">Working with restart runs 71 and their 72 generation through <b>mrun</b> requires certain entries in 73 the 79 74 mrun-configuration file and in the parameter file, which are 80 75 described and explained in the following. The configuration file must 81 76 contain the following entries (example for the IBM Regatta of the 82 77 HLRN): </p> 83 <ul> 84 <pre style="line-height: 100%;"><font style="font-size: 10pt;" 85 size="2">%write_binary true restart</font><br><font 86 style="font-size: 10pt;" size="2">#</font><br><a 87 href="chapter_3.4.html#PARIN"><font style="font-size: 10pt;" size="2">PARIN</font></a><font 88 style="font-size: 10pt;" size="2"> in:job:npe d3# ~/palm/current_version/JOBS/$fname/INPUT _p3d</font><br><font 89 style="font-size: 10pt;" size="2">PARIN in:job:npe d3f ~/palm/current_version/JOBS/$fname/INPUT _p3df</font><br><a 90 href="chapter_3.4.html#BININ"><font style="font-size: 10pt;" size="2">BININ</font></a><font 91 style="font-size: 10pt;" size="2"> in:loc d3f ~/palm/current_version/JOBS/$fname/OUTPUT _d3d</font><br><font 92 style="font-size: 10pt;" size="2">#</font><br><a 93 href="chapter_3.4.html#BINOUT"><font style="font-size: 10pt;" size="2">BINOUT</font></a><font 94 style="font-size: 10pt;" size="2"> out:loc restart ~/palm/current_version/JOBS/$fname/OUTPUT _d3d</font></pre> 95 </ul> 96 <p style="line-height: 100%;">The <b>mrun</b> call for the 78 <ul> <pre style="line-height: 100%;"><font style="font-size: 10pt;" size="2">%write_binary true restart</font><br><font style="font-size: 10pt;" size="2">#</font><br><a href="chapter_3.4.html#PARIN"><font style="font-size: 10pt;" size="2">PARIN</font></a><font style="font-size: 10pt;" size="2"> in:job:npe d3# ~/palm/current_version/JOBS/$fname/INPUT _p3d</font><br><font style="font-size: 10pt;" size="2">PARIN in:job:npe d3f ~/palm/current_version/JOBS/$fname/INPUT _p3df</font><br><a href="chapter_3.4.html#BININ"><font style="font-size: 10pt;" size="2">BININ</font></a><font style="font-size: 10pt;" size="2"> in:loc d3f ~/palm/current_version/JOBS/$fname/OUTPUT _d3d</font><br><font style="font-size: 10pt;" size="2">#</font><br><a href="chapter_3.4.html#BINOUT"><font style="font-size: 10pt;" size="2">BINOUT</font></a><font style="font-size: 10pt;" size="2"> out:loc restart ~/palm/current_version/JOBS/$fname/OUTPUT _d3d</font></pre></ul> 79 <p style="line-height: 100%;">The <b>mrun</b> 80 call for the 97 81 initialization run of the job chain must look as follows: </p> 98 <ul> 99 <pre style="margin-bottom: 0.5cm; line-height: 100%;"><font 100 style="font-size: 10pt;" size="2">mrun -h ibmh -d abcde -t 900 -r "d3# restart"</font></pre> 101 </ul> 102 <p style="line-height: 100%;">The specification of the environment 103 variable <tt><tt><font style="font-size: 10pt;" size="2">writ</font></tt></tt><tt><tt><font 104 style="font-size: 10pt;" size="2">e_binary</font><font 105 style="font-size: 11pt;" size="2">, </font></tt></tt>which must be 82 <ul> <pre style="margin-bottom: 0.5cm; line-height: 100%;"><font style="font-size: 10pt;" size="2">mrun -h ibmh -d abcde -t 900 -r "d3# restart"</font></pre></ul> 83 <p style="line-height: 100%;">The specification of the 84 environment 85 variable <tt><tt><font style="font-size: 10pt;" size="2">writ</font></tt></tt><tt><tt><font style="font-size: 10pt;" size="2">e_binary</font><font style="font-size: 11pt;" size="2">, </font></tt></tt>which 86 must be 106 87 assigned the value <tt><tt><font style="font-size: 10pt;" size="2">true</font></tt></tt>, 107 88 is essential. Only in this case the model writes 108 binary-coded data for a possible restart run to the local file <tt><tt><a 109 href="chapter_3.4.html#BINOUT">BINOUT</a></tt></tt> 89 binary-coded data for a possible restart run to the local file <tt><tt><a href="chapter_3.4.html#BINOUT">BINOUT</a></tt></tt> 110 90 at the end of the run. Then of course this output file must be stored 111 91 on a permanent file with an appropriate file connection statement 112 92 (last line of the example above). As you can see, both instructions 113 93 (variable declaration and connection statements) are only carried out 114 by <b>mrun</b>, if the character string <tt><tt><font 115 style="font-size: 10pt;" size="2">restart</font></tt></tt> 94 by <b>mrun</b>, if the character string <tt><tt><font style="font-size: 10pt;" size="2">restart</font></tt></tt> 116 95 is given for the option <tt><font style="font-size: 10pt;" size="2">-r</font> 117 </tt>in the <span style="font-weight: bold;">mrun</span> call. Thus 96 </tt>in the <span style="font-weight: bold;">mrun</span> 97 call. Thus 118 98 the example above can also be used 119 99 if no restart runs are intended. In such cases the character string … … 122 102 can simply be omitted. </p> 123 103 <p style="line-height: 100%;">Only by the specification of 124 <tt><font style="font-size: 10pt;" size="2">write_binary=true</font><font 125 style="font-size: 11pt;" size="2">126 </font><font face="Thorndale, serif">the</font></tt>model is104 <tt><font style="font-size: 10pt;" size="2">write_binary=true</font><font style="font-size: 11pt;" size="2"> 105 </font><font face="Thorndale, serif">the</font></tt> 106 model is 127 107 instructed to compute the remaining CPU time after each time step and 128 108 stop, if the run is not going to be completed and finished briefly … … 130 110 this time. Actually the stop takes place when the 131 111 difference from the available job time (determined by the <b>mrun</b> 132 option <tt><font style="font-size: 10pt;" size="2">-t</font></tt>) and 112 option <tt><font style="font-size: 10pt;" size="2">-t</font></tt>) 113 and 133 114 the time used so far by the job becomes smaller than the time given 134 by the model variable <a 135 href="chapter_4.2.html#termination_time_needed">termination_time_needed</a>. 136 With the variable <b>termination_time_needed </b>the userdetermines,115 by the model variable <a href="chapter_4.2.html#termination_time_needed">termination_time_needed</a>. 116 With the variable <b>termination_time_needed </b>the user 117 determines, 137 118 how much time is needed for binary copying of the data for restart 138 119 runs, as 139 120 well as for the following data archiving and transfer of result data 140 121 etc. (as long as this is part of the job). Thus, as soon as the 141 remaining job time is less than <b>termination_time_needed</b>, the 122 remaining job time is less than <b>termination_time_needed</b>, 123 the 142 124 model stops 143 125 the time step procedure and copies the data for a restart run to the … … 148 130 the fact that a restart run must be started and leads to the 149 131 start of an appropriate job. </p> 150 <p style="line-height: 100%;"><font color="#000000">During the initial 151 phase of a restart ru</font>n different actions than during the initial 132 <p style="line-height: 100%;"><font color="#000000">During 133 the initial 134 phase of a restart ru</font>n different actions than during the 135 initial 152 136 phase of an initial run of the model are neccessary. In this 153 137 case the model must read in the binary data written by the preceding 154 138 run at the beginning of the run. Beyond that it also reads the 155 139 initialization parameters from this file. Therefore these do not need 156 to be indicated in the parameter file (local name <a 157 href="chapter_3.4.html#PARIN">PARIN</a>). 140 to be indicated in the parameter file (local name <a href="chapter_3.4.html#PARIN">PARIN</a>). 158 141 If they are indicated nevertheless and if their value deviates from 159 142 their value of the initial run, then this is ignored. There is 160 143 exactly one exception to this rule: with the help of the 161 initialization parameter <a 162 href="chapter_4.1.html#initializing_actions">initializing_actions</a> 144 initialization parameter <a href="chapter_4.1.html#initializing_actions">initializing_actions</a> 163 145 it is determined whether the job is a restart run or an 164 146 initial run. If <b>initializing_actions</b> = 165 <i>read_restart_data</i>, then it is a restart 147 “<i>read_restart_data”</i>, then it is 148 a restart 166 149 run, otherwise an initial run. The previous remarks make it 167 150 clear that the model obviously needs two different parameter files … … 173 156 parameters with values different from the initial run may appear in 174 157 this file, but they will be ignored), which 175 must have the value <i>read_restart_data</i>.158 must have the value “<i>read_restart_data”</i>. 176 159 Therefore the user must produce two different parameter files if he 177 160 wants to operate job chains. Since the model always expects the 178 parameter file on the local file <tt>PARIN</tt>, two different file 161 parameter file on the local file <tt>PARIN</tt>, two 162 different file 179 163 connection statements must be given for this file in the 180 164 configuration file. One may be active only at the initial run, 181 the other one only at restart runs. The <b>mrun </b>call for the 165 the other one only at restart runs. The <b>mrun </b>call 166 for the 182 167 initial run shown above activates the first of the two 183 specified connection statements, because the character string <tt><font 184 style="font-size: 10pt;" size="2">d3#</font></tt> 168 specified connection statements, because the character string <tt><font style="font-size: 10pt;" size="2">d3#</font></tt> 185 169 with the option <tt><font style="font-size: 10pt;" size="2">-r</font></tt> 186 170 coincides with the character 187 171 string in the third column of the connection statement. Obviously 188 172 the next statement must be active</p> 189 <ul> 190 <pre style="margin-bottom: 0.5cm; line-height: 100%;"><font 191 style="font-size: 10pt;" size="2">PARIN in:job:npe d3f ~/palm/current_version/JOBS/$fname/INPUT _p3df</font></pre> 192 </ul> 193 <p style="line-height: 100%;">with the restart runs. Given that t<font 194 color="#000000">his statement only gets</font> active if the option <tt><font 195 style="font-size: 10pt;" size="2">-r</font></tt> is given the value 196 <tt><font style="font-size: 11pt;" size="2">d3f</font></tt> and that 173 <ul> <pre style="margin-bottom: 0.5cm; line-height: 100%;"><font style="font-size: 10pt;" size="2">PARIN in:job:npe d3f ~/palm/current_version/JOBS/$fname/INPUT _p3df</font></pre></ul> 174 <p style="line-height: 100%;">with the restart runs. Given 175 that t<font color="#000000">his statement only gets</font> 176 active if the option <tt><font style="font-size: 10pt;" size="2">-r</font></tt> is given the value 177 <tt><font style="font-size: 11pt;" size="2">d3f</font></tt> 178 and that 197 179 the <b>mrun</b> call for this restart run is produced 198 automatically (thus not by the user), <b>mrun</b> obviously has to 180 automatically (thus not by the user), <b>mrun</b> 181 obviously has to 199 182 replace <tt><font style="font-size: 10pt;" size="2">"d3#"</font></tt> 200 183 of the initial run with <tt><tt><font style="font-size: 10pt;" size="2">"d3f"</font> 201 </tt></tt>within the call of this restart run. Actually, with restart 184 </tt></tt>within the call of this restart run. Actually, 185 with restart 202 186 runs all <tt><font style="font-size: 10pt;" size="2">"#"</font></tt> 203 characters within the strings given for the options <tt><font 204 style="font-size: 10pt;" size="2"><font face="Cumberland, monospace">-r</font></font></tt><font 205 style="font-size: 10pt;" size="2"><font face="Cumberland, monospace"> 206 , </font></font><tt><font style="font-size: 10pt;" size="2"><font 207 face="Cumberland, monospace">-i</font></font></tt> 208 and <tt><font style="font-size: 10pt;" size="2">-o</font></tt> are 209 replaced by <tt><font style="font-size: 10pt;" size="2">f</font></tt>. 187 characters within the strings given for the options <tt><font style="font-size: 10pt;" size="2"><font face="Cumberland, monospace">-r</font></font></tt><font style="font-size: 10pt;" size="2"><font face="Cumberland, monospace"> 188 , </font></font><tt><font style="font-size: 10pt;" size="2"><font face="Cumberland, monospace">-i</font></font></tt> 189 and <tt><font style="font-size: 10pt;" size="2">-o</font></tt> 190 are 191 replaced by <tt><font style="font-size: 10pt;" size="2">“f”</font></tt>. 210 192 </p> 211 <p style="line-height: 100%;">For example, for the initial run 193 <p style="line-height: 100%;">For example, for the initial 194 run 212 195 the permanent file </p> 213 <ul> 214 <pre style="margin-bottom: 0.5cm; line-height: 100%;">~/palm/current_version/JOBS/abcde/INPUT/abcde_p3d</pre> 215 </ul> 216 <p style="line-height: 100%;">and for restart runs the permanent file<span 217 style="font-family: monospace;"> </span></p> 218 <ul style="font-family: monospace;"> 219 <pre style="margin-bottom: 0.5cm; line-height: 100%;"><font 220 style="font-size: 10pt;" size="2">~/palm/current_version/JOBS/abcde/INPUT/abcde_p3df</font></pre> 221 </ul> 222 <p style="line-height: 100%;">is used. Only with restart runs the 223 local file <tt>BININ</tt> is made available as input file, because 196 <ul> <pre style="margin-bottom: 0.5cm; line-height: 100%;">~/palm/current_version/JOBS/abcde/INPUT/abcde_p3d</pre></ul> 197 <p style="line-height: 100%;">and for restart runs the 198 permanent file<span style="font-family: monospace;"> </span></p> 199 <ul style="font-family: monospace;"> <pre style="margin-bottom: 0.5cm; line-height: 100%;"><font style="font-size: 10pt;" size="2">~/palm/current_version/JOBS/abcde/INPUT/abcde_p3df</font></pre></ul> 200 <p style="line-height: 100%;">is used. Only with restart 201 runs the 202 local file <tt>BININ</tt> is made available as input file, 203 because 224 204 the appropriate file connection statement also contains the 225 205 character string <tt><font style="font-size: 10pt;" size="2">"d3f"</font></tt> … … 230 210 the corresponding output file (local name BINOUT) are identical and 231 211 read </p> 232 <ul> 233 <pre style="margin-bottom: 0.5cm; line-height: 100%;"><font 234 style="font-size: 10pt;" size="2">~/palm/current_version/JOBS/abcde/OUTPUT/abcde_d3d.</font></pre> 235 </ul> 236 <p style="line-height: 100%;">However, after the file produced by the 212 <ul> <pre style="margin-bottom: 0.5cm; line-height: 100%;"><font style="font-size: 10pt;" size="2">~/palm/current_version/JOBS/abcde/OUTPUT/abcde_d3d.</font></pre></ul> 213 <p style="line-height: 100%;">However, after the file 214 produced by the 237 215 previous job was read in by the model and after at the local file 238 216 <tt>BINOUT </tt>was produced at the end of the job, the 239 restart job does not overwrite this permanent file (<tt> 240 /<font 241 style="font-size: 10pt;" size="2">abcde_d3d</font></tt>) 217 restart job does not overwrite this permanent file (<tt>…/<font style="font-size: 10pt;" size="2">abcde_d3d</font></tt>) 242 218 with the new data. Instead of that, it is examined whether already 243 a permanent file with the name <tt><font style="font-size: 10pt;" 244 size="2"> 245 /abcde_d3d</font> 246 <font face="Thorndale, serif">exists </font></tt>when copying the 219 a permanent file with the name <tt><font style="font-size: 10pt;" size="2">…/abcde_d3d</font> 220 <font face="Thorndale, serif">exists </font></tt>when 221 copying the 247 222 output file (<tt><font style="font-size: 10pt;" size="2">BINOUT</font></tt>) 248 of <b>mrun</b>. If this is the case, <tt><font 249 style="font-size: 10pt;" size="2">BINOUT</font></tt> 250 is copied to the file<font style="font-size: 10pt;" size="2"><font 251 face="Cumberland, monospace"> 252 </font></font><tt><font style="font-size: 10pt;" size="2"><font 253 face="Cumberland, monospace"> 254 /abcde_d3d.1</font></font></tt><font 255 style="font-size: 10pt;" size="2"><font face="Cumberland, monospace">.</font></font> 256 Even if this file is already present, <tt><font 257 style="font-size: 10pt;" size="2"> 258 /abcde_d3d.2</font></tt> 223 of <b>mrun</b>. If this is the case, <tt><font style="font-size: 10pt;" size="2">BINOUT</font></tt> 224 is copied to the file<font style="font-size: 10pt;" size="2"><font face="Cumberland, monospace"> 225 </font></font><tt><font style="font-size: 10pt;" size="2"><font face="Cumberland, monospace">…/abcde_d3d.1</font></font></tt><font style="font-size: 10pt;" size="2"><font face="Cumberland, monospace">.</font></font> 226 Even if this file is already present, <tt><font style="font-size: 10pt;" size="2">…/abcde_d3d.2</font></tt> 259 227 is tried etc. For an input file the highest existing cycle 260 228 of the respective permanent file is copied. In the example above this 261 229 means: the initial run creates the permanent file 262 <tt><font style="font-size: 10pt;" size="2"> 263 /abcde_d3d</font></tt><font 264 style="font-size: 11pt;" size="2">,</font> 265 the first restart run uses this file and creates <tt> 266 /<font 267 style="font-size: 10pt;" size="2">abcde_d3d.1</font></tt>, 268 the second restart run creates <tt><font style="font-size: 10pt;" 269 size="2"> 270 /abcde_d3d.2</font></tt><font style="font-size: 10pt;" 271 size="2"> 230 <tt><font style="font-size: 10pt;" size="2">…/abcde_d3d</font></tt><font style="font-size: 11pt;" size="2">,</font> 231 the first restart run uses this file and creates <tt>…/<font style="font-size: 10pt;" size="2">abcde_d3d.1</font></tt>, 232 the second restart run creates <tt><font style="font-size: 10pt;" size="2">…/abcde_d3d.2</font></tt><font style="font-size: 10pt;" size="2"> 272 233 </font>etc. After completion of the job chain the user can still 273 234 access all files created by the jobs. This makes it possible for the 274 235 user for example to restart the model run of a certain job of the job 275 236 chain again. </p> 276 <p style="line-height: 100%;">Therefore restart jobs can not only be 277 started automatically through <b>mrun</b>, but also manually by the 237 <p style="line-height: 100%;">Therefore restart jobs can 238 not only be 239 started automatically through <b>mrun</b>, but also 240 manually by the 278 241 user. This is necessary e.g. whenever after the end of a job chain 279 242 it is decided that the simulation must be continued further, because 280 243 the phenomenon which should be examined did not reach the desired 281 state yet. In such cases the <b>mrun</b> options completely 282 correspond to those of the initial call; simply the <tt><font 283 style="font-size: 10pt;" size="2">"#"</font></tt> characters in the 284 arguments of options <tt><font style="font-size: 10pt;" size="2"><font 285 face="Cumberland, monospace">-r</font></font></tt><font 286 style="font-size: 10pt;" size="2"><font face="Cumberland, monospace">, 287 </font></font><tt><font style="font-size: 10pt;" size="2"><font 288 face="Cumberland, monospace">-i</font></font></tt> 289 and <tt><font style="font-size: 10pt;" size="2">-o</font></tt> must be 244 state yet. In such cases the <b>mrun</b> options 245 completely 246 correspond to those of the initial call; simply the <tt><font style="font-size: 10pt;" size="2">"#"</font></tt> 247 characters in the 248 arguments of options <tt><font style="font-size: 10pt;" size="2"><font face="Cumberland, monospace">-r</font></font></tt><font style="font-size: 10pt;" size="2"><font face="Cumberland, monospace">, 249 </font></font><tt><font style="font-size: 10pt;" size="2"><font face="Cumberland, monospace">-i</font></font></tt> 250 and <tt><font style="font-size: 10pt;" size="2">-o</font></tt> 251 must be 290 252 replaced by <tt><font style="font-size: 10pt;" size="2">"f"</font></tt>. 291 253 </p> 292 <hr> 293 <p style="line-height: 100%;"><br> 294 <font color="#000080"><font color="#000080"><a href="chapter_3.2.html"><font 295 color="#000080"><img src="left.gif" name="Grafik1" align="bottom" 296 border="2" height="32" width="32"></font></a><a href="index.html"><font 297 color="#000080"><img src="up.gif" name="Grafik2" align="bottom" 298 border="2" height="32" width="32"></font></a><a href="chapter_3.4.html"><font 299 color="#000080"><img src="right.gif" name="Grafik3" align="bottom" 300 border="2" height="32" width="32"></font></a></font></font></p> 301 <p style="line-height: 100%;"><i>Last change: </i> 14/04/05 (SR)</p> 302 </body> 303 </html> 254 <hr><p style="line-height: 100%;"><br> 255 <font color="#000080"><font color="#000080"><a href="chapter_3.2.html"><font color="#000080"><img src="left.gif" name="Grafik1" align="bottom" border="2" height="32" width="32"></font></a><a href="index.html"><font color="#000080"><img src="up.gif" name="Grafik2" align="bottom" border="2" height="32" width="32"></font></a><a href="chapter_3.4.html"><font color="#000080"><img src="right.gif" name="Grafik3" align="bottom" border="2" height="32" width="32"></font></a></font></font></p><p style="line-height: 100%;"><i>Last change: </i> 256 $Id$</p> 257 </body></html>
Note: See TracChangeset
for help on using the changeset viewer.