This document contains local support information for PALM users on the Linux Cluster Tsubame of the Tokyo Institute of Technology, Tokyo, Japan (http://www.gsic.titech.ac.jp/~ccwww). Japanese and English user manuals are available for download (http://www.gsic.titech.ac.jp/~ccwww/tebiki).
Installation:
The
following lines should be added to your ~/.profile (for ksh users - create if necessary) or ~/.bashrc (for bash users). These settings will be active from the next login.
export
LD_LIBRARY_PATH=~mkanda/svn/lib
export PATH=~mkanda/svn/bin:$PATHUsage:
Since
Tsubame is located behind a firewall, the usual address svn://130.75.105.45 is unreachable from Tsubame. Instead, please login to media-o
and use svn://p9c.cc.titech.ac.jp:36900.
This port works in both directions, i.e. for svn update as well as svn commit commands.
Please add the following line to the output command (OC:) section of your .mrun.config to ensure correct output of 2D and 3D data for PALM runs on more than one CPU :
OC:[[ \$( echo \$localhost | cut -c1-3 ) = lct ]] && combine_plot_fields.xThe implementation of the n1ge command on
Tsubame requires modifications to mrun and subjob for PALM to
run on Tsubame. Since mrun
must be executed on one PE only, a one-PE "submitting job" is created
when the user calls mrun.
At the time when this "submitting job" is actually executed, it starts
the main PALM job in the foreground. However, this only works if
enough resources are available at that time. Therefore PALM users on
Tsubame are advised to consider the queue load of the available queues
for the main job (qstat -g c) before job submission.
According to the current queue configuration (http://www.gsic.titech.ac.jp/~ccwww/tgc/q_e.html), the following mrun options are available:
| mrun option | explanation | possible values | default value |
-q | queue name for main job | novice, sla1, sla2, sla3, RAM64GB, RAM128GB | sla3 |
-n | queue name for "submitting job" (always runs on 1 PE only) | novice, sla1, sla2, sla3 | sla3 |
-g | group number | name of the N1GE_GROUP that is required to utilize the bes or sla queues | none |
Note that the default value "sla3" of the
mrun option -q may
not always be the best choice. For code development, debugging or short
test runs the value "novice" for both options may be suitable.
Production runs should use the respective production queues (see http://www.gsic.titech.ac.jp/~ccwww/tgc/q_e.html for details).
The
computer center advised us that due to restrictions of the batch system
the value "novice" should not be mixed with the other queues. Please
specify either a combination of slan for -n and slan or RAM* for -q, or specify the combination "novice" + "novice".
The group number contains the accounting information. You must specify it using the mrun option -g. Alternatively, you may set also a default value in your .mrun.config file like this:
%group_number
12345678
lctit parallel
The mrun option -t
contains the CPU time demand for the main job in s which will be
automatically converted into min for the n1ge command. The
option -m
contains the memory demand for the main job in MB which will be
automatically converted into GB for the n1ge command.
Finally, PALM batch jobs on Tsubame require the mrun option -b.
In order to use NetCDF commands like ncdump and the graphical NetCDF tools NCL and Ncview, please add the following lines to ~/.profile (for ksh users) or ~/.bashrc (for bash users):
# NetCDF
export PATH=$PATH:~mkanda/netcdf-3.6.2/bin:~mkanda/netcdf-3.6.2/include# NCL
export NCARG_ROOT=~mkanda/ncl
PATH=~mkanda/ncl/bin:$PATH
# Ncview
export PATH=$PATH:~mkanda/ncview/bin
export XAPPLRESDIR=~mkanda/.app-defaults