Home


About us
PALM
PALM-4U
Installation
Release notes
Documentation
Getting Help
Gallery
Developers area

Changeset 1090


Ignore:
Timestamp:
Feb 2, 2013 7:06:13 AM (12 years ago)
Author:
raasch
Message:

script adjustments for Kyushu-University computing center (lckyut)

Location:
palm/trunk/SCRIPTS
Files:
4 edited

Legend:

Unmodified
Added
Removed

  • palm/trunk/SCRIPTS/batch_scp

    r693 r1090  
    11#! /bin/ksh
    2 # batch_scp - Shellskript         Version:  @(#)batch_scp 1.0a     25/10/05
     2
     3# batch_scp - script for automatic file/directory transfer using scp
     4
     5#--------------------------------------------------------------------------------#
     6# This file is part of PALM.
     7#
     8# PALM is free software: you can redistribute it and/or modify it under the terms
     9# of the GNU General Public License as published by the Free Software Foundation,
     10# either version 3 of the License, or (at your option) any later version.
     11#
     12# PALM is distributed in the hope that it will be useful, but WITHOUT ANY
     13# WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
     14# A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
     15#
     16# You should have received a copy of the GNU General Public License along with
     17# PALM. If not, see <http://www.gnu.org/licenses/>.
     18#
     19# Copyright 1997-2012  Leibniz University Hannover
     20#--------------------------------------------------------------------------------#
     21#
     22# Current revisions:
     23# ------------------
     24# code put under GPL (PALM 3.9)
     25# adjustments for Kyushu-University computing center (lckyut)
     26# old changelog messages removed
     27#
     28# Former revisions:
     29# -----------------
    330# $Id$
    4 
    5      # Prozedur zum automatischen Transfer von Dateien mittels scp
    6      #
    7      # batch_scp hat 4 Argumente:
    8      #         $1 = IP-Adresse des Zielrechners
    9      #         $2 = zu uebertragende Datei
    10      #         $3 = Verzeichnis, in das kopiert werden soll
    11      #         $4 = Dateiname der Zieldatei
    12 
    13 
    14      #                 ACHTUNG!!!!!!!!
    15      #
    16      # batch_scp hat noch folgenden grossen Mangel: wenn die Verzeichnisse
    17      # auf den jeweiligen Remote-Rechnern aeltere Dateien beinhalten,
    18      # die beim "ls -al" eine Jahreszahl und keine Uhrzeit (aa:bb) liefern,
    19      # dann kann batch_scp die Dateinamen nicht ueberpruefen und findet
    20      # z.B. bei -g die entsprechende Datei nicht!
    21      #
    22      #
    23 
    24 
    25      # letzte Aenderung:
    26      # 29/11/01 - Siggi - Entwicklungsbeginn
    27      # 04/01/02 - Siggi - Version 1.0 funktionsfaehig
    28      # 15/02/02 - Siggi - Verzeichnis-Listing umgestellt von "ls -al" auf "ls -1"
    29      # 30/05/02 - Siggi - Abbruch mit exit 1 bei scp- oder ssh-Fehler
    30      # 12/06/02 - Siggi - Version 1.0a, parent directories are also created, if
    31      #                    directory on remote host does not exist
    32      # 18/09/02 - Siggi - Fehlerabfragen korrigiert (waren wegen Klammerung
    33      #                    unwirksam)
    34      # 12/03/03 - Siggi - errfile and filelist are not stored in /tmp any more
    35      #                    errors in execution of ssh does not lead
    36      #                    to an abort on the NEC-system at DKRZ
    37      # 01/04/03 - Siggi - small error concerning creation of catalogs removed
    38      # 23/01/04 - Siggi - additional test output for scp on hurrikan
    39      # 02/12/04 - Siggi - additional check of file size on remote host after scp.
    40      #                    If the file sizes on local and remote host are equal,
    41      #                    the scp exit status is ignored
    42      # 03/12/04 - Siggi - additional checks of ssh actions independent of the
    43      #                    ssh exit status
    44      # 11/03/05 - Siggi - arguments are output in case of error exit
    45      # 25/10/05 - Siggi - put of catalogs realized
    46      # 27/04/09 - Marcus- use option -p for all scp calls
    47      # 08/03/11 - Siggi - adjustments for ibmkisti: this machine allows
    48      #                    outgoing ssh/scp connections only from the
    49      #                    interactive nodes (gaiad). All ssh/scp traffic is
    50      #                    done via this interactive node.
     31#
     32# 08/03/11 - Siggi - adjustments for ibmkisti: this machine allows
     33#                    outgoing ssh/scp connections only from the
     34#                    interactive nodes (gaiad). All ssh/scp traffic is
     35#                    done via this interactive node.
     36# 04/01/02 - Siggi - first version finished
     37# 29/11/01 - Siggi - script development started
     38#
     39#--------------------------------------------------------------------------------#
     40# batch_scp - script for automatic file/directory transfers using scp
     41#
     42# batch_scp has 4 arguments:
     43#               $1 = IP-addres of remote (target) machine
     44#               $2 = file to be transferred
     45#               $3 = directory of remote machine, where file should be copied to
     46#               $4 = filename that file should be given on remote machine
     47#
     48# ATTENTION: problems might occur if directories on remote machine include very
     49#            old files for which "ls -al" does give "year" as modification
     50#            time instead of "hh:mm". In such a case, batch_scp cannot check the
     51#            filename and may not find the file (e.g. if option -g is used).
     52#--------------------------------------------------------------------------------#
    5153
    5254
    … …  
    6062 check=false
    6163 delete=false
    62  errfile=ftpcopy.errfile.$random
     64 errfile=batch_scp.errfile.$random
    6365 filelist=filelist.$random
    6466 get=false
    … …  
    8183 trap 'if [[ $locat != normal ]]
    8284       then
    83           cat $filelist $errfile
     85          [[ -f "$filelist" ]]  &&  cat $filelist
     86          [[ -f "$errfile"  ]]  &&  cat $errfile
    8487          rm -rf $filelist $errfile
    8588          printf " +++ BATCH_SCP terminated \n"
    … …  
    296299    ssh $local_user@gaiad "ssh $1 -l $remote_user \"cd $3; ls -1; echo '*** list complete'\" "  > $filelist  2>&1
    297300 else
    298     ssh $1 -l $remote_user "cd $3; ls -1; echo '*** list complete'" > $filelist  2>&1
     301    ssh -p 22 $1 -l $remote_user "cd $3; ls -1; echo '*** list complete'" > $filelist  2>&1
    299302 fi
    300303 ssh_status=$?
    … …  
    506509       ssh $local_user@gaiad "ssh $1 -l $remote_user \"cd $3\" " > $filelist
    507510    else
    508        ssh $1 -l $remote_user "cd $3" > $filelist
     511       ssh $1 -p 22 -l $remote_user "cd $3" > $filelist
    509512    fi
    510513    if [[ $? != 0 ]]
    … …  
    569572             if [[ $catalog_copy = false ]]
    570573             then
    571                 scp -p $2 $remote_user@$1:$catalog_name$4$zyklusnr  > /dev/null
     574                scp -P 22 -p $2 $remote_user@$1:$catalog_name$4$zyklusnr  > /dev/null
    572575             else
    573                 scp -p -r $2 $remote_user@$1:$catalog_name$4$zyklusnr  > /dev/null
     576                scp -P 22 -p -r $2 $remote_user@$1:$catalog_name$4$zyklusnr  > /dev/null
    574577             fi
    575578          fi
    … …  
    587590                remote_size=`ssh $local_user@gaiad "ssh $1 -l $remote_user \"ls -al $catalog_name$4$zyklusnr\" "`
    588591             else
    589                 remote_size=`ssh $1 -l $remote_user "ls -al $catalog_name$4$zyklusnr"`
     592                remote_size=`ssh $1 -p 22 -l $remote_user "ls -al $catalog_name$4$zyklusnr"`
    590593             fi
    591594             remote_size=`echo $remote_size | cut -d" " -f5`
    … …  
    604607             ssh $local_user@gaiad "cd $local_wdir; scp -p $2 $remote_user@$1:${catalog_name}batch_scp_append_file.$random"  > /dev/null
    605608          else
    606              scp -p $2 $remote_user@$1:${catalog_name}batch_scp_append_file.$random  > /dev/null
     609             scp -P 22 -p $2 $remote_user@$1:${catalog_name}batch_scp_append_file.$random  > /dev/null
    607610          fi
    608611          if [[ $? != 0 ]]
    … …  
    617620                remote_size=`ssh $local_user@gaiad "ssh $1 -l $remote_user \"ls -al ${catalog_name}batch_scp_append_file.$random\" "`
    618621             else
    619                 remote_size=`ssh $1 -l $remote_user "ls -al ${catalog_name}batch_scp_append_file.$random"`
     622                remote_size=`ssh $1 -p 22 -l $remote_user "ls -al ${catalog_name}batch_scp_append_file.$random"`
    620623             fi
    621624             remote_size=`echo $remote_size | cut -d" " -f5`
    … …  
    636639             ssh $local_user@gaiad "ssh $1 -l $remote_user \"cd $3; cat batch_scp_append_file.$random >> $4$zyklusnr; rm batch_scp_append_file.$random; echo '*** append complete'\" " > $filelist
    637640          else
    638              ssh $1 -l $remote_user "cd $3; cat batch_scp_append_file.$random >> $4$zyklusnr; rm batch_scp_append_file.$random; echo '*** append complete'" > $filelist
     641             ssh $1 -p 22 -l $remote_user "cd $3; cat batch_scp_append_file.$random >> $4$zyklusnr; rm batch_scp_append_file.$random; echo '*** append complete'" > $filelist
    639642          fi
    640643          if [[ $? != 0 ]]
    … …  
    656659          ssh $local_user@gaiad "ssh $1 -l $remote_user \"mkdir -p $3\" "
    657660       else
    658           ssh $1 -l $remote_user "mkdir -p $3"
     661          ssh $1 -p 22 -l $remote_user "mkdir -p $3"
    659662       fi
    660663       if [[ $? != 0 ]]
    … …  
    666669          ssh $local_user@gaiad "cd $local_wdir; scp -p $2 $remote_user@$1:$catalog_name$4$zyklusnr"  > /dev/null
    667670       else
    668           scp -p $2 $remote_user@$1:$catalog_name$4$zyklusnr  > /dev/null
     671          scp -P 22 -p $2 $remote_user@$1:$catalog_name$4$zyklusnr  > /dev/null
    669672       fi
    670673       if [[ $? != 0 ]]
    … …  
    690693             ssh $local_user@gaiad "cd $local_wdir; scp -p $remote_user@$1:$catalog_name$4$zyklusnr $2"  > /dev/null
    691694          else
    692              scp -p $remote_user@$1:$catalog_name$4$zyklusnr $2  > /dev/null
     695             scp -P 22 -p $remote_user@$1:$catalog_name$4$zyklusnr $2  > /dev/null
    693696          fi
    694697          if [[ $? != 0 ]]

  • palm/trunk/SCRIPTS/mbuild

    r1084 r1090  
    11#!/bin/ksh
     2
     3# mbuild - script for compiling the PALM code and its utility programs
    24
    35#--------------------------------------------------------------------------------#
    … …  
    1921#
    2022# Current revisions:
    21 # -----------------
     23# ------------------
     24# adjustments for Kyushu-University computing center (lckyut)
     25# old changelog messages removed
    2226#
    2327# Former revisions:
    … …  
    3842# code put under GPL (PALM 3.9)
    3943#
     44# 12/06/02 - Siggi - first version finished
     45# 06/05/02 - Siggi - script development started
     46#
     47#--------------------------------------------------------------------------------#
    4048# mbuild - script for compiling the PALM code and its utility programs
    41 
    42      # Procedure to compile code on local and remote hosts using the
    43      # make-mechanism. The source code must be provided on the local host.
    44 
    45      # Last changes:
    46      # 06/05/02 - Siggi - start development
    47      # 12/06/02 - Siggi - first version finished
    48      # 12/09/02 - Siggi - ibmh (hanni.hlrn.de) included
    49      # 19/12/02 - Siggi - ibmb included
    50      # 05/02/03 - Siggi - hostname nobela included
    51      # 04/03/03 - Siggi - host nech included
    52      # 06/03/03 - Siggi - make_call_string is echoed without '' to
    53      #                    file LAST_MAKE_CALL (otherwise error on NEC, because
    54      #                    '' are part of compiler options
    55      # 16/03/03 - Siggi - Two underscores are placed in front of every define
    56      #                    string, in case that palm.f90 version contains
    57      #                    such strings
    58      # 16/04/03 - Siggi - First extensions for linux machines
    59      # 24/06/03 - Siggi - host orkan included
    60      # 17/07/03 - Siggi - IP adress set to new "cross" machine at DKRZ
    61      # 24/07/03 - Siggi - host maestro admitted
    62      # 06/08/03 - Siggi - host gregale admitted
    63      # 05/11/03 - Siggi - hosts irifi and quanero are now belonging to lcmuk
    64      # 19/11/03 - Heiko - on lcmuk, mbuild does not tar the *.i files
    65      # 08/01/04 - Siggi - additional preprocessor directive for ibm included
    66      #                    (-D$OMP=OMP) in order to avoid problems with
    67      #                    OMP_NUM_THREADS
    68      # 09/01/04 - Siggi - action above cancelled
    69      # 28/01/04 - Siggi - action above re-cancelled
    70      # 08/03/04 - Siggi - host scirocco admitted
    71      # 26/03/04 - Siggi - .o and .mod files are also deleted in depository, if
    72      #                    the respective .f90 file is not listed in the makefile
    73      #                    and deletion is demanded by the user
    74      # 12/04/04 - Siggi - scp2 instead of scp used for transfer from decalpha
    75      #                    due to error in ssh installation (otherwise a prompt
    76      #                    for the password appears)
    77      # 23/07/04 - Siggi - changes due to the new berni configuration
    78      #                    (federation switch)
    79      # 08/09/04 - Siggi - hanni IP address changed to 130.75.4.10
    80      # 23/09/04 - Joerg - correction of IP-Address for 'cross' (DKRZ)
    81      # 09/03/05 - Siggi - on nech, mbuild does not tar the *.i files
    82      # 31/03/05 - Siggi - mbuild does not tar *.i files any more
    83      # 24/04/05 - Siggi - netcdf support on lcmuk
    84      # 25/04/05 - Siggi - netcdf support on gfdl3 (decalpha)
    85      # 12/05/05 - Siggi - netcdf support on ibm
    86      #                    set OBJECT_MODE=64 for compiling on ibmb, ibmh
    87      # 18/05/05 - Siggi - netcdf support on nec
    88      # 19/05/05 - Siggi - IP addres 134.75.155.74 changed to 165.132.26.56
    89      # 23/05/05 - Siggi - netcdf support on ibms
    90      # 01/06/05 - Siggi - reset of cpp_options to "" on lcmuk
    91      # 30/06/05 - Siggi - netcdf support on bora
    92      # 20/10/05 - Siggi - update of netcdf-version on decalpha (gfdl3)
    93      # 04/11/05 - Siggi - netcdf 3.6.0-p1 on ibmh/ibmb
    94      # 30/12/05 - Siggi - change of IP adresses in subnet 130.75.105
    95      #                    host gfdl5 (ibmy) admitted
    96      # 10/01/06 - Siggi - cpp directive for NetCDF 64bit support
    97      # 20/01/06 - Siggi - cpp directive for ibmy
    98      # 09/02/06 - Marcus- compile only once on lcmuk (as on ibmh/ibmb)
    99      # 10/02/06 - Siggi - modifications for scp on decalpha
    100      # 13/04/06 - Siggi - ostria admitted
    101      # 19/04/06 - Siggi - preprocessor directive -D$OMP=OMP for ibm removed
    102      # 23/05/05 - Siggi - lctit (SUN Fire X4600) admitted
    103      # 29/05/05 - Siggi - atmos (lcide) admitted
    104      # 23/08/06 - Siggi - netcdf support for scirocco (notebook)
    105      # 24/11/06 - Siggi - breva and levanto admitted
    106      # 07/02/07 - Siggi - adapted for RIAM (neck)
    107      # 10/02/07 - Siggi - all hpmuk-related code removed
    108      # 02/03/07 - Siggi - compilation of utility programs and transfer of
    109      #                    scripts to remote hosts added (option -u)
    110      # 14/03/07 - Siggi - fimm admitted, revision number added to terminal
    111      #                    output
    112      # 16/03/07 - Siggi - adjustments (netcdf) for lctit
    113      #                    adjustments for running under pdksh, local ip-addres
    114      #                    is not determined any more
    115      # 30/03/07 - Siggi - cpp-directives/options + netcdf-options are read
    116      #                    from configuration file
    117      #                    host identifier (local_host) is read from config file
    118      # 10/10/07 - Siggi - bugfix: handling of comment lines
    119      # 18/07/08 - Siggi - adapted for lcsgih
    120      # 21/07/08 - Siggi - mainprog (executable) is added to the tar-file
    121      #                    ({mainprog}_current_version)
    122      # 02/10/08 - Siggi - adapted for lcxt4
    123      # 14/11/08 - Siggi - update mechanism completely revised: source tarfile
    124      #                    remains in source directory, one depository per block
    125      #                    (given in the config-file) is created, allways all
    126      #                    files from the source directory are copied to the
    127      #                    respective depository, no additional file checks are
    128      #                    done any more (version 2.1)
    129      # 13/01/09 - Marcus- re-enable compilation more than once on lcmuk (as on
    130      #                    other machines)
    131      # 21/03/09 - Siggi - -u copies also copies process_dvr_output and
    132      #                    .dvrserver.config
    133      # 21/04/09 - Siggi - adjustments for new IBM at DKRZ, which is now ibmh
    134      # 25/08/09 - BjornM- adapted for lck
    135      # 01/12/09 - BjornM- re-adjustments for lcxt4
    136      # 01/02/10 - Siggi - adjustments for lcxt5m
    137      # 03/02/10 - Siggi - make options (mopts) to be set by configuration file
    138      #                    implemented, remoted mode option -r completely
    139      #                    removed, t3e related parts removed
    140      # 26/02/10 - BjornM- re-adjustments for lcxt4 (new modules)
    141      # 01/03/10 - Siggi - mpilib replaced by modules: explicit module settings
    142      #                    removed; modules should now be given in the
    143      #                    configuration file
    144      # 02/03/10 - Siggi - utility programs are created separately for each
    145      #                    block in the configuration file,
    146      #                    parallel make for utility programs
    147      # 12/03/10 - Siggi - bugfix: remode_md was given a wrong value
    148      # 16/08/10 - BjornM- bugfix: call of modules on local host, new workaround
    149      #                    for lcxt4
    150      # 07/09/10 - Siggi - bugfix for wrong netcdf/3.6.3 module on lcsgi
    151      # 08/12/10 - Siggi - initialization of the module command changed for
    152      #                    SGI-ICE/lcsgi
    153      #                    adjustments for Kyushu Univ. (lcrte, ibmku)
    154      # 14/12/10 - Siggi - adjustments for new Tsubame system at Tokyo
    155      #                    institute of technology (lctit)
    156      # 04/03/11 - BjornM- re-adjustments for lcxt4
    157      # 06/03/11 - Siggi - adjustments for ibmkisti
    158      # 03/04/11 - Micha - added lckordi
    159      # 17/08/11 - Siggi - HLRN access via hicegate0 (130.75.4.101)
    160      # 27/11/11 - Siggi - Bugfix for make options setting (-j)
    161      # 12/12/11 - BjornM- added mrun GUI to ulitity programs
    162      # 25/01/12 - BjornM- compilation of check_namelist_files (namelist file
    163      #                    check program) has been added to the utility branch),
    164      #                    modfified the compilation of mrungui
    165      # 31/01/12 - BjornM- bugfix for compilation of utility programs on
    166      #                    multiple localhost branches
    167      # 08/03/12 - BjornM- bugfix: cpp directive for namelist check program
    168      #                    corrected
    169      # 02/05/12 - Bjoern- lcxt4 is now lcxe6, adjustments for lcxe6,
    170      #                    bugfixes for the parameter file check, initial
    171      #                    commands on ssh calls sourced out to .mrun.config
    172      # 05/06/12 - Bjoern- bugfixes for execution of login_init_cmd
    173      # 18/06/12 - Siggi - option -v (silent mode) added
    174      # 25/06/12 - Siggi - option -K (block conditions) added
    175      # 27/09/12 - Siggi - compiler options (F90FLAGS) added for namelist check
    176      # 02/10/12 - Siggi - added lcsb
    177      # 02/11/12 - Siggi - added lckiaps
     49#
     50# Procedure to compile code on local and remote hosts using the
     51# make-mechanism. The source code must be provided on the local host.
     52#--------------------------------------------------------------------------------#
    17853
    17954
    … …  
    581456        (lck)            remote_addres=165.132.26.61;;
    582457        (lckiaps)        remote_addres=118.128.66.223;;
     458        (lckyut)         remote_addres=133.5.4.37;;
    583459        (lctit)          remote_addres=10.1.6.170;;
    584460        (lcxe6)          remote_addres=129.177.20.113;;

  • palm/trunk/SCRIPTS/mrun

    r1084 r1090  
    2121#
    2222# Current revisions:
    23 # -----------------
     23# ------------------
     24# adjustments for Kyushu-University computing center (lckyut)
    2425#
    2526# Former revisions:
    … …  
    4041# code put under GPL (PALM 3.9)
    4142#
    42 # 21/03/94 - Siggi - first version (1.0)
    43 # 03/03/94 - Siggi - start with developing the script
     43# 21/03/94 - Siggi - first version finished
     44# 03/03/94 - Siggi - script development started
     45#
     46#--------------------------------------------------------------------------------#
     47# mrun - script for running PALM jobs
    4448#--------------------------------------------------------------------------------#
    4549
    … …  
    502506    do_remote=true
    503507    case  $host  in
    504         (ibm|ibmh|ibmkisti|ibmku|ibms|ibmy|nech|necriam|lckyoto|lcsgib|lcsgih|unics|lcxe6|lcxt5m|lck|lckiaps|lckordi|lcsb)  true;;
     508        (ibm|ibmh|ibmkisti|ibmku|ibms|ibmy|nech|necriam|lckyoto|lcsgib|lcsgih|unics|lcxe6|lcxt5m|lck|lckiaps|lckordi|lckyut|lcsb)  true;;
    505509        (*)  printf "\n"
    506510             printf "\n  +++ sorry: execution of batch jobs on remote host \"$host\""
    … …  
    666670                   do_remote=true
    667671                   case  $host  in
    668                        (ibm|ibmh|ibmkisti|ibmku|ibms|ibmy|lckyoto|lcsgib|lcsgih|nech|necriam|unics|lcxe6|lcxt5m|lck|lckiaps|lckordi|lcsb)  true;;
     672                       (ibm|ibmh|ibmkisti|ibmku|ibms|ibmy|lckyoto|lcsgib|lcsgih|nech|necriam|unics|lcxe6|lcxt5m|lck|lckiaps|lckordi|lckyut|lcsb)  true;;
    669673                       (*)  printf "\n  +++ sorry: execution of batch jobs on remote host \"$host\""
    670674                            printf "\n      is not available"
    … …  
    950954    do_remote=true
    951955    case  $host  in
    952         (ibm|ibmh|ibmkisti|ibmku|ibms|ibmy|lckyoto|lcsgib|lcsgih|nech|necriam|unics|lcxe6|lcxt5m|lck|lckiaps|lckordi|lcsb)  true;;
     956        (ibm|ibmh|ibmkisti|ibmku|ibms|ibmy|lckyoto|lcsgib|lcsgih|nech|necriam|unics|lcxe6|lcxt5m|lck|lckiaps|lckordi|lckyut|lcsb)  true;;
    953957        (*)  printf "\n"
    954958             printf "\n  +++ sorry: execution of batch jobs on remote host \"$host\""
    … …  
    11021106        (lckiaps)    queue=normal;;
    11031107        (lckyoto)    queue=ph;;
     1108        (lckyut)     queue=cx-dbg;;
    11041109        (lcsgib)     queue=testq;;
    11051110        (lcsgih)     queue=testq;;
    … …  
    18491854 lopts="$lopts $netcdf_lib $dvr_lib"
    18501855 ROPTS="$ropts"
    1851  if [[ ( $(echo $host | cut -c1-3) = nec  ||  $(echo $host | cut -c1-3) = ibm  ||  $host = lckyoto  ||  $host = lcsgih  ||  $host = lcsgib  ||  $host = lctit  ||  $host = lcfimm  ||  $host = lcxe6 ||  $host = lcxt5m || $host = lck || $host = lckiaps || $host = lckordi || $host = lcsb )  &&  -n $numprocs ]]
    1852  then
     1856# if [[ ( $(echo $host | cut -c1-3) = nec  ||  $(echo $host | cut -c1-3) = ibm  ||  $host = lckyoto  ||  $host = lcsgih  ||  $host = lcsgib  ||  $host = lctit  ||  $host = lcfimm  ||  $host = lcxe6 ||  $host = lcxt5m || $host = lck || $host = lckiaps || $host = lckordi || $host = lcsb || $host )  &&  -n $numprocs ]]
     1857# then
    18531858    XOPT="-X $numprocs"
    1854  fi
     1859# fi
    18551860
    18561861
    … …  
    36103615                then
    36113616                   mpirun -np $ii  -f $PBS_NODEFILE  a.out  <  runfile_atmos  $ROPTS
     3617                elif [[ $host = lckyu* ]]
     3618                then
     3619                   mpiexec -n $ii   a.out  <  runfile_atmos  $ROPTS
    36123620                else
    36133621                   mpiexec  -machinefile hostfile  -n $ii  a.out  < runfile_atmos  $ROPTS

  • palm/trunk/SCRIPTS/subjob

    r1047 r1090  
    11#!/bin/ksh
     2
     3# subjob - script for automatic generation and submission of batch-job files
     4#          for various batch queuing systems
     5
    26#--------------------------------------------------------------------------------#
    37# This file is part of PALM.
    … …  
    1822#
    1923# Current revisions:
    20 # -----------------
     24# ------------------
     25# adjustments for Kyushu-University computing center (lckyut)
     26# old changelog messages removed
    2127#
    2228# Former revisions:
    … …  
    2733# code put under GPL (PALM 3.9)
    2834#
    29 # subjob - Plot-Shellskript         Version:  @(#)SUBJOB 1.0
    30      # Prozedur zum automatischen Generieren von Batch-Jobs, die unter NQS
    31      # laufen sollen und deren Ergebnis (Dayfile) zum Job-generierenden
    32      # Host zurueckgeschickt werden sollen
    33 
    34 
    35      # letzte Aenderung:
    36      # 29/06/94 - Siggi - Beginn mit Erstellung der Originalversion
    37      # 08/07/94 - Siggi - Originalversion abgeschlossen (Version 1.0)
    38      # 06/02/98 - Siggi - berte validiert
    39      # 27/01/01 - Siggi - ground.yonsei.ac.kr validiert, Jobs zur T3E in Korea
    40      #                    moeglich
    41      # 08/02/01 - Siggi - alle subjob-Meldungen ins englische uebersetzt
    42      # 25/05/02 - Siggi - Unterstuetzung des LoadLeveler
    43      # 30/05/02 - Siggi - Validierung fuer ibm-Rechner in Seoul (nobel) sowie
    44      #                    allgemeine Anpassungen fuer ibm-Rechner
    45      # 15/10/02 - Siggi - Neue Default-Jobklasse (p_normal) fuer IBM in Seoul
    46      #                    Ruecktransfer des Jobprotokolls fuer diese
    47      #                    Maschine abgeschaltet
    48      # 31/10/02 - Siggi - berni validiert
    49      # 06/11/02 - Siggi - Neue Jobklassen auf ibmb und ibmh
    50      # 08/11/02 - Siggi - quanero validiert
    51      # 11/12/02 - Siggi - Notification fuer Transfer-Jobs abgeschaltet
    52      # 23/01/03 - Siggi - hostname nobel changed to nobela
    53      # 06/02/03 - Siggi - gregale validated
    54      # 12/02/03 - Siggi - orkan and maestro validated
    55      # 21/02/03 - Siggi - all nobel nodes in Seoul validated
    56      # 12/03/03 - Siggi - nec at DKRZ validated
    57      # 13/03/03 - Siggi - new nqs resource variable Memory
    58      # 07/04/03 - Siggi - processor request option -c on nech needs tasks per
    59      #                    node
    60      # 11/04/03 - Siggi - network on ibms has attribute "shared"
    61      # 31/07/03 - Siggi - nqs2 on nech implemented (provisional: -h nech2)
    62      #                    cxxl added to ibmh
    63      # 29/08/03 - Siggi - changes in job queues and communication system on
    64      #                    ibms
    65      # 24/10/03 - Siggi - using alternate hanni address 130.75.4.2
    66      # 30/10/03 - Siggi - nech is not supported any more
    67      # 10/11/03 - Siggi - nech2 renamed to nech
    68      # 20/11/03 - Siggi - submit command on nech changed from qsub.test to qsub
    69      # 29/03/04 - Siggi - ground not supported any more, gfdl3 validated
    70      # 31/03/04 - Siggi - new option -N for node usage
    71      # 12/04/04 - Siggi - scp2 instead of scp used for transfer from decalpha
    72      #                    due to error in ssh installation (otherwise a prompt
    73      #                    for the password appears)
    74      # 23/07/04 - Siggi - changes due to the new berni configuration
    75      #                    (federation switch)
    76      # 01/09/04 - Gerald  new job-classes on hanni
    77      # 08/09/04 - Siggi - hanni IP address changed to 130.75.4.10
    78      # 23/11/04 - Siggi - new job class cdata on hanni and berni
    79      # 03/12/04 - Siggi - notification on ibm switched of in case of
    80      #                    delete_dayfile = true, node usage in cdev set to
    81      #                    shared
    82      # 16/02/05 - Gerald  hababai validated
    83      # 29/03/05 - Micha - new job class channi on hanni
    84      # 11/05/05 - Siggi - ConsumableMemory is now required as resource keyword
    85      #                    on ibms
    86      # 24/05/05 - Siggi - Default queue on ibms changed from p_normal_1.3 to
    87      #                    p_normal
    88      # 30/06/05 - Siggi - network changed for queue cdev from "us" to "ip"
    89      # 12/07/05 - Siggi - in network.mpi on ibmh/ibmb "csss" changed to
    90      #                    "sn_all", new job class cexp
    91      # 08/09/05 - Siggi - IP-address of gfdl3 changed
    92      # 31/10/05 - Siggi - new job class pp on hurricane, serial jobs on
    93      #                    hurricane (with -X0)
    94      # 01/11/05 - Siggi - missing queue for jobs submitted on nech (for nech)
    95      #                    added
    96      # 30/12/05 - Siggi - change of IP adresses in subnet 130.75.105
    97      # 09/02/06 - Siggi - ibmy admitted
    98      # 10/02/06 - Siggi - scp2 changed to /bin/scp on decalpha
    99      # 13/04/06 - Siggi - ostria admitted
    100      # 18/04/06 - Siggi - new option -O for OpenMP usage
    101      # 24/05/06 - Siggi - lctit admitted, ftpjob renamed scpjob
    102      # 25/07/06 - Siggi - gfdl5 (ibmy) admitted for submitting jobs
    103      # 27/09/06 - Siggi - breg/hreg extended with berni/hanni
    104      # 25/10/06 - Siggi - data_limit set to 1.76 GByte on hanni and berni
    105      # 28/11/06 - Siggi - levanto admitted
    106      # 13/02/07 - Siggi - hpmuk releated code removed
    107      # 01/03/07 - Siggi - adjustments for RIAM machines gate and NEC-SX8 (n-sx)
    108      # 12/04/07 - Siggi - option -f (filetransfer protocol) removed, scp only
    109      # 27/07/07 - Siggi - autan admitted
    110      # 03/08/07 - Marcus- lcfimm admitted
    111      # 08/10/07 - Siggi - further job classes added for hanni (csoloh...)
    112      # 15/10/07 - Siggi - preliminary adjustments for lctit based on Jin's
    113      #                    suggestions
    114      # 19/10/07 - Marcus- add new optional argument -g group_number
    115      # 19/10/07 - Siggi - a ";" was missing in the last change done by Marcus
    116      # 30/10/07 - Marcus- further adjustments for queues on lctit
    117      # 15/05/08 - Siggi - adjustments for lcxt4 (Bergen Center for Computational
    118      #                    Science)
    119      # 14/07/08 - Siggi - adjustments for lcsgih
    120      # 23/09/08 - Gerald- paesano admitted
    121      # 02/10/08 - Siggi - PBS adjustments for lcxt4
    122      # 02/03/09 - Siggi - Adjustments for new NEC-SX9 at RIAM
    123      # 16/04/09 - Marcus- Adjustments for lcsgib and lcsgih
    124      # 21/04/09 - Siggi - adjustments for new IBM at DKRZ, which is now ibmh
    125      # 18/05/09 - Siggi - Settings for serial jobs on lcsgi changed
    126      # 24/06/09 - BjornM- adjustments for lcxt4 (loading modules manually)
    127      # 08/07/09 - Siggi - option -e added (email notification on lcsgih/b)
    128      # 20/07/09 - Siggi - On lcsgi, jobs for returning the job protocol are
    129      #                    now run on the data nodes (feature=data)
    130      # 25/08/09 - BjornM- adapted for lck
    131      # 26/08/09 - Marcus- caurus admitted; optional qos feature hiprio on lcsgi
    132      # 03/09/09 - Siggi - PBS sgi feature directive only used if explicitly
    133      #                    set in the config file by the user
    134      # 16/10/09 - Carolin-adjustments for archiving on SGI-ICE of binary files;
    135      #                    adding special1q
    136      # 01/12/09 - BjornM- re-adjustments for lcxt4, added sno (130.75.105.113)
    137      # 16/10/09 - Carolin-adding permq
    138      # 01/02/10 - Siggi - adapted for lcxt5m and lckyoto (Fujitsu HX600)
    139      # 03/02/10 - Siggi - bug in serial jobs removed
    140      # 26/02/10 - BjornM- re-adjustments for lcxt4 (new modules, email
    141      #                    notification)
    142      # 01/03/10 - Siggi - loading of modules controlled by environment variable
    143      #                    module_calls
    144      # 17/08/10 - BjornM- re-adjustments for lcxt4 (location of qsub)
    145      # 25/08/10 - BjornM- account geofysisk replaced by guest for lcxt4
    146      # 25/08/10 - Siggi - new variable project_account in pbs-statements for
    147      #                    lcxt4
    148      # 08/12/10 - Siggi - initialization of the module command changed for
    149      #                    SGI-ICE/lcsgi
    150      #                    adjustments for Kyushu Univ. (lcrte, ibmku)
    151      # 14/12/10 - Siggi - adjustments for new Tsubame system at Tokyo
    152      #                    institute of technology (lctit)
    153      # 02/02/11 - Siggi - further asjustments for Tsubame concerning openMP
    154      # 06/03/11 - Siggi - adjustments for ibmkisti
    155      # 17/03/11 - Siggi - adjustments for openmp usage on ibmkisti
    156      # 03/04/11 - Micha - added lckordi
    157      # 17/08/11 - Siggi - hicegate0 added
    158      # 18/08/11 - Siggi - workaround on lcsgi in order to avoid appends to file
    159      #                    last_job_transfer_protocol
    160      # 21/08/11 - Siggi - inferno admitted
    161      # 29/11/11 - Siggi - adjustments for lcsgih/lcsgib queues, pingui admitted
    162      # 21/12/11 - Theres- solano admitted
    163      # 31/01/12 - Matthias - adjustments for ibmh
    164      # 02/05/12 - Bjoern  - lcxt4 is now lcxe6, adjustments for lcxe6,
    165      #                      bugfixes for the parameter file check, initial
    166      #                      commands on ssh calls sourced out to .mrun.config
    167      # 02/10/12 - Siggi - adjustments for lcsb
    168      # 01/11/12 - Siggi - adjustments for lckiaps
    169 
     35# 08/07/94 - Siggi - first version finished
     36# 29/06/94 - Siggi - script development started
     37#--------------------------------------------------------------------------------#
     38# subjob - script for automatic generation and submission of batch-job files
     39#          for various batch queuing systems
     40#--------------------------------------------------------------------------------#
    17041
    17142
    … …  
    284155     (scirocco)              local_addres=172.20.25.41;   local_host=lcmuk;;
    285156     (solano)                local_addres=130.75.105.110; local_host=lcmuk;;
     157     (sugoka*)               local_addres=172.31.120.1;   local_host=lckyut;;
    286158     (sun1|sun2)             local_addres=130.75.6.1;     local_host=unics;;
    287159     (sx-*)                  local_addres=172.16.1.131;   local_host=necriam;;
    … …  
    423295        (lckiaps) remote_addres=118.128.66.223; submcom=/cm/shared/apps/pbspro/11.0.2.110766/bin/qsub;;
    424296        (lckordi) remote_addres=210.219.61.8; submcom=/usr/torque/bin/qsub;;
     297        (lckyut)  remote_addres=133.5.4.37; submcom=/usr/bin/pjsub;;
    425298        (lcsb)    remote_addres=147.46.30.151; submcom=/usr/torque/bin/qsub;;
    426299        (lctit)   queue=S; remote_addres=10.1.6.165; submcom=/opt/pbs/tools/bin/t2sub;;
    … …  
    474347        (lckyoto) case  $ndq  in
    475348                     (eh|ph)                                 error=false;;
     349                     (*)                                     error=true;;
     350                 esac;;
     351        (lckyut) case  $ndq  in
     352                     (cx-dbg|cx-single|cx-small|cx-middle|cx-large)  error=false;;
    476353                     (*)                                     error=true;;
    477354                 esac;;
    … …  
    11341011    fi
    11351012
     1013 elif [[ $remote_host = lckyut ]]
     1014 then
     1015    cat > $job_to_send << %%END%%
     1016#!/bin/bash
     1017#PJM -L "vnode=$numprocs"
     1018#PJM -L "vnode-core=1"
     1019#PJM -L "rscgrp=$queue"
     1020#PJM --no-stging
     1021#PJM -L "elapse=$timestring"
     1022#PJM --mpi proc=$numprocs
     1023#PJM -o $remote_dayfile
     1024#PJM -j
     1025
     1026export LANG=en_US.UTF-8
     1027%%END%%
     1028
    11361029 elif [[ $remote_host = nech ]]
    11371030 then
    … …  
    12511144    echo "trap '"                               >>  $job_to_send
    12521145    echo "set +vx"                              >>  $job_to_send
    1253     if [[ $(echo $remote_host | cut -c1-3) = ibm  ||  $(echo $remote_host | cut -c1-5) = lcsgi  ||  $(echo $remote_host | cut -c1-3) = nec  ||  $remote_host = lckiaps ]]
     1146    if [[ $(echo $remote_host | cut -c1-3) = ibm  ||  $(echo $remote_host | cut -c1-5) = lcsgi  ||  $(echo $remote_host | cut -c1-3) = nec  ||  $remote_host = lckiaps  ||  $remote_host = lckyut ]]
    12541147    then
    12551148       if [[ $remote_host = ibmh ]]
    … …  
    12771170       then
    12781171          return_queue=express
     1172       elif [[ $remote_host = lckyut ]]
     1173       then
     1174          return_queue=cx-single
    12791175       else
    12801176          return_queue=unknown
    … …  
    13361232          echo "#PBS -j o"                         >>  $job_to_send
    13371233          echo " "                                 >>  $job_to_send
     1234          echo "set -x"                            >>  $job_to_send
     1235          echo "batch_scp  -d  -w 10  -u $local_user $local_addres  $remote_dayfile  \"$job_catalog\"  $local_dayfile"  >>  $job_to_send
     1236          echo "[[ \"\$for_subjob_to_do\" != \"\" ]]  &&  eval \$for_subjob_to_do"  >>  $job_to_send
     1237          echo "%%END%%"                           >>  $job_to_send
     1238
     1239       elif [[ $remote_host = lckyut ]]
     1240       then
     1241          echo "cat > scpjob.$kennung << %%END%%"  >>  $job_to_send
     1242          echo "#!/bin/bash"                       >>  $job_to_send
     1243          echo "#PJM -L \"vnode=1\""               >>  $job_to_send
     1244          echo "#PJM -L \"rscgrp=$return_queue\""  >>  $job_to_send
     1245          echo "#PJM --no-stging"                  >>  $job_to_send
     1246          echo "#PJM -L \"elapse=30:00\""          >>  $job_to_send
     1247          echo "#PJM -o \$HOME/job_queue/last_job_transfer_protocol"  >>  $job_to_send
     1248          echo "#PJM -j"                           >>  $job_to_send
     1249          echo " "                                 >>  $job_to_send
     1250          echo "export LANG=en_US.UTF-8"           >>  $job_to_send
    13381251          echo "set -x"                            >>  $job_to_send
    13391252          echo "batch_scp  -d  -w 10  -u $local_user $local_addres  $remote_dayfile  \"$job_catalog\"  $local_dayfile"  >>  $job_to_send
    … …  
    14031316          echo "ssh ${remote_username}@${remote_addres}  \"$submcom ${job_catalog}/scpjob.$kennung\" "  >>  $job_to_send
    14041317          echo "rm  ${job_catalog}/scpjob.$kennung"          >>  $job_to_send
     1318       elif [[ $remote_host = lckyut ]]
     1319       then
     1320          echo "scp -P 22 scpjob.$kennung  ${remote_username}@${remote_addres}:job_queue"           >>  $job_to_send
     1321          echo "ssh -p 22 ${remote_username}@${remote_addres}  \"cd job_queue; $submcom scpjob.$kennung; rm scpjob.$kennung\" "  >>  $job_to_send
    14051322       else
    14061323          echo "$submcom  scpjob.$kennung"      >>  $job_to_send
    … …  
    15201437       then
    15211438          eval  $submcom  $job_on_remhost
    1522        elif [[  $local_host = lcfimm  ||  $local_host = lctit  ||  $localhost = lcxe6  ||  $localhost = lck  || $localhost = lckordi ||  $localhost = lcyon || $localhost = lcsb ]]
     1439       elif [[  $local_host = lcfimm  ||  $local_host = lctit  ||  $localhost = lcxe6  ||  $localhost = lck  || $localhost = lckordi ||  $localhost = lcyon || $localhost = lcsb  ||  $localhost = lckyut ]]
    15231440       then
    15241441          chmod  u+x  $job_on_remhost
Note: See TracChangeset for help on using the changeset viewer.
                                                                                                                                                                                                                                                                                                                                                                               
  | Impressum | ©Leibniz Universität Hannover |