Home

Context Navigation

source: palm/trunk/SCRIPTS/mrun @ 55

Last change on this file since 55 was 54, checked in by raasch, 18 years ago
property Id set for most of all files, small bug in mbuild fixed
Property svn:keywords set to `Id`
File size: 147.1 KB

Rev	Line
[1]	1	#!/bin/ksh
	2
	3	##!/home/DSRC/NC/tatuyama/pub/ksh NEEDED ON NEC AT RIAM !!!!!!!!!!
	4
[54]	5	# mrun - Plot-Shellskript Version: @(#)MRUN 2.0 07/03/07
	6	# $Id: mrun 54 2007-03-08 00:00:02Z raasch $
[1]	7
	8	# Prozedur zum Starten von Modellaeufen
	9
	10	# letzte Aenderung:
	11	# 03/03/94 - Siggi - Entwicklungsbeginn
	12	# 21/03/94 - Siggi - Entwicklungsabschluss (Version 1.0)
	13	# 09/12/94 - Siggi - allexport unterbunden; statt dessen werden jetzt ein-
	14	# zelne Variable exportiert. Dies ist noetig, weil an-
	15	# sonsten irgendein Speicher ueberlaeuft und von mrun
	16	# aufgerufene Programme (z.B. stageout) mit Fehlern
	17	# abbrechen (too many arguments)
	18	# 20/06/95 - Siggi - noclobber muss explizit abgeschaltet, da jetzt
	19	# defaultmaessig im RRZN-Profile eingeschaltet
	20	# 10/06/97 - Siggi - Zusaetzliche Option -C bei cpp, damit // nicht
	21	# wegfallen
	22	# 25/04/00 - Siggi - Version 1.5
	23	# remote_addres und remote_user in return_addres bzw.
	24	# return_username umbenannt, return_username wird dem
	25	# mrun-Aufruf im Job ueber Option -U mitgegeben.
	26	# Neue Variable remote_username, die beim interaktiven
	27	# mrun-Aufruf zwingend ueber die neue Option -u oder
	28	# in der Konfigurationsdatei angegeben werden muss.
	29	# 08/02/01 - Siggi - Alle mrun-Meldungen ins englische uebersetzt,
	30	# Version 1.7
	31	# 05/05/02 - Siggi - Uebersetzungen mittels make-Mechanismus moeglich
	32	# 21/11/02 - Siggi - ENV-Variable XLFRTEOPTS wird vor Ausfuehrung auf
	33	# IBM gesetzt, um Record-Laenge der NAMELIST-Dateien,
	34	# zu bestimmen; Praeprozessoroptionen haben nun alle
	35	# die Form -Dtext=text, weil auf IBM sonst text durch
	36	# die leere Zeichenkette ersetzt wird
	37	# 01/08/03 - Siggi - Test of implementing dvrp+1PE on ibm
	38	# 07/01/04 - Siggi - additional preprocessor directive for ibm included
	39	# (-D$OMP=OMP) in order to avoid problems with
	40	# OMP_NUM_THREADS
	41	# 04/01/05 - Siggi - archiving on hanni and berni is done in separate
	42	# tar files, one for each node used
	43	# 07/01/04 - Siggi - old code for t3e, vpp and hpcs eliminated
	44	# 09/01/05 - Siggi - workaround for getting the IP address on gfdl3
	45	# 17/01/05 - Siggi - job for data transfer to local machine now within
	46	# job class c1 (HLRN)
	47	# 27/01/05 - Siggi - IP address workaround for gfdl3 removed
	48	# 28/01/05 - Siggi - tar-filenames on $PERM (HLRN) now include the
	49	# base filename and cycle number
	50	# 16/02/05 - Gerald - hababai validated
	51	# 14/03/05 - Siggi - abort on NEC after first runtime error (F_ERRCNT)
	52	# 29/03/05 - Marcus - berni*-en0 validated
	53	# 21/04/05 - Siggi - transfer-job-protocol for avs-data is no more stored
	54	# 24/04/05 - Siggi - netcdf support on lcmuk
	55	# 25/04/05 - Siggi - netcdf support on gfld3 (decalpha)
	56	# 11/05/05 - Siggi - files with non-numeric extensions (.xxx after cycle
	57	# number) can now be used within interactive runs
	58	# 12/05/05 - Siggi - netcdf support on ibm
	59	# 13/05/05 - Siggi - error in tar-filenames on $PERM (HLRN) removed
	60	# 18/05/05 - Siggi - netcdf support on nec
	61	# 24/05/05 - Siggi - netcdf support on ibms
	62	# 14/06/05 - Siggi - sleep interval after submit of restart job increased
	63	# to 30 seconds
	64	# 28/06/05 - Siggi - bora adjusted to new intel compiler (LD_LIBRARY_
	65	# PATH is temporarily set)
	66	# 07/09/05 - Siggi - setting of MP_EAGER_LIMIT switched off because it
	67	# caused warnings in the job protocol
	68	# 20/10/05 - Siggi - update of netcdf-version on decalpha (gfdl3)
	69	# 25/10/05 - Siggi - error in listing files for getting the cycle number
	70	# removed
	71	# 26/10/05 - Siggi - new paths for dvrp-library, transfer of catalogs
	72	# realized by file attribute trpe
	73	# 28/10/05 - Siggi - if existing, the file extension is output in case
	74	# of missing input file - no job abort in case of
	75	# missing input files with file extensions
	76	# (preliminary solution)
	77	# 31/10/05 - Siggi - data transfer from nech now within seperate job
	78	# 04/11/05 - Siggi - netcdf 3.6.0-p1 on ibmh/ibmb
	79	# 07/12/05 - Siggi - gallego and elephanta admitted
	80	# 30/12/05 - Siggi - gfdl5 (ibmy) admitted
	81	# 10/01/06 - Siggi - cpp directive for NetCDF 64bit support
	82	# 20/01/06 - Siggi - cpp directive for ibmy
	83	# 09/02/06 - Siggi - ibmy admitted for batch mode
	84	# 13/04/06 - Siggi - ostria admitted
	85	# 18/04/06 - Siggi - usage of OpenMP implemented (additional option -O)
	86	# 10/05/06 - Siggi - environment variable XLSMPOPTS set on IBM for
	87	# optimization of OpenMP runs (option stacksize is
	88	# necessary for large gridpoint numbers because
	89	# otherwise a segmentation fault occurs)
	90	# 23/05/05 - Siggi - lctit (SUN Fire X4600) admitted
	91	# 21/08/06 - Siggi - standard mrun path on scirocco is as on bora
	92	# 23/08/06 - Siggi - netcdf support for scirocco (notebook)
	93	# 20/09/06 - Marcus - add DVRP library on nech
	94	# 24/10/06 - Siggi - Environment variables for PALM steering are written
	95	# on local NAMELIST file ENVPAR
	96	# 24/11/06 - Siggi - levanto admitted, maestro switched to ifc 9.1
	97	# 28/11/06 - Siggi - call to interpret_config.x_levanto because of SuSe 10
	98	# 06/02/07 - Siggi - add DVRP library on lcmuk
	99	# 07/02/07 - Siggi - revision of source code handling, source code is
	100	# now expected in directory given by source_path and
	101	# not in the current working directory, source code
	102	# files provided in add_source_path are allways
	103	# compiled, source code to be compiled is allways
	104	# sampled in SOURCES_FOR_RUN_$fname which is a
	105	# subdirectory of the current working directory,
[5]	106	# -s TEST changed to -s WRITE_PERMIT or -s WP,
[1]	107	# adapted for RIAM (neck)
[5]	108	# bugfix for cycle number of output file (extout)
	109	# all hpmuk-related code removed
[14]	110	# 14/02/07 - Siggi - dvrp support for neck added
[21]	111	# 28/02/07 - Siggi - empty lines in configuration file are accepted
[26]	112	# mrun_path replaced by PALM_BIN,
	113	# all machines are calling interpret_config.x
[1]	114
	115
	116
	117
	118	# VARIABLENVEREINBARUNGEN + DEFAULTWERTE
	119
	120	set +o allexport # SICHERHEITSHALBER UNTERBINDEN, DA SONST EVTL. STAGEOUT
	121	# NICHT LAUEFT (TOO MANY ARGUMENTS - PROBLEM)
	122	set +o noclobber # EXISTIERENDE DATEIEN DUERFEN UEBERSCHRIEBEN WERDEN
	123
	124	AddFilenames=""
	125	additional_conditions=""
	126	add_source_path=""
	127	afname=""
	128	archive_save=true
	129	archive_system=none
	130	compiler_name=""
	131	cond1=""
	132	cond2=""
	133	config_file=.mrun.config
	134	cpp_opts=""
	135	cpumax=0
	136	cpurest=0
	137	delete_temporary_catalog=true
	138	do_batch=false
	139	do_compile=true
	140	do_remote=false
	141	do_stagein=true
	142	do_stageout=true
	143	do_trace=false
	144	email_notification=""
	145	exclude=""
	146	executable=""
	147	execution_error=false
	148	filetransfer_protocol=""
	149	fname=test
	150	fromhost=""
	151	host=""
	152	host_file=""
	153	hp=""
	154	ignore_archive_error=false
	155	input_list=""
	156	interpreted_config_file=""
	157	job_on_file=""
	158	keep_data_from_previous_run=false
	159	localhost_realname=$(hostname)
	160	local_compile=false
	161	locat=normal
	162	mainprog=""
	163	makefile=""
	164	mc=$0
	165	while [[ $(echo $mc \| grep -c "/") != 0 ]]
	166	do
	167	mc=`echo $mc \| cut -f2- -d"/"`
	168	done
	169	module_files=""
	170	MODULE_OPTIONS=""
	171	mrun_script_name=$mc
	172	netcdf_support=false
	173	node_usage=default
	174	numprocs=""
	175	OOPT=""
	176	openmp=false
	177	output_list=""
	178	package_list=""
	179	punkte="..........................................................."
	180	queue=none
	181	read_from_config=""
[5]	182	restart_run=false
[27]	183	return_addres=$(nslookup `hostname` 2>&1 \| grep "Address:" \| tail -1 \| awk '{print $2}')
[1]	184	if [[ $return_addres = 130.75.105.158 ]]
	185	then
	186	return_addres=172.20.25.41
	187	echo "+++ WARNING: return_addres changed to $return_addres !!!!!"
	188	fi
	189	# if [[ $return_addres = 210.117.65.1 ]]
	190	# if [[ $(nslookup `hostname` 2>&1 \| grep -c 210.117.65.1) = 1 ]]
	191	# then
	192	# [[ $localhost_realname = gfdl3.yonsei.ac.kr ]] && return_addres=165.132.26.56
	193	# fi
	194	return_password=""
	195	return_username=$LOGNAME
	196	remotecall=false
	197	remote_username=""
	198	run_mode=""
	199	scirocco=false
	200	store_on_archive_system=true
	201	striche=" ----------------------------------------------------------------------------"
	202	silent=false
	203	source_list=""
	204	source_path=SOURCE
	205	tasks_per_node=""
	206	threads_per_task=1
	207	tmpcreate=false
	208	tmp_data_catalog=""
	209	transfer_problems=false
	210	usern=$LOGNAME
	211	use_makefile=""
	212	working_directory=`pwd`
	213	TOPT=""
	214	XOPT=""
	215	zeit=$( date \| cut -c 12-19 )
	216
	217	typeset -i iec=0 iic=0 iin=0 ioc=0 iout=0 memory=0 stagein_anz=0 stageout_anz=0
	218	typeset -i cputime i ii icycle inode ival jobges jobsek maxcycle minuten nodes pes sekunden tp1
	219
	220	typeset -R40 calltime
	221	typeset -L20 spalte1
	222	typeset -L40 spalte2
	223	typeset -L60 spalte3
	224	typeset -L35 string1=`date`
	225	typeset -L12 string2=$usern
	226	typeset -L12 string3=$localhost_realname
	227	typeset -L12 string4
	228	typeset -L12 string5
[54]	229	typeset -L20 version="MRUN 2.0"
[1]	230
	231
	232
	233	# EINZELNE VARIABLE FUER HAUPTPROGRAMM EXPORTIEREN
	234	export cpurest fname host localhost return_addres return_username remotecall tasks_per_node
	235
	236	# FOLGENDE VARIABLEN MUESSEN FUER DIE INTERPRETATION DER KONFIGURATIONSDATEI
	237	# EXPORTIERT WERDEN
	238	export afname config_file cpp_opts cpumax do_batch do_trace fname fromhost
	239	export input_list memory numprocs output_list queue run_mode
	240
	241
	242
	243	# FEHLERBEHANDLUNG
	244	# BEI EXIT:
	245	trap 'if [[ $locat != localhost ]]
	246	then
[22]	247	# if [[ ! -f ${mrun_path}/statistik/mrun_statistik ]]
	248	# then
	249	# cat > ${mrun_path}/statistik/mrun_statistik << %STATEND%
	250	#MRUN-calls on $localhost
	251	#
	252	#date and time user localhost remotehost termination mrun-command
	253	#--------------------------------------------------------------------------------------------------------------------
	254	#%STATEND%
	255	# chmod 666 ${mrun_path}/statistik/mrun_statistik
	256	# fi
	257	#
	258	# # EINTRAG IN DIE STATISTIK-DATEI
	259	# string4=$host
	260	# string5=$locat
	261	# if [[ "$job_on_file" = "" && $locat != control_c && $locat != user_abort ]]
	262	# then
	263	# if [[ $do_batch = true ]]
	264	# then
	265	# printf "$string1$string2$string3$string4$string5$mrun_com \n" >> ${mrun_path}/statistik/mrun_statistik
	266	# else
	267	# printf "$string1$string2$string3$string4$string5$mc \n" >> ${mrun_path}/statistik/mrun_statistik
	268	# fi
	269	# fi
	270	continue
[1]	271	fi
	272
	273	if [[ $locat != normal && $locat != control_c && $locat != local_compile ]]
	274	then
	275
	276	# EVENTUELLE ERROR-KOMMANDOS ABARBEITEN
	277	(( i = 0 ))
	278	while (( i < iec ))
	279	do
	280	(( i = i + 1 ))
	281	printf "\n *** Execution of ERROR-command:\n"
	282	printf " >>> ${err_command[$i]}\n"
	283	eval ${err_command[$i]}
	284	done
	285	if [[ -n $interpreted_config_file ]] then
	286	rm -rf $interpreted_config_file
	287	fi
	288	if [[ $tmpcreate = true ]]
	289	then
	290	printf "\n *** Contents of \"$TEMPDIR\":\n"
	291	ls -al; cd
	292	[[ $delete_temporary_catalog = true ]] && rm -rf $TEMPDIR
	293	fi
	294	if [[ -f ~/job_queue/JOBINFO.$QSUB_REQID ]]
	295	then
	296	rm -rf ~/job_queue/JOBINFO.$QSUB_REQID
	297	fi
	298	printf "\n\n+++ MRUN killed \n\n"
	299	elif [[ $locat != control_c ]]
	300	then
	301	printf "\n\n --> all actions finished\n\n"
	302	printf " Bye, bye $usern !!\n\n"
	303	fi' exit
	304
	305
	306	# BEI TERMINAL-BREAK:
	307	trap '[[ $tmpcreate = true ]] && (cd; rm -rf $TEMPDIR)
	308	if [[ -f ~/job_queue/JOBINFO.$QSUB_REQID ]]
	309	then
	310	rm -rf ~/job_queue/JOBINFO.$QSUB_REQID
	311	fi
	312	printf "\n+++ MRUN killed by \"^C\" \n\n"
	313	locat=control_c
	314	exit
	315	' 2
	316
	317
[22]	318	# CHECK IF THE PATH FOR THE PALM BINARIES (SCRIPTS+UTILITY-PROGRAMS) HAS
	319	# BEEN SET
	320	if [[ "$PALM_BIN" = "" ]]
	321	then
	322	printf "\n +++ environment variable PALM_BIN has not been set"
	323	printf "\n please set it to the directory where the PALM scripts are located"
	324	locat=palm_bin; exit
	325	fi
	326
	327
[1]	328	# EVTL. NAMEN DES LOKALEN RECHNERS MIT UEBERGREIFENDEM NAMEN VERSEHEN
	329	case $localhost_realname in
[22]	330	(atmos) localhost=lcide;;
	331	(bora\|breva\|elephanta\|gallego\|gregale\|hababai\|irifi\|levanto\|maestro\|orkan\|ostria\|poniente\|quanero\|scirocco\|vorias) localhost=lcmuk;;
	332	(breg-en0\|berni-en0) localhost=ibmb; archive_system=tivoli;;
	333	(cs*) localhost=nech; archive_system=ut;;
	334	(gate\|n-sx) PATH=$MRUN_BIN:$PATH:/usr/bin/nqsII; localhost=neck;;
	335	(ground.yonsei.ac.kr) localhost=decalpha;;
	336	(gfdl3.yonsei.ac.kr) localhost=decalpha;;
	337	(gfdl5) localhost=ibmy;;
	338	(hreg-en0\|hanni-en0) localhost=ibmh; archive_system=tivoli;;
	339	(nobel*) localhost=ibms;;
	340	(sun1\|sun2) localhost=unics;;
	341	(tgg*) localhost=lctit;;
[1]	342	(*) printf "\n +++ Host \"$localhost_realname\" not admitted for mrun \n"
	343	locat=localhost; exit;;
	344	esac
	345
	346	[[ $localhost_realname = scirocco ]] && scirocco=true
	347
	348
	349
	350	# SHELLSCRIPT-OPTIONEN EINLESEN UND KOMMANDO NEU ZUSAMMENSETZEN, FALLS ES
	351	# FUER FOLGEJOBS BENOETIGT WIRD
	352	while getopts :a:AbBc:Cd:D:f:Fh:H:i:IkK:m:M:n:o:Op:P:q:r:R:s:St:T:u:U:vxX: option
	353	do
	354	case $option in
	355	(a) afname=$OPTARG;;
	356	(A) store_on_archive_system=false; mc="$mc -A";;
	357	(b) do_batch=true; mc="$mc -b";;
	358	(B) delete_temporary_catalog=false; mc="$mc -B";;
	359	(c) config_file=$OPTARG; mc="$mc -c$OPTARG";;
[5]	360	(C) restart_run=true; mc="$mc -C";;
[1]	361	(d) fname=$OPTARG; mc="$mc -d$OPTARG";;
	362	(D) cpp_opts="$cpp_opts $OPTARG"; mc="$mc -D$OPTARG";;
	363	(f) filetransfer_protocol=$OPTARG; mc="$mc -f$OPTARG";;
	364	(F) job_on_file="-D"; mc="$mc -F";;
	365	(h) host=$OPTARG; mc="$mc -h$OPTARG";;
	366	(H) fromhost=$OPTARG; mc="$mc -H$OPTARG";;
	367	(i) input_list=$OPTARG; mc="$mc -i'$OPTARG'";;
	368	(I) ignore_archive_error=true; mc="$mc -I";;
	369	(k) keep_data_from_previous_run=true; mc="$mc -k";;
	370	(K) additional_conditions="$OPTARG"; mc="$mc -K$OPTARG";;
	371	(m) memory=$OPTARG; mc="$mc -m$OPTARG";;
	372	(M) makefile=$OPTARG; mc="$mc -M$OPTARG";;
	373	(n) node_usage=$OPTARG; mc="$mc -n$OPTARG";;
	374	(o) output_list=$OPTARG; mc="$mc -o'$OPTARG'";;
	375	(O) use_openmp=true; mc="$mc -O";;
	376	(p) package_list=$OPTARG; mc="$mc -p'$OPTARG'";;
	377	(P) return_password=$OPTARG; mc="$mc -P$OPTARG";;
	378	(q) queue=$OPTARG; mc="$mc -q$OPTARG";;
	379	(r) run_mode=$OPTARG; mc="$mc -r'$OPTARG'";;
	380	(R) remotecall=true;return_addres=$OPTARG; mc="$mc -R$OPTARG";;
	381	(s) source_list=$OPTARG; mc="$mc -s'$OPTARG'";;
	382	(S) read_from_config=false; mc="$mc -S";;
	383	(t) cpumax=$OPTARG; mc="$mc -t$OPTARG";;
	384	(T) tasks_per_node=$OPTARG; mc="$mc -T$OPTARG";;
	385	(u) remote_username=$OPTARG; mc="$mc -u$OPTARG";;
	386	(U) return_username=$OPTARG; mc="$mc -U$OPTARG";;
	387	(v) silent=true; mc="$mc -v";;
	388	(x) do_trace=true;set -x; mc="$mc -x";;
	389	(X) numprocs=$OPTARG; mc="$mc -X$OPTARG";;
	390	(\?) printf "\n +++ unknown option $OPTARG \n"
	391	printf "\n --> type \"$0 ?\" for available options \n"
	392	locat=parameter;exit;;
	393	esac
	394	done
	395
	396
	397	# EVTL. POSITIONSPARAMETER EINLESEN
	398	# ZUR ZEIT GIBT ES NUR DEN PARAMETER ? (=KURZINFO)
	399	shift OPTIND-1
	400
	401
	402	# KURZE AUFRUFBESCHREIBUNG WIRD HIER AUSGEGEBEN
	403	if [[ "$1" = "?" ]]
	404	then
	405	(printf "\n *** mrun can be called as follows:\n"
	406	printf "\n $mrun_script_name -b -c.. -d.. -D.. -f.. -F -h.. -i.. -I -K.. -m.. -o.. -p.. -r.. -R -s.. -t.. -T.. -v -x -X.. <modus>\n"
	407	printf "\n Description of available options:\n"
	408	printf "\n Option Description Default-Value"
	409	printf "\n -a base name of input files equiv. -d"
	410	printf "\n -A no archiving when using file-attribute fl"
	411	printf "\n -b batch-job on local machine ---"
	412	printf "\n -B do not delete temporary directory at end ---"
	413	printf "\n -c configuration file .mrun.config"
	414	printf "\n -d base name of files attached to program test"
	415	printf "\n -D preprocessor(cpp)-directives \"\" "
	416	printf "\n -f filetransfer protocol (ftp or scp) depending on -h"
	417	printf "\n -F create remote job file only ---"
	418	printf "\n -h execution host $localhost_realname"
	419	printf "\n -i INPUT control list \"\" "
	420	printf "\n -I archiving errors of previous batch-jobs"
	421	printf "\n will be ignored"
	422	printf "\n -k keep data from previous run"
	423	printf "\n -K additional conditions for controling"
	424	printf "\n usage of conditional code and"
	425	printf "\n env-variables in configuration file \"\" "
	426	printf "\n -m memory demand in MB (batch-jobs) 0 MB"
	427	printf "\n -M Makefile name Makefile"
	428	printf "\n -n node usage (shared/not_shared) depending on -h"
	429	printf "\n -o OUTPUT control list \"\" "
	430	printf "\n -O use OpenMP ---"
	431	printf "\n -p software package list \"\" "
	432	printf "\n -q queue \"$queue\" "
	433	printf "\n -r run control list (combines -i -o) \"\" "
	434	printf "\n -s filenames of routines to be compiled \"\" "
	435	printf "\n must end with .f, .f90, .F, or .c !"
	436	printf "\n use \"..\" for more than one file and wildcards"
	437	printf "\n -s TEST compiles all files with w-permit"
	438	printf "\n -S config file interpreted by shellscript ---"
	439	printf "\n -t allowed cpu-time in seconds (batch) 0"
	440	printf "\n -T tasks per node depending on -h"
	441	printf "\n -u username on remote machine \"\" "
	442	printf "\n -v no prompt for confirmation ---"
	443	printf "\n -x tracing of mrun for debug purposes ---"
	444	printf "\n -X # of processors (on parallel machines) 1"
	445	printf "\n "
	446	printf "\n Possible values of positional parameter <modus>:"
	447	printf "\n \"?\" - this outline \n\n") \| more
	448	exit
	449	elif [[ "$1" != "" ]]
	450	then
	451	printf "\n +++ positional parameter $1 unknown \n"
	452	locat=parameter; exit
	453	fi
	454
	455
	456
	457	# KURZE STARTMELDUNG
	458	printf "\n*** $version "
	459	printf "\n will be executed. Please wait ..."
	460
	461
	462	# BASISNAME DER INPUT-DATEIEN GLEICH ALLGEMEINEM BASISNAMEN SETZEN,
	463	# WENN NICHT VOM BENUTZER ANDERS BESTIMMT
	464	[[ "$afname" = "" ]] && afname=$fname
	465
	466
	467	# EVTL. RUN-MODUS DEN I/O-LISTEN HINZUFUEGEN
	468	if [[ "$run_mode" != "" ]]
	469	then
	470	input_list="$input_list $run_mode"
	471	output_list="$output_list $run_mode"
	472	fi
	473
	474
	475	# RECHNERNAMEN ABSPEICHERN, VON DEM AUS JOB GESTARTET WIRD,
	476	# ALLERDINGS NUR DANN, WENN NICHT PER OPTION -H BEREITS EIN WERT
	477	# ZUGEWIESEN WURDE (MRUN MACHT DIES IMMER, WENN ES SELBST BATCH-JOBS
	478	# STARTET)
	479	if [[ "$fromhost" = "" ]]
	480	then
	481	fromhost=$localhost
	482	fi
	483
	484
	485	# PRUEFEN, OB MRUN NUR TESTWEISE AUF DER LOKALEN MASCHINE KOMPILIEREN SOLL
	486	if [[ "$source_list" = LOCAL_COMPILE_TEST ]]
	487	then
	488	source_list=TEST
	489	local_compile=true
	490	host=$localhost
	491	fi
	492
	493
	494	# PRUEFEN, OB AUF REMOTE-MASCHINE GERECHNET WERDEN SOLL
	495	# WERT VON do_remote WIRD FUER DATEIVERBINDUNGEN BENOETIGT.
	496	# WENN AUF REMOTE-MASCHINE GERECHNET WIRD, IST GLEICHZEITIG KLAR,
	497	# DASS EIN BATCH-JOB GESTARTET WERDEN MUSS
	498	if [[ -n $host && "$host" != $localhost ]]
	499	then
	500	do_batch=true
	501	do_remote=true
	502	case $host in
[5]	503	(ibm\|ibmb\|ibmh\|ibms\|ibmy\|nech\|neck\|lctit\|unics) true;;
[1]	504	(*) printf "\n"
	505	printf "\n +++ execution host \"$host\" not admitted for MRUN,"
	506	printf "\n or NQS not available on execution host!"
	507	locat=nqs; (( iec = 0 )); exit;;
	508	esac
	509	else
	510	host=$localhost
	511	fi
	512
	513
	514	# ZUSATZBEDINGUNGEN (OPTION -K) AUSWERTEN
	515	if [[ -n $additional_conditions ]]
	516	then
	517	echo $additional_conditions \| cut -d" " -f1-3 \| read cond1 cond2 dummy
	518	# cond1=`echo $additional_conditions \| cut -d" " -f1`
	519	# cond2=`echo $additional_conditions \| cut -d" " -s -f2`
	520	# dummy=`echo $additional_conditions \| cut -d" " -s -f3`
	521	if [[ -n $dummy ]]
	522	then
	523	printf "\n +++ more than 2 additional conditions given for Option \"-K\""
	524	locat=options; exit
	525	fi
	526	fi
	527
	528
	529	# PRUEFEN, OB EVTL. BEI VORHERGEHENDEM LAUF (KETTENJOB) EINE
	530	# ARCHIVIERUNG FEHLGESCHLAGEN IST
	531	if [[ -f ~/job_queue/ARCHIVE_ERROR_$fname ]]
	532	then
	533	if [[ $ignore_archive_error = false ]]
	534	then
	535	printf "\n +++ data archiving of previous run failed"
	536	printf "\n see directory \~/job_queue on remote machine"
	537	locat=archive; exit
	538	else
	539	printf "\n +++ warning: data archiving in a previous run failed"
	540	printf "\n MRUN continues, trying to get backup copy"
	541	fi
	542	fi
	543
	544
	545	# PRUEFEN, OB KONFIGURATIONS-DATEI VORHANDEN
	546	if [[ ! -f $config_file ]]
	547	then
	548	printf "\n\n +++ configuration file: "
	549	printf "\n $config_file"
	550	printf "\n does not exist"
	551	locat=connect; exit
	552	fi
	553
	554
	555
	556	# LESEN UND INTERPRETIEREN DER KONFIGURATIONS-DATEI VOM SHELLSCRIPT AUS
	557	# VORUEBERGEHEND ZWINGEND AUF LINUX-RECHNERN
	558	if [[ "$read_from_config" = false ]]
	559	then
	560
	561	[[ $silent = false ]] && printf "\n Reading the configuration file... "
	562	while read zeile
	563	do
	564	[[ $silent = false ]] && printf "."
	565
	566
	567	# ZUERST EVENTUELL VORKOMMENDE ENVIRONMENT-VARIABLEN DURCH IHRE WERTE
	568	# ERSETZEN
	569	eval zeile=\"$zeile\"
	570
	571
	572	# INTERPRETATION DER ZEILE
[21]	573	if [[ "$(echo $zeile)" = "" ]]
[1]	574	then
[21]	575	# LEERZEILE, KEINE AKTION
	576	continue
[1]	577
[21]	578	elif [[ "$(echo $zeile \| cut -c1)" = "#" ]]
	579	then
	580
[1]	581	# ZEILE IST KOMMENTARZEILE
	582	true
	583
	584	elif [[ "$(echo $zeile \| cut -c1)" = "%" ]]
	585	then
	586
	587	# ZEILE DEFINIERT ENVIRONMENT-VARIABLE
	588	zeile=$(echo $zeile \| cut -c2-)
	589	echo $zeile \| cut -d" " -f1-5 \| read var value for_host for_cond1 for_cond2
	590	# var=`echo $zeile \| cut -d" " -f1`
	591	# value=`echo $zeile \| cut -d" " -s -f2`
	592	# for_host=`echo $zeile \| cut -d" " -s -f3`
	593	# for_cond1=`echo $zeile \| cut -d" " -s -f4`
	594	# for_cond2=`echo $zeile \| cut -d" " -s -f5`
	595
	596	if [[ "$for_host" = "" \|\| ( "$for_host" = $host && "$for_cond1" = "$cond1" && "$for_cond2" = "$cond2" ) \|\| $(echo "$input_list$output_list"\|grep -c "$for_host") != 0 ]]
	597	then
	598
	599	# BEI COMPILER- CPP- ODER LINKEROPTIONEN EVTL ":" DURCH " "
	600	# ERSETZEN. "::" WIRD DURCH ":" ERSETZT.
	601	value=`echo $value \| sed 's/::/%DUM%/g' \| sed 's/:/ /g' \| sed 's/%DUM%/:/g'`
	602
	603
	604	# ENVIRONMENT-VARIABLE WIRD WERT AUS KONFIGURATIONSDATEI
	605	# ZUGEWIESEN, WENN SIE SELBST NOCH KEINEN WERT UEBER DIE
	606	# ENTSPRECHENDE SCRIPT-OPTION ERHALTEN HAT. SOLCHE
	607	# VARIABLEN HAETTEN DANN DEN WERT "" ODER IM INTEGER-FALL DEN
	608	# WERT 0. ALLGEMEINE REGEL ALSO: SCRIPT-OPTION GEHT UEBER
	609	# KONFIGURATIONSDATEI
	610	if [[ "$(eval echo \$$var)" = "" \|\| "$(eval echo \$$var)" = "0" ]]
	611	then
	612	eval $var=\$value
	613
	614	# EVTL. BILDSCHIRMAUSGABEN DER ENVIRONMENT-VARIABLEN
	615	if [[ $do_trace = true ]]
	616	then
	617	printf "\n*** ENVIRONMENT-VARIABLE $var = $value"
	618	fi
	619	fi
	620
	621	# WENN ENVIRONMENT-VARIABLE HOST VEREINBART, DANN SOFORT AUSWERTEN
	622	# WERT VON do-remote WIRD BEI DATEIVERBINDUNGEN BENOETIGT
	623	# WENN AUF REMOTE-MASCHINE GERECHNET WIRD, IST GLEICHZEITIG KLAR,
	624	# DASS EIN BATCH-JOB GESTARTET WERDEN MUSS
	625	if [[ $var = host ]]
	626	then
	627	if [[ -n $host && "$host" != $localhost ]]
	628	then
	629	do_batch=true
	630	do_remote=true
	631	case $host in
	632	(ibm\|ibms\|ibmy\|lctit\|nech\|neck\|unics) true;;
	633	(*) printf "\n +++ no queueing system available on execution host \"$host\""
	634	locat=nqs; exit;;
	635	esac
	636	else
	637	host=$localhost
	638	fi
	639	fi
	640
	641	# VOM BENUTZER DEFINIERTE ENVIRONMENT VARIABLEN MUESSEN PRINZIPIELL
	642	# EXPORTIERT WERDEN, DA SIE VIELLEICHT IN WEITER UNTEN AUFZURUFEN-
	643	# DEN PROGRAMMEN BENOETIGT WERDEN
	644	export $var
	645	fi
	646
	647
	648
	649
	650	elif [[ "$(echo $zeile \| cut -c1-3)" = "EC:" ]]
	651	then
	652
	653	# ZEILE DEFINIERT ERROR-KOMMANDO
	654	(( iec = iec + 1 ))
	655	zeile=$(echo $zeile \| cut -c4-)
	656	err_command[$iec]="$zeile"
	657
	658	elif [[ "$(echo $zeile \| cut -c1-3)" = "IC:" ]]
	659	then
	660
	661	# ZEILE DEFINIERT INPUT-KOMMANDO
	662	(( iic = iic + 1 ))
	663	zeile=$(echo $zeile \| cut -c4-)
	664	in_command[$iic]="$zeile"
	665
	666	elif [[ "$(echo $zeile \| cut -c1-3)" = "OC:" ]]
	667	then
	668
	669	# ZEILE DEFINIERT OUTPUT-KOMMANDO
	670	(( ioc = ioc + 1 ))
	671	zeile=$(echo $zeile \| cut -c4-)
	672	out_command[$ioc]="$zeile"
	673
	674	else
	675
	676	# ZEILE DEFINIERT DATEIVERBINDUNG. EINLESEN DER DATEIEIGENSCHAFTEN
	677	# s2a: in/out - Feld
	678	# s2b: loc - Feld (optional)
	679	# s2c: tr/ar - Feld (optional)
	680	echo $zeile \| cut -d" " -f1-2 \| read s1 s2
	681	# s1=`echo $zeile \| cut -d" " -f1`
	682	# s2=`echo $zeile \| cut -d" " -s -f2`
	683	s2a=$(echo $s2 \| cut -d":" -f1)
	684	if [[ $(echo $s2 \| grep -c ":") = 0 ]]
	685	then
	686	s2b=""
	687	s2c=""
	688	else
	689	echo $s2 \| cut -d":" -f2-3 \| sed 's/:/ /g' \| read s2b s2c
	690	# s2b=`echo $s2 \| cut -d":" -f2 \| sed 's/:/ /g'`
	691	# s2c=`echo $s2 \| cut -d":" -s -f3 \| sed 's/:/ /g'`
	692	fi
	693	echo $zeile \| cut -d" " -f3-6 \| read s3 s4 s5 s6
	694	# s3=`echo $zeile \| cut -d" " -f3`
	695	# s4=`echo $zeile \| cut -d" " -s -f4`
	696	# s5=`echo $zeile \| cut -d" " -s -f5`
	697	# s6=`echo $zeile \| cut -d" " -s -f6`
	698
	699
	700	# ABSPEICHERN DER DATEIVERBINDUNG, FALLS IN INPUT- ODER OUTPUT-LIST
	701	# VERMERKT. VARIABLE S3 KANN AUCH LISTE ENTHALTEN (FELDTRENNER ":")
	702	# DATEIVERBINDUNG WIRD DANN NICHT ABGESPEICHERT UND GEPRUEFT, WENN
	703	# PROGRAMMLAUF AUF REMOTE-MASCHINE ERFOLGT UND DATEI NUR LOKAL VOR-
	704	# HANDEN SEIN MUSS (D.H. s2b = loc)
	705	IFSALT="$IFS"; IFS="$IFS:"
	706	if [[ "$s2a" = in && ! ( $do_remote = true && ( "$s2b" = loc \|\| "$s2b" = locopt ) ) ]]
	707	then
	708	found=false
	709	for actual in $input_list
	710	do
	711	for formal in $s3
	712	do
	713	[[ $actual = $formal \|\| "$formal" = "-" ]] && found=true
	714	done
	715	done
	716	if [[ $found = true ]]
	717	then
	718	(( iin = iin + 1 ))
	719	localin[$iin]=$s1; transin[$iin]=$s2b; actionin[$iin]=$s2c;
	720	typein[$iin]=$s3; pathin[$iin]=$s4; endin[$iin]=$s5;
	721	extin[$iin]=$s6
	722	fi
	723	elif [[ "$s2a" = out && ! ( $do_remote = true && "$s2b" = loc ) ]]
	724	then
	725	found=false
	726	for actual in $output_list
	727	do
	728	for formal in $s3
	729	do
	730	[[ $actual = $formal \|\| "$formal" = "-" ]] && found=true
	731	done
	732	done
	733	if [[ $found = true ]]
	734	then
	735	(( iout = iout + 1 ))
	736	localout[$iout]=$s1; actionout[$iout]=$s2c; typeout[$iout]=$s3;
	737	pathout[$iout]=$s4; endout[$iout]=$s5; extout[$iout]=$s6
	738	fi
	739	elif [[ "$s2a" != in && "$s2a" != out ]]
	740	then
	741	printf "\n +++ I/O-attribute in configuration file $config_file has the invalid"
	742	printf "\n value \"$s2\". Only \"in\" and \"out\" are allowed!"
	743	locat=connect; exit
	744	fi
	745	IFS="$IFSALT"
	746	fi
	747	done < $config_file
	748
	749	else
	750
	751
	752	# INTERPRETATION DER KONFIGURATIONSDATEI MITTELS FORTRAN 90 - PROGRAMM
	753	[[ $silent = false ]] && printf "..."
	754	export cond1 cond2 config_file do_remote do_trace input_list localhost output_list
	755	export interpreted_config_file=.icf.$RANDOM
	756
[26]	757	# case $localhost_realname in
	758	# (caurus\|twister) interpret_config.hp;;
	759	# (bora\|breva\|elephanta\|gallego\|gregale\|hababai\|maestro\|orkan\|ostria) export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/muksoft/packages/intel_ifc/8.1/lib/; interpret_config_9.x;;
	760	# (levanto\|vorias) export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/muksoft/packages/intel/ifort/9.1.037/lib/; interpret_config_9.x_levanto;;
	761	# (quanero) export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/intel_fc_80/lib; interpret_config_9.x;;
	762	# (*) interpret_config.x;;
	763	# esac
[1]	764
[26]	765	interpret_config.x
[1]	766
	767	# AUSFUEHRUNG DER GENERIERTEN SHELL-KOMMANDOS IN DIESER SHELL
	768	chmod u+x $interpreted_config_file
	769	export PATH=$PATH:.
	770	. $interpreted_config_file
	771	rm $interpreted_config_file
	772
	773	fi
	774
	775
	776	# QUELLTEXTVERZEICHNIS AUF LOKALER MASCHINE AUS KONFIGURATIONSDATEI
	777	# BESTIMMEN (WUERDE SONST EVTL. DAS VERZEICHNIS DES JEWEILS UNTER -h
	778	# ANGEGEBENEN REMOTE-RECHNERS SEIN)
	779	# BEI BATCH-JOBS SIND DIE ZU UEBERSETZENDEN PROGRAMMTEILE SCHON KOMPLETT
	780	if [[ "$SOURCES_COMPLETE" = "" ]]
	781	then
	782
	783	# ZUERST PRUEFEN, OB EIN GLOBALER QUELLTEXTPFAD FUER ALLE RECHNER
	784	# VEREINBART WURDE
	785	source_path=""
	786	line=""
	787	grep "%source_path" $config_file > tmp_mrun
	788	while read line
	789	do
	790	if [[ "$line" != "" && $(echo $line \| cut -c1) != "#" ]]
	791	then
	792	if [[ "$(echo $line \| cut -d" " -f3)" = "" ]]
	793	then
	794	global_source_path=`echo $line \| cut -d" " -f2`
	795	fi
	796	fi
	797	done < tmp_mrun
	798
	799	line=""
	800	found=false
	801	grep " $localhost" $config_file \| grep "%source_path" > tmp_mrun
	802	while read line
	803	do
	804	if [[ "$line" != "" && $(echo $line \| cut -c1) != "#" ]]
	805	then
	806	if [[ $found = true ]]
	807	then
	808	printf "\n\n +++ more than one source path found in configuration file"
	809	printf "\n for local host \"$localhost\" "
	810	locat=source_path; exit
	811	fi
	812	source_path=`echo $line \| cut -d" " -f2`
	813	found=true
	814	fi
	815	done < tmp_mrun
	816	rm tmp_mrun
	817
	818	if [[ "$source_path" = "" ]]
	819	then
	820	if [[ "$global_source_path" != "" ]]
	821	then
	822	source_path=$global_source_path
	823	else
	824	printf "\n\n +++ no source path found in configuration file"
	825	printf "\n for local host \"$localhost\" "
	826	locat=source_path; exit
	827	fi
	828	fi
	829	eval source_path=$source_path
	830
	831	if [[ ! -d $source_path ]]
	832	then
	833	printf "\n\n +++ source path \"$source_path\" on local host"
	834	printf "\n \"$localhost\" does not exist"
	835	locat=source_path; exit
	836	fi
	837
	838	fi
	839
	840
	841
	842	# NOCHMAL PRUEFEN, OB AUF REMOTE-MASCHINE GERECHNET WERDEN SOLL
	843	# (HOST KANN IN KONFIGURATIONSDATEI ANDERS FESTGELEGT WORDEN SEIN)
	844	# WERT VON do_remote WIRD FUER DATEIVERBINDUNGEN BENOETIGT.
	845	# WENN AUF REMOTE-MASCHINE GERECHNET WIRD, IST GLEICHZEITIG KLAR,
	846	# DASS EIN BATCH-JOB GESTARTET WERDEN MUSS
	847	if [[ -n $host && "$host" != $localhost ]]
	848	then
	849	do_batch=true
	850	do_remote=true
	851	case $host in
[5]	852	(ibm\|ibmb\|ibmh\|ibms\|ibmy\|lctit\|nech\|neck\|unics) true;;
[1]	853	(*) printf "\n"
	854	printf "\n +++ execution host \"$host\" not admitted for MRUN,"
	855	printf "\n or no queueing system available on execution host!"
	856	locat=nqs; (( iec = 0 )); exit;;
	857	esac
	858	else
	859	host=$localhost
	860	fi
	861
	862
	863	# PRUEFUNG EINIGER PROZESSORZAHLANGABEN BEI RECHNUNGEN AUF PARALLELRECHNERN
	864	if [[ "$cond1" = parallel \|\| "$cond2" = parallel ]]
	865	then
	866
	867	# PRUEFEN, OB DIE ANZAHL DER ZU VERWENDENDEN PES ANGEGEBEN WURDE
	868	if [[ ! -n $numprocs ]]
	869	then
	870	printf "\n"
	871	printf "\n +++ option \"-K parallel\" requires additional specification"
	872	printf "\n of the number of processors to be used by"
	873	printf "\n mrun-option \"-X\" or by environment-variable"
	874	printf "\n \"numprocs\" in the configuration file"
	875	locat=numprocs; (( iec = 0 )); exit
	876	fi
	877
	878	# PRUEFEN, OB DIE PROZESSORANZAHL PRO KNOTEN ANGEGEBEN WURDE (GGF.
	879	# DEFAULT-WERT SETZEN) UND OB SIE EIN GANZZAHLIGER TEILER DER
	880	# GESAMTPROZESSORANZAHL IST
	881	if [[ $host = nech \|\| $host = neck \|\| $host = ibmh \|\| $host = ibmb \|\| $host = ibms ]]
	882	then
	883	[[ "$tasks_per_node" = "" ]] && tasks_per_node=6
	884	(( ival = $tasks_per_node ))
	885	(( pes = numprocs ))
	886	# if [[ $(echo $package_list \| grep -c dvrp_graphics+1PE) = 1 ]]
	887	# then
	888	# (( pes = pes - 1 ))
	889	# fi
	890	(( ii = pes / ival ))
	891	if (( pes - ii * ival > 0 ))
	892	then
	893	printf "\n"
	894	printf "\n +++ tasks per node (option \"-T\") must be an integral"
	895	printf "\n divisor of the total number of processors (option \"-X\")"
	896	printf "\n values of this mrun-call: \"-T $tasks_per_node\" \"-X $numprocs\""
	897	locat=tasks_per_node; (( iec = 0 )); exit
	898	fi
	899	fi
	900
	901	# IBMY HAT NUR EINEN KNOTEN
	902	if [[ $host = ibmy ]]
	903	then
	904	if [[ "$tasks_per_node" != "" && "$tasks_per_node" != "$numprocs" ]]
	905	then
	906	printf "\n"
	907	printf "\n +++ on ibmy, tasks per node (option \"-T\") must be equal to the"
	908	printf "\n total number of processors (option \"-X\")"
	909	printf "\n values of this mrun-call: \"-T $tasks_per_node\" \"-X $numprocs\""
	910	locat=tasks_per_node; (( iec = 0 )); exit
	911	fi
	912	fi
	913
	914	# FALLS OPENMP PARALLELISIERUNG VERWENDET WERDEN SOLL, ANZAHL VON THREADS
	915	# SETZEN UND ZAHL DER TASKS PRO KNOTEN AUF 1 SETZEN
	916	if [[ $use_openmp = true ]]
	917	then
	918	threads_per_task=$tasks_per_node
	919	tasks_per_node=1
	920	fi
	921	OOPT="-O $threads_per_task"
	922	TOPT="-T $tasks_per_node"
	923
	924	# GESAMTZAHL DER KNOTEN BESTIMMEN
	925	if [[ "$tasks_per_node" != "" ]]
	926	then
	927	(( nodes = numprocs / ( tasks_per_node * threads_per_task ) ))
	928	fi
	929
	930	# PRUEFEN, OB NODE USAGE EINEN ERLAUBTEN WERT HAT BZW. DEN DEFAULT
	931	# SETZEN
	932	if [[ $node_usage = default ]]
	933	then
	934	if [[ $host = ibms ]]
	935	then
	936	node_usage=shared
	937	else
	938	node_usage=not_shared
	939	fi
	940	fi
	941	if [[ $node_usage != shared && $node_usage != not_shared ]]
	942	then
	943	printf "\n"
	944	printf "\n +++ node usage (option \"-n\") is only allowed to be set"
	945	printf "\n \"shared\" or \"not_shared\""
	946	locat=tasks_per_node; (( iec = 0 )); exit
	947	fi
	948
	949	fi
	950
	951	# PRUEFEN, OB HOSTFILE EXISTIERT
	952	if [[ -n $hostfile ]]
	953	then
	954	if [[ ! -f $hostfile ]]
	955	then
	956	printf "\n"
	957	printf "\n +++ hostfile \"$hostfile\" does not exist"
	958	locat=hostfile; exit
	959	fi
	960	fi
	961
	962	# PRUEFEN, OB RHOSTS DATEI EXISTIERT. GEGEBENENFALLS ANLEGEN BZW. ERWEITERN
	963	if [[ $host = ibmy && $do_remote = false ]]
	964	then
	965	if [[ ! -f $HOME/.rhosts ]]
	966	then
	967	echo "gfdl5.yonsei.ac.kr" > $HOME/.rhosts
	968	printf "\n\n *** file:"
	969	printf "\n $HOME/.rhosts"
	970	printf "\n was created\n"
	971	fi
	972	if [[ $(grep -c gfdl5.yonsei.ac.kr $HOME/.rhosts) = 0 ]]
	973	then
	974	echo "gfdl5.yonsei.ac.kr" >> $HOME/.rhosts
	975	printf "\n\n *** file:"
	976	printf "\n $HOME/.rhosts"
	977	printf "\n was extended by the name of the current host\n"
	978	fi
	979	fi
	980
	981	# DEFAULT-DATEITRANSFER-PROTOKOLL FESTLEGEN
	982	if [[ "$filetransfer_protocol" = "" ]]
	983	then
	984	case $host in
	985	(nech\|neck\|ibm\|ibmb\|ibmh\|ibms\|ibmy\|lctit) filetransfer_protocol=scp;;
	986	(*) filetransfer_protocol=ftp;;
	987	esac
	988	fi
	989
	990	# PRUEFEN, OB ZULAESSIGES DATEITRANSFER-PROTOKOLL GEWAEHLT WURDE
	991	if [[ "$filetransfer_protocol" != ftp && "$filetransfer_protocol" != scp ]]
	992	then
	993	printf "\n"
	994	printf "\n +++ illegal value \"$filetransfer_protocol\" for Option -f"
	995	printf "\n only \"ftp\" or \"scp\" are allowed"
	996	locat=options; exit
	997	fi
	998
	999
	1000	# FALLS NICHT VORGEGEBEN, DEFAULT-QUEUE AUF DER ZIELMASCHINE FESTLEGEN
	1001	if [[ $queue = none ]]
	1002	then
	1003	case $host in
	1004	(ibmb) if [[ $node_usage = shared ]]
	1005	then
	1006	queue=cshare
	1007	else
	1008	queue=csolo
	1009	fi;;
	1010	(ibmh) if [[ $node_usage = shared ]]
	1011	then
	1012	queue=cshare
	1013	else
	1014	queue=csolo
	1015	fi;;
	1016	(ibmy) queue=parallel;;
	1017	(lctit) queue=default;;
	1018	(nech) queue=none;;
	1019	(neck) queue=P;;
	1020	(unics) queue=unics;;
	1021	esac
	1022	fi
	1023
	1024
	1025	# EVTL. NETCDF-UNTERSTUETZUNG SETZEN
	1026	case $host in
	1027	(decalpha) netcdf_support=true
	1028	netcdf_path=/usr/local/netcdf-3.5.1;;
	1029	(ibmb\|ibmh) netcdf_support=true
	1030	netcdf_path=/aws/dataformats/netcdf-3.6.0-p1/64-32;;
	1031	(ibms) netcdf_support=true
	1032	netcdf_path=/applic/lib/NETCDF64;;
	1033	(ibmy) netcdf_support=true
	1034	netcdf_path=/usr1/users/raasch/pub/netcdf-3.6.0-p1;;
	1035	(lcmuk) netcdf_support=true
	1036	if [[ $scirocco = true ]]
	1037	then
	1038	netcdf_path=/opt/netcdf/3.6.2-beta4
	1039	else
	1040	netcdf_path=/muksoft/packages/netcdf/linux
	1041	fi;;
	1042	(lctit) netcdf_support=true
	1043	netcdf_path=/home2/usr5/mkanda/netcdf-3.6.1;;
	1044	(nech) netcdf_support=true
	1045	netcdf_path=/pool/SX-6/netcdf/netcdf-3.6.0-p1;;
	1046	(neck) netcdf_support=true
	1047	netcdf_path=/home/DSRC/NC/tatuyama/pub/netcdf-3.6.0-p1;;
	1048	esac
	1049
	1050
	1051	# VOLLSTAENDIGE DATEINAMEN DER INPUT-FILES BILDEN,
	1052	# INPUT-DATEIEN AUF VORHANDENSEIN PRUEFEN UND EVTL. HOECHSTE ZYKLUSNUMMER
	1053	# ERMITTELN
	1054	(( i = 0 ))
	1055	while (( i < iin ))
	1056	do
	1057	(( i = i + 1 ))
	1058	(( maxcycle = 0 ))
	1059
	1060	# NAMENSBILDUNG (EVTL. IST FESTER DATEINAME VORGEGEBEN)
	1061	if [[ "${actionin[$i]}" = di ]]
	1062	then
	1063	remotepathin[$i]=${pathin[$i]}/${endin[$i]} # REMOTE-PFAD ERST AUF
	1064	# REM-MASCHINE AUSWERTEN
	1065	eval filename=${pathin[$i]}/${endin[$i]}
	1066	else
	1067	remotepathin[$i]=${pathin[$i]}/${afname}${endin[$i]} # REMOTE-PFAD ERST AUF
	1068	# REM-MASCHINE AUSWERTEN
	1069	eval filename=${pathin[$i]}/${afname}${endin[$i]}
	1070	fi
	1071
	1072	# PRUEFEN AUF VORHANDENSEIN
	1073	if [[ $(ls $filename* 2>&1 \| grep -c "not found") = 1 \|\| \
	1074	$(ls $filename* 2>&1 \| grep -c "No such file") = 1 \|\| \
	1075	$(ls $filename* 2>&1 \| grep -c "does not exist") = 1 ]]
	1076	then
	1077	# DATEIEN MIT EXTENSION (Z.B. NC) MUESSEN NICHT ZWINGEND VORHANDEN
	1078	# SEIN, DESHALB IN SOLCHEN FAELLEN KEIN ABBRUCH. DIES IST NUR EINE
	1079	# VORUEBERGEHENDE LOESUNG (OKT 05). NICHT ZWINGEND BENOETIGTE
	1080	# EINGABEDATEIEN SOLLTEN EINE SPEZIELLE OPTION IN DER DATEI-
	1081	# VERBINDUNGSANWEISUNG BEKOMMEN (Z.B. inopt?)
	1082	if [[ "${transin[$i]}" != "locopt" ]]
	1083	then
	1084	printf "\n\n +++ INPUT-file: "
	1085	if [[ "${extin[$i]}" = "" ]]
	1086	then
	1087	printf "\n $filename"
	1088	else
	1089	printf "\n $filename.${extin[$i]}"
	1090	fi
	1091	printf "\n does not exist\n"
	1092	locat=input; exit
	1093	else
	1094	transin[$i]="unavailable"
	1095	fi
	1096	else
	1097
	1098	# ZYKLUSNUMMER FESTSTELLEN
	1099	ls -1 -d $filename > filelist 2>/dev/null
	1100	ls -1 -d $filename.* >> filelist 2>/dev/null
	1101	while read zeile
	1102	do
	1103	cycle=$(print $zeile \| cut -f2 -d".")
	1104	if [[ "$cycle" = "$zeile" ]]
	1105	then
	1106	(( icycle = 0 ))
	1107	elif [[ "$cycle" = "${extin[$i]}" ]]
	1108	then
	1109	(( icycle = 0 ))
	1110	else
	1111	(( icycle = $cycle ))
	1112	fi
	1113	if (( icycle > maxcycle ))
	1114	then
	1115	(( maxcycle = icycle ))
	1116	file_to_be_used=$zeile
	1117	fi
	1118	done <filelist
	1119	rm filelist
	1120
	1121	# EVTL. ZYKLUSNUMMER AN DATEINAMEN ANFUEGEN
	1122	if (( maxcycle > 0 ))
	1123	then
	1124	if [[ "${extin[$i]}" != " " && "${extin[$i]}" != "" ]]
	1125	then
	1126	filename=${filename}.$maxcycle.${extin[$i]}
	1127	else
	1128	filename=${filename}.$maxcycle
	1129	fi
	1130	else
	1131	if [[ "${extin[$i]}" != " " && "${extin[$i]}" != "" ]]
	1132	then
	1133	filename=${filename}.${extin[$i]}
	1134	fi
	1135	fi
	1136
	1137	# DATEINAMEN OHNE PFAD ABER MIT ZYKLUSNUMMER ABSPEICHERN,
	1138	# DA SPAETER BEI RUECKSPEICHERN VOM ARCHIVSYSTEM EVTL. BENOETIGT
	1139	absnamein[$i]=$filename
	1140	if (( maxcycle > 0 ))
	1141	then
	1142	if [[ "${actionin[$i]}" = di ]]
	1143	then
	1144	frelin[$i]=${endin[$i]}.$maxcycle
	1145	else
	1146	frelin[$i]=${afname}${endin[$i]}.$maxcycle
	1147	fi
	1148	else
	1149	if [[ "${actionin[$i]}" = di ]]
	1150	then
	1151	frelin[$i]=${endin[$i]}
	1152	else
	1153	frelin[$i]=${afname}${endin[$i]}
	1154	fi
	1155	fi
	1156
	1157	fi
	1158	done
	1159
	1160
	1161	# VOLLSTAENDIGE DATEINAMEN (OHNE $ ODER ~) DER OUTPUT-FILES BILDEN,
	1162	# OUTPUT-DATEIEN AUF VORHANDENSEIN PRUEFEN UND EVTL. HOECHSTE ZYKLUSNUMMER
	1163	# ERMITTELN ODER, FALLS NICHT VORHANDEN, PRUEFEN, OB SIE SICH ANLEGEN LASSEN
	1164	# DIESE AKTIONEN FINDEN NICHT STATT, WENN PROGRAMM AUF REMOTE-MASCHINE
	1165	# RECHNET UND DATEI ANSCHLIESSEND TRANSFERIERT WERDEN SOLL!
	1166	(( i = 0 ))
	1167	while (( i < iout ))
	1168	do
	1169	(( i = i + 1 ))
	1170	if [[ ! ( $fromhost != $localhost && ( "${actionout[$i]}" = tr \|\| "${actionout[$i]}" = tra \|\| "${actionout[$i]}" = trpe ) ) ]]
	1171	then
	1172	if [[ "${actionout[$i]}" = tr \|\| "${actionout[$i]}" = trpe ]]
	1173	then
	1174	actionout[$i]=""
	1175	elif [[ "${actionout[$i]}" = tra ]]
	1176	then
	1177	actionout[$i]=a
	1178	fi
	1179	(( maxcycle = 0 ))
	1180	eval filename=${pathout[$i]}/${fname}${endout[$i]}
	1181	eval catalogname=${pathout[$i]}
	1182	if [[ $(ls $filename* 2>&1 \| grep -c "not found") = 1 \|\| \
	1183	$(ls $filename* 2>&1 \| grep -c "No such file") = 1 \|\| \
	1184	$(ls $filename* 2>&1 \| grep -c "does not exist") = 1 ]]
	1185	then
	1186
	1187	# OUTPUT-DATEI NICHT VORHANDEN. PRUEFEN, OB ANLEGEN MOEGLICH.
	1188	if cat /dev/null > $filename
	1189	then
	1190	rm $filename
	1191	else
	1192
	1193	# PRUEFEN, OB KATALOG VORHANDEN UND EVTL. DIESEN ANLEGEN
	1194	if [[ ! -d $catalogname ]]
	1195	then
	1196	if mkdir -p $catalogname
	1197	then
	1198	printf "\n\n *** directory:"
	1199	printf "\n $catalogname"
	1200	printf "\n was created\n"
	1201	else
	1202	printf "\n\n +++ OUTPUT-file:"
	1203	printf "\n $filename"
	1204	printf "\n cannot be created, because directory does not exist"
	1205	printf "\n and cannot be created either"
	1206	printf "\n"
	1207	locat=output ; exit
	1208	fi 2>/dev/null
	1209	else
	1210	printf "\n\n +++ OUTPUT-file:"
	1211	printf "\n $filename"
	1212	printf "\n cannot be created, although directory exists"
	1213	printf "\n"
	1214	locat=output ; exit
	1215	fi
	1216	fi 2>/dev/null
	1217	else
	1218
	1219	# ZYKLUSNUMMER FESTSTELLEN
	1220	ls -1 -d $filename > filelist 2>/dev/null
	1221	ls -1 -d $filename.* >> filelist 2>/dev/null
	1222	while read zeile
	1223	do
	1224	cycle=$(print $zeile \| cut -f2 -d".")
	1225	if [[ "$cycle" = "$zeile" \|\| "$cycle" = ${extout[$i]} ]]
	1226	then
	1227	(( icycle = 1 ))
	1228	else
	1229	(( icycle = $cycle + 1 ))
	1230	fi
	1231	if (( icycle > maxcycle ))
	1232	then
	1233	(( maxcycle = icycle ))
	1234	fi
	1235	done <filelist
	1236	rm filelist
	1237	fi
	1238
	1239	# EVTL. ZYKLUSNUMMER AN DATEINAMEN ANFUEGEN UND PRUEFEN, OB SICH
	1240	# DATEI AUCH WIRKLICH ANLEGEN LAESST. IM APPEND-FALL MUSS AN DEN
	1241	# HOECHSTEN VORHANDENEN ZYKLUS-ANGEHAENGT WERDEN. ANLAGEPRUEFUNG
	1242	# IST DANN NATUERLICH NICHT ZULAESSIG BZW. MOEGLICH
	1243	if [[ "${actionout[$i]}" != a ]]
	1244	then
	1245	if (( maxcycle > 0 ))
	1246	then
	1247	filename=${filename}.$maxcycle
	1248	if cat /dev/null > $filename
	1249	then
	1250	rm $filename
	1251	else
	1252	printf "\n +++ OUTPUT-file:"
	1253	printf "\n $filename"
	1254	printf "\n cannot be created"
	1255	locat=output ; exit
	1256	fi
	1257	fi
	1258	else
	1259	(( maxcycle = maxcycle - 1 ))
	1260	if (( maxcycle > 0 ))
	1261	then
	1262	filename=${filename}.$maxcycle
	1263	fi
	1264	fi
	1265
	1266	# DATEINAMEN OHNE PFAD ABER MIT ZYKLUSNUMMER ABSPEICHERN,
	1267	# DA SPAETER BEI ABLAGE AUF ARCHIVSYSTEM BZW. FUER
	1268	# DATEI OUTPUT_FILE_CONNECTIONS EVTL. BENOETIGT
	1269	pathout[$i]=$filename
	1270	if (( maxcycle > 0 ))
	1271	then
	1272	frelout[$i]=${fname}${endout[$i]}.$maxcycle
	1273	else
	1274	frelout[$i]=${fname}${endout[$i]}
	1275	fi
	1276
	1277	fi
	1278	done
	1279
	1280
	1281	# PRUEFEN, OB ENTWEDER HAUPTPROGRAMM ODER NUR EIN AUSFUEHRBARES
	1282	# PROGRAMM VEREINBART WURDE (IN DIESEM FALL BRAUCHT IM WEITEREN NICHT
	1283	# UEBERSETZT ZU WERDEN)
	1284	if [[ "$mainprog" = "" && "$executable" = "" ]]
	1285	then
	1286	printf "\n +++ neither main program nor executable defined"
	1287	locat=source; exit
	1288	elif [[ "$mainprog" != "" && "$executable" != "" ]]
	1289	then
	1290	printf "\n +++ main program as well as executable defined"
	1291	locat=source; exit
	1292	elif [[ "$mainprog" = "" && "$executable" != "" ]]
	1293	then
	1294	do_compile=false
	1295	fi
	1296
	1297
	1298	# ALLE ZU UEBERSETZENDEN PROGRAMMTEILE ZUSAMMENSAMMELN
	1299	# BEI BATCH-JOBS IST DIES NICHT NOETIG, WEIL DIE PROGRAMMTEILE BEREITS DURCH
	1300	# DEN MRUN-AUFRUF ERMITTELT SIND, DER DEN BATCH-JOB GENERIERT HAT, UND
	1301	# IM VERZEICHNIS SOURCES_FOR_RUN_... ABGELEGT SIND
	1302	if [[ $do_compile = true && "$SOURCES_COMPLETE" = "" ]]
	1303	then
	1304
[5]	1305	[[ "$source_list" = LM ]] && source_list=LOCALLY_MODIFIED
	1306	[[ "$source_list" = WP ]] && source_list=WRITE_PERMIT
[1]	1307
[5]	1308	if [[ $restart_run != true ]]
[1]	1309	then
	1310	rm -rf SOURCES_FOR_RUN_$fname
	1311	mkdir SOURCES_FOR_RUN_$fname
	1312	fi
	1313
	1314
	1315	# EVTL. ZU UEBERSETZENDE PROGRAMMTEILE DADURCH BESTIMMEN, DASS GEPRUEFT
	1316	# WIRD, WELCHE DATEIEN WRITE-PERMIT BESITZEN (DATEIEN SOLLTEN DESHALB
	1317	# UNTER RCS-KONTROLLE STEHEN). MIT DER VARIABLEN EXCLUDE KOENNEN
	1318	# BESTIMMTE DATEIEN GRUNDSAETZLICH VON DER UEBERSETZUNG AUSGESCHLOSSEN
	1319	# WERDEN. DIES IST NUR BEI MANUELLEM START VON MRUN DURCH DEN BENUTZER
	1320	# MOEGLICH, DA BEI AUTOMATISCHEN RESTARTS TEST DURCH DIE TATSAECHLICH
	1321	# ZU UBERSETZENDEN DATEIEN ERSETZT IST.
	1322	# ALLE ERMITTELTEN DATEIEN WERDEN IM VERZEICHNIS SOURCES_FOR_RUN_...
	1323	# GESAMMELT
[5]	1324	if [[ "$source_list" = WRITE_PERMIT ]]
[1]	1325	then
	1326
	1327	source_list=""
	1328	cd $source_path
	1329
	1330
	1331	# ALLE MOEGLICHEN QUELLCODEDATEIEN AUFLISTEN
	1332	Names=$(ls -1 *.f90 2>&1)
	1333	[[ $(echo $Names \| grep -c '*.f90') = 0 ]] && Filenames="$Names"
	1334	Names=$(ls -1 *.F90 2>&1)
	1335	[[ $(echo $Names \| grep -c '*.F90') = 0 ]] && Filenames="$Filenames $Names"
	1336	Names=$(ls -1 *.F 2>&1)
	1337	[[ $(echo $Names \| grep -c '*.F') = 0 ]] && Filenames="$Filenames $Names"
	1338	Names=$(ls -1 *.f 2>&1)
	1339	[[ $(echo $Names \| grep -c '*.f') = 0 ]] && Filenames="$Filenames $Names"
	1340	Names=$(ls -1 *.c 2>&1)
	1341	[[ $(echo $Names \| grep -c '*.c') = 0 ]] && Filenames="$Filenames $Names"
	1342
	1343
	1344	# DATEIEN MIT WRITE-PERMIT NACH SOURCES_FOR_RUN_... KOPIEREN
	1345	for dateiname in $Filenames
	1346	do
	1347	if [[ -w $dateiname ]]
	1348	then
	1349	if [[ "$exclude" = "" \|\| $(echo $exclude \| grep -c $dateiname) = 0 ]]
	1350	then
	1351	cp $dateiname $working_directory/SOURCES_FOR_RUN_$fname
	1352	source_list=$source_list"$dateiname "
	1353	fi
	1354	fi
	1355	done
	1356
	1357	cd - > /dev/null
	1358
	1359
[5]	1360	elif [[ "$source_list" = LOCALLY_MODIFIED ]]
	1361	then
	1362
	1363	# MODIFIZIERTE DATEIEN DER SVN-ARBEITSKOPIE BESTIMMEN
	1364	source_list=""
	1365	cd $source_path
	1366
	1367
	1368	# PRUEFEN, OB VERZEICHNIS UEBERHAUPT UNTER SVN-KONTROLLE STEHT
	1369	if [[ ! -d .svn ]]
	1370	then
	1371	printf "\n\n +++ source directory"
	1372	printf "\n \"$source_path\" "
	1373	printf "\n is not under control of \"subversion\"."
	1374	printf "\n Please do not use mrun-option \"-s LOCALLY MODIFIED\"\n"
	1375	fi
	1376
	1377
	1378	# ALLE MODIFIZIERTEN QUELLCODEDATEIEN AUFLISTEN
	1379	Filenames=""
	1380	svn status \| while read line
	1381	do
	1382	firstc=`echo $line \| cut -c1`
	1383	if [[ $firstc = M \|\| $firstc = "?" ]]
	1384	then
	1385	Name=`echo "$line" \| cut -c8-`
	1386	extension=`echo $Name \| cut -d. -f2`
	1387	if [[ "$extension" = f90 \|\| "$extension" = F90 \|\| "$extension" = f \|\| "$extension" = F \|\| "$extension" = c ]]
	1388	then
	1389	Filenames="$Filenames "$Name
	1390	fi
	1391	fi
	1392	done
	1393
	1394
	1395	# DATEIEN NACH SOURCES_FOR_RUN_... KOPIEREN
	1396	for dateiname in $Filenames
	1397	do
	1398	cp $dateiname $working_directory/SOURCES_FOR_RUN_$fname
	1399	source_list=$source_list"$dateiname "
	1400	done
	1401
	1402	cd - > /dev/null
	1403
	1404
[1]	1405	# MITTELS OPTION -s ANGEGEBENE DATEIEN NACH SOURCES_FOR_RUN_... KOPIEREN
	1406	# BEI AUTOMATISCHEN FORTSETZUNGSLAEUFEN SIND DORT SCHON ALLE DATEIEN
	1407	# VORHANDEN
[5]	1408	elif [[ "$source_list" != "" && $restart_run != true ]]
[1]	1409	then
	1410
	1411	cd $source_path
	1412
	1413	for filename in $source_list
	1414	do
	1415
	1416	# QUELLTEXT-DATEI DARF KEINE PFADE BEINHALTEN
	1417	if [[ $(print $filename \| grep -c "/") != 0 ]]
	1418	then
	1419	printf "\n +++ source code file: $filename"
	1420	printf "\n must not contain (\"/\") "
	1421	locat=source; exit
	1422	fi
	1423
	1424	if [[ ! -f $filename ]]
	1425	then
	1426	printf "\n +++ source code file: $filename"
	1427	printf "\n does not exist"
	1428	locat=source; exit
	1429	else
	1430	cp $filename $working_directory/SOURCES_FOR_RUN_$fname
	1431	fi
	1432
	1433	done
	1434
	1435	cd - > /dev/null
	1436
	1437	fi
	1438
	1439
	1440	# PRUEFEN, OB ENTWEDER HAUPTPROGRAMM VORHANDEN UND ES EVTL. IN DER
	1441	# LISTE DER ZU UEBERSETZENDEN PROGRAMMTEILE MIT ENTHALTEN IST (WENN
	1442	# NICHT, WIRD ES DIESER LISTE HINZUGEFUEGT)
[5]	1443	if [[ $restart_run != true ]]
[1]	1444	then
	1445
	1446	if [[ ! -f "$source_path/$mainprog" ]]
	1447	then
	1448	printf "\n\n +++ main program: $mainprog"
	1449	printf "\n does not exist in source directory"
	1450	printf "\n \"$source_path\"\n"
	1451	locat=source; exit
	1452	else
	1453	if [[ $(echo $source_list \| grep -c $mainprog) = 0 ]]
	1454	then
	1455	cp $source_path/$mainprog SOURCES_FOR_RUN_$fname
	1456	source_list=${mainprog}" $source_list"
	1457	fi
	1458	fi
	1459	fi
	1460
	1461
	1462	# DATEIEN AUS ZUSAETZLICHEM QUELLVERZEICHNIS HINZUFUEGEN
[5]	1463	if [[ $restart_run != true && "$add_source_path" != "" ]]
[1]	1464	then
	1465
	1466	# GIBT ES DAS VERZEICHNIS UEBERHAUPT?
	1467	if [[ ! -d $add_source_path ]]
	1468	then
	1469	printf "\n\n +++ WARNING: additional source code directory"
	1470	printf "\n \"$add_source_path\" "
	1471	printf "\n does not exist or is not a directory."
	1472	printf "\n No source code will be used from this directory!\n"
	1473	add_source_path=""
	1474	sleep 3
	1475	else
	1476
	1477	cd $add_source_path
	1478
	1479	Names=$(ls -1 *.f90 2>&1)
	1480	[[ $(echo $Names \| grep -c '*.f90') = 0 ]] && AddFilenames="$Names"
	1481	Names=$(ls -1 *.F90 2>&1)
	1482	[[ $(echo $Names \| grep -c '*.F90') = 0 ]] && AddFilenames="$AddFilenames $Names"
	1483	Names=$(ls -1 *.F 2>&1)
	1484	[[ $(echo $Names \| grep -c '*.F') = 0 ]] && AddFilenames="$AddFilenames $Names"
	1485	Names=$(ls -1 *.f 2>&1)
	1486	[[ $(echo $Names \| grep -c '*.f') = 0 ]] && AddFilenames="$AddFilenames $Names"
	1487	Names=$(ls -1 *.c 2>&1)
	1488	[[ $(echo $Names \| grep -c '*.c') = 0 ]] && AddFilenames="$AddFilenames $Names"
	1489
	1490	cd - > /dev/null
	1491	cd SOURCES_FOR_RUN_$fname
	1492
	1493	for dateiname in $AddFilenames
	1494	do
	1495	if [[ -f $dateiname ]]
	1496	then
	1497	printf "\n +++ source code file \"$dateiname\" found in additional"
	1498	printf "\n source code directory \"$add_source_path\" "
	1499	printf "\n as well as in directory \"$source_path\"."
	1500	locat=source; exit
	1501	fi
	1502
	1503	cp $add_source_path/$dateiname .
	1504	source_list="$source_list $dateiname"
	1505
	1506	# Default User-Interface von der Liste entfernen, falls Datei
	1507	# ein User-Interface enthaelt
	1508	if [[ $( cat $dateiname \| grep -c "END SUBROUTINE user_parin" ) != 0 ]]
	1509	then
[50]	1510	if [[ $dateiname != user_interface.f90 && -f user_interface.f90 ]]
[1]	1511	then
	1512	rm -rf user_interface.f90
	1513	source_list=`echo $source_list \| sed -e 's/user_interface.f90//'`
[50]	1514	printf "\n\n *** default \"user_interface.f90\" removed from the files to be translated"
[1]	1515	printf "\n since a user-interface is found in file"
	1516	printf "\n \"$add_source_path/$dateiname\" \n"
	1517	sleep 3
	1518	else
	1519	printf "\n\n *** user-interface file \"$dateiname\" "
	1520	printf "\n added to the files to be translated \n"
	1521	sleep 3
	1522	fi
	1523	fi
	1524	done
	1525
	1526	cd - > /dev/null
	1527	fi
	1528	fi
	1529
	1530
	1531	# ALLE UNTERPROGRAMME, DIE ZU VEREINBARTEN SOFTWAREPAKETEN GEHOEREN,
	1532	# DER LISTE DER ZU UEBERSETZENDEN DATEIEN HINZUFUEGEN
[5]	1533	if [[ $restart_run != true && -n $package_list ]]
[1]	1534	then
	1535
	1536	cd $source_path
	1537
	1538	for package in $package_list
	1539	do
	1540
	1541	[[ $package = "dvrp_graphics+1PE" ]] && package=dvrp_graphics
	1542
	1543	# ERMITTELE ALLE DATEIEN, DIE ZUM PAKET GEHOEREN
	1544	# FEHLERMELDUNGEN WERDEN ABGEFANGEN, DA * AUCH VERZEICHNISSNAMEN
	1545	# LIEFERT
	1546	package_source_list=`grep "defined( __$package " * 2>/dev/null \| cut -f1 -d:`
	1547
	1548
	1549	# FUEGE DIESE DATEIEN DER LISTE DER ZU UEBERSETZENDEN DATEIEN
	1550	# HINZU, FALLS SIE NOCH NICHT DAZUGEHOEREN
	1551	for source_list_name in $package_source_list
	1552	do
	1553	if [[ $(echo $source_list \| grep -c $source_list_name) = 0 ]]
	1554	then
	1555
	1556	# NUR DATEIEN MIT GUELTIGEN ENDUNGEN VERWENDEN
	1557	ending=`echo $source_list_name \| cut -f2 -d.`
	1558	if [[ "$ending" = f90 \|\| "$ending" = F90 \|\| "$ending" = f \|\| "$ending" = F \|\| "$ending" = c ]]
	1559	then
	1560	cp $source_list_name $working_directory/SOURCES_FOR_RUN_$fname
	1561	source_list="$source_list $source_list_name"
	1562	fi
	1563	fi
	1564	done
	1565	done
	1566
	1567	cd - > /dev/null
	1568	fi
	1569
	1570
	1571	# MAKEFILE AUF VORHANDENSEIN PRUEFEN UND KOPIEREN
	1572	# BEI RESTART-LAEUFEN LIEGT ES SCHON IM VERZEICHNIS SOURCES_FOR_RUN...
[5]	1573	if [[ "$use_makefile" = true && "$restart_run" != true ]]
[1]	1574	then
	1575	[[ "$makefile" = "" ]] && makefile=$source_path/Makefile
	1576	if [[ ! -f $makefile ]]
	1577	then
	1578	printf "\n +++ file \"$makefile\" does not exist"
	1579	printf "\n provide this file or set \"use_makefile=false\" in configuration file\n"
	1580	locat=make; exit
	1581	else
	1582	cp $makefile SOURCES_FOR_RUN_$fname/Makefile
	1583	fi
	1584	fi
	1585
	1586	fi # do_compile=true
	1587
	1588
	1589	# FALLS PROGRAMMTEILE UEBERSETZT WERDEN SOLLEN, FOLGEN JETZT EINIGE
	1590	# UEBERPRUEFUNGEN UND DAS SETZEN DER PRAEPROZESSOR-DIREKTIVEN
	1591	if [[ $do_compile = true ]]
	1592	then
	1593
	1594	# PRAEPROZESSOR-DIREKTIVEN ZUM SELEKTIVEN AUSWAEHLEN VON CODETEILEN
	1595	# ZUSAMMENSETZEN
	1596	if [[ $(echo $localhost \| cut -c1-3) = ibm ]]
	1597	then
	1598	PPOPTS=-Dibm=ibm
	1599	elif [[ $(echo $localhost \| cut -c1-3) = nec ]]
	1600	then
	1601	PPOPTS=-Dnec
	1602	else
	1603	PPOPTS=-D$localhost
	1604	fi
	1605	if [[ $(echo $localhost \| cut -c1-3) = ibm ]]
	1606	then
	1607	[[ -n $cond1 ]] && PPOPTS="$PPOPTS -D$cond1=$cond1"
	1608	[[ -n $cond2 ]] && PPOPTS="$PPOPTS -D$cond2=$cond2"
	1609	if [[ $netcdf_support = true ]]
	1610	then
	1611	PPOPTS="$PPOPTS -Dnetcdf=netcdf"
	1612	[[ $localhost != ibms ]] && PPOPTS="$PPOPTS -Dnetcdf_64bit=netcdf_64bit"
	1613	fi
	1614	if [[ $localhost = ibmy ]]
	1615	then
	1616	PPOPTS="$PPOPTS -Dibmy_special=ibmy_special"
	1617	fi
	1618	else
	1619	[[ -n $cond1 ]] && PPOPTS="$PPOPTS -D$cond1"
	1620	[[ -n $cond2 ]] && PPOPTS="$PPOPTS -D$cond2"
	1621	if [[ $netcdf_support = true ]]
	1622	then
	1623	PPOPTS="$PPOPTS -Dnetcdf"
	1624	[[ $localhost != decalpha && $localhost != lctit ]] && PPOPTS="$PPOPTS -Dnetcdf_64bit"
	1625	fi
	1626	fi
	1627	if [[ -n $package_list ]]
	1628	then
	1629	for package in $package_list
	1630	do
	1631	if [[ $(echo $localhost \| cut -c1-3) = ibm ]]
	1632	then
	1633	if [[ $package != "dvrp_graphics+1PE" ]]
	1634	then
	1635	PPOPTS="$PPOPTS -D$package=$package"
	1636	else
	1637	PPOPTS="$PPOPTS -Ddvrp_graphics"
	1638	export use_seperate_pe_for_dvrp_output=true
	1639	fi
	1640	else
	1641	if [[ $package != "dvrp_graphics+1PE" ]]
	1642	then
	1643	PPOPTS="$PPOPTS -D$package"
	1644	else
	1645	PPOPTS="$PPOPTS -Ddvrp_graphics"
	1646	export use_seperate_pe_for_dvrp_output=true
	1647	fi
	1648	fi
	1649	done
	1650	fi
	1651	if [[ -n $cpp_opts ]]
	1652	then
	1653	for popts in $cpp_opts
	1654	do
	1655	PPOPTS="$PPOPTS -D$popts"
	1656	done
	1657	fi
	1658
	1659	# -D ARGUMENTEN EINEN DOPPELTEN UNTERSTRICH VORANSTELLEN
	1660	PPOPTS=`echo " $PPOPTS" \| sed 's/ -D/ -D__/g'`
	1661
	1662	# OPTION ZUM EINSCHALTEN DES PRAEPROZESSORS HINZUFUEGEN
	1663	if [[ $host = decalpha ]]
	1664	then
	1665	C_PPOPTS="-cpp -DMPI_REAL=MPI_DOUBLE_PRECISION -DMPI_2REAL=MPI_2DOUBLE_PRECISION $PPOPTS"
	1666	elif [[ $(echo $host \| cut -c1-3) = ibm ]]
	1667	then
	1668	# blanks in PPOPTS muessen durch "," ersetzt werden
	1669	PPOPTS=`echo $PPOPTS \| sed 's/ -/,-/g'`
	1670	C_PPOPTS="-qsuffix=cpp=f90 -WF,-DMPI_REAL=MPI_DOUBLE_PRECISION,$PPOPTS"
	1671	elif [[ $host = lcmuk ]]
	1672	then
	1673	C_PPOPTS="$PPOPTS -DMPI_REAL=MPI_DOUBLE_PRECISION -DMPI_2REAL=MPI_2DOUBLE_PRECISION"
	1674	elif [[ $host = lctit ]]
	1675	then
	1676	C_PPOPTS="$PPOPTS -Mpreprocess -DMPI_REAL=MPI_DOUBLE_PRECISION -DMPI_2REAL=MPI_2DOUBLE_PRECISION"
	1677	elif [[ $(echo $host \| cut -c1-3) = nec ]]
	1678	then
	1679	C_PPOPTS="-Ep -DMPI_REAL=MPI_REAL8 -DMPI_2REAL=MPI_2REAL8 -DSCFFT=dzfft -DCSFFT=zdfft $PPOPTS"
	1680	else
	1681	C_PPOPTS="$PPOPTS"
	1682	fi
	1683
	1684	else
	1685
	1686
	1687	# BEI LOKALEN RECHNUNGEN PRUEFEN, OB EXECUTABLE VORHANDEN
	1688	if [[ $do_remote = false ]]
	1689	then
	1690	if [[ ! -f $executable ]]
	1691	then
	1692	printf "\n +++ executable file: $executable"
	1693	printf "\n does not exist"
	1694	locat=executable; exit
	1695	fi
	1696	fi
	1697	fi
	1698
	1699
	1700	# JOBMODUS FESTSTELLEN
	1701	if [[ "$ENVIRONMENT" = BATCH && $localhost != lctit \|\| "$QUEUE" != interactive && $localhost = lctit ]]
	1702	then
	1703	jobmo=BATCH
	1704	else
	1705	jobmo=INTERACTIVE
	1706	fi
	1707
	1708
	1709	# HOSTSPEZIFISCHE DEFAULT-COMPILER SETZEN, FALLS NICHT BEREITS
	1710	# DURCH BENUTZER ANDERWEITIG VEREINBART
	1711	if [[ "$compiler_name" = "" ]]
	1712	then
	1713	case $localhost in
[5]	1714	(unics) compiler_name=f77;;
[1]	1715	(lcmuk) if [[ "$cond1" = parallel \|\| "$cond2" = parallel ]]
	1716	then
	1717	compiler_name=mpif90
	1718	else
	1719	compiler_name=ifc
	1720	fi;;
	1721	(lctit) if [[ "$cond1" = parallel \|\| "$cond2" = parallel ]]
	1722	then
	1723	compiler_name=mpif90
	1724	echo ">>> compiler=mpif90"
	1725	else
	1726	compiler_name=pgf95
	1727	echo ">>> compiler=pgf95"
	1728	fi;;
	1729	(nech) compiler_name=sxmpif90;;
	1730	(decalpha) compiler_name=f95;;
	1731	(ibm\|ibmb\|ibmh\|ibms\|ibmy) compiler_name=mpxlf95_r;;
	1732	(*) printf "\n +++ unknown host name: $localhost \n";
	1733	locat=localhost; exit;;
	1734	esac
	1735	fi
	1736
	1737
	1738	# COMPILER AUF NECK UEBERSCHREIBEN
	1739	[[ $localhost = neck ]] && compiler_name=mpif90
	1740
	1741
	1742
	1743	# PRAEPROZESSOR-PFAD BESTIMMEN
	1744	case $localhost in
	1745	(decalpha\|ibm\|ibmb\|ibmh\|ibms\|ibmy\|unics) praeproc=cpp;;
	1746	esac
	1747
	1748
	1749	# TEMPORAEREN KATALOGNAMEN BESTIMMEN
	1750	kennung=$RANDOM
	1751	if [[ "$tmp_user_catalog" = "" ]]
	1752	then
	1753	if [[ $localhost = ibmb \|\| $localhost = ibmh ]]
	1754	then
	1755	tmp_user_catalog=$TMPDIR
	1756	elif [[ $localhost = nech ]]
	1757	then
	1758	tmp_user_catalog=$WRKSHR
	1759	else
	1760	tmp_user_catalog=/tmp
	1761	fi
	1762	fi
	1763	TEMPDIR=$tmp_user_catalog/${usern}.$kennung
	1764
	1765
	1766	# KATALOGNAMEN FUER ZWISCHENSPEICHERUNG VON FORTSETZUNGSLAUFDATEIEN
	1767	# BESTIMMEN
	1768	if [[ "$tmp_data_catalog" = "" ]]
	1769	then
	1770	if [[ $localhost = ibmb \|\| $localhost = ibmh ]]
	1771	then
	1772	tmp_data_catalog=$WORK/mrun_restart_data
	1773	elif [[ $localhost = nech ]]
	1774	then
	1775	tmp_data_catalog=$WRKSHR/mrun_restart_data
	1776	else
	1777	tmp_data_catalog=/tmp/mrun_restart_data
	1778	fi
	1779	fi
	1780
	1781
	1782	# EVENTUELL BEI LOKALEN RECHNUNGEN $-ZEICHEN IN ENVIRONMENT-VARIABLEN
	1783	# ERSETZEN
	1784	if [[ $do_remote = false && $do_compile = true ]]
	1785	then
	1786	eval fopts=\"$fopts\"
	1787	eval lopts=\"$lopts\"
	1788	fi
	1789
	1790
	1791
	1792	# COMPILE- UND LINK-OPTIONEN BESTIMMEN
	1793	FOPTS="$fopts"
	1794	LOPTS="$lopts"
	1795	ROPTS="$ropts"
	1796	if [[ ( $(echo $host \| cut -c1-3) = nec \|\| $(echo $host \| cut -c1-3) = ibm \|\| $host = lctit ) && -n $numprocs ]]
	1797	then
	1798	XOPT="-X $numprocs"
	1799	fi
	1800
	1801
	1802	# BEI VERWENDUNG VON DVRP-SOFTWARE MUSS ENTSPECHENDE GRAFIK-BIBLIOTHEK
	1803	# MIT EINGEBUNDEN WERDEN
	1804	if [[ -n $package_list ]]
	1805	then
	1806	if [[ $(echo $package_list \| grep -c dvrp_graphics) != 0 ]]
	1807	then
	1808
[14]	1809	# DVRP-GRAFIK GIBT ES NUR AUF DER IBM AM HLRN, AUF NECH/NECK UND
[1]	1810	# DEM NOTEBOOK SCIROCCO
[14]	1811	if [[ $host != ibmh && $host != nech && $host != neck && $host != lcmuk && "$local_compile" != true ]]
[1]	1812	then
	1813	printf "\n"
	1814	printf "\n +++ option -p with package \"dvrp_graphics\" only allowed on nech and ibmh"
	1815	locat=parameter; exit
	1816	fi
	1817	if [[ $host = ibmh ]]
	1818	then
	1819	if [[ $cond1 = debug \|\| $cond2 = debug ]]
	1820	then
	1821	FOPTS="-I/home/h/hzazchmi/lib/debug $FOPTS"
	1822	LOPTS="$LOPTS -L/home/h/hzazchmi/lib/debug -lDVRP2 -lftp"
	1823	else
	1824	FOPTS="-I/home/h/nikevita/lib_stable/optimize $FOPTS"
	1825	LOPTS="$LOPTS -L/home/h/nikevita/lib_stable/optimize -lDVRP2 -lftp"
	1826	fi
	1827	elif [[ $host = nech ]]
	1828	then
[14]	1829	FOPTS="-I/import/ds9b/ipf2/b/b323014/lib/optimize $FOPTS"
	1830	LOPTS="$LOPTS -L/import/ds9b/ipf2/b/b323014/lib/optimize -lDVRP2 -lftp"
	1831	elif [[ $host = neck ]]
	1832	then
	1833	FOPTS="-I/home/DSRC/NC/tatuyama/pub/dvrp $FOPTS"
	1834	LOPTS="$LOPTS -L/home/DSRC/NC/tatuyama/pub/dvrp -lDVRP2 -lftp"
[1]	1835	elif [[ $host = lcmuk ]]
	1836	then
	1837	FOPTS="-I/usr/local/lib/dvrp/optimize $FOPTS"
	1838	LOPTS="$LOPTS -L/usr/local/lib/dvrp/optimize -lDVRP2 -lftp"
	1839	fi
	1840	fi
	1841	fi
	1842
	1843
	1844	# EVTL. EINBINDEN DER NETCDF-BIBLIOTHEK
	1845	if [[ $netcdf_support = true ]]
	1846	then
	1847	case $host in
	1848	(decalpha) FOPTS="-I${netcdf_path}/include $FOPTS"
	1849	LOPTS="$LOPTS -L${netcdf_path}/lib -lnetcdf";;
	1850	(ibmb\|ibmh) FOPTS="-I ${netcdf_path}/include-64 $FOPTS"
	1851	LOPTS="$LOPTS -L${netcdf_path}/lib -lnetcdf";;
	1852	(ibms) FOPTS="-I /applic/netcdf64/src/f90 $FOPTS"
	1853	LOPTS="$LOPTS -L${netcdf_path} -lnetcdf";;
	1854	(ibmy) FOPTS="-I ${netcdf_path}/include $FOPTS"
	1855	LOPTS="$LOPTS -L${netcdf_path}/lib -lnetcdf";;
	1856	(lcmuk\|nech\|neck) FOPTS="-I ${netcdf_path}/include $FOPTS"
	1857	LOPTS="$LOPTS -L${netcdf_path}/lib -lnetcdf";;
	1858	(lctit) FOPTS="-I ${netcdf_path}/src/f90 $FOPTS"
	1859	LOPTS="$LOPTS -L${netcdf_path}/src/libsrc -lnetcdf";;
	1860
	1861	esac
	1862	fi
	1863
	1864	# PRUEFEN DER CPU-ZEIT. (CPUMAX WIRD ALS ENV-VARIABLE VOM HAUTPRO-
	1865	# GRAMM BENOETIGT
	1866	done=false
	1867	while [[ $done = false ]]
	1868	do
	1869	cputime=$cpumax
	1870	if (( $cputime == 0 ))
	1871	then
	1872	printf "\n +++ cpu-time is undefined"
	1873	printf "\n >>> Please type CPU-time in seconds as INTEGER:"
	1874	printf "\n >>> "
	1875	read cputime 1>/dev/null 2>&1
	1876	else
	1877	done=true
	1878	fi
	1879	cpumax=$cputime
	1880	done
	1881
	1882	(( minuten = cputime / 60 ))
	1883	(( sekunden = cputime - minuten * 60 ))
	1884
	1885
	1886	# PRUEFEN DER KERNSPEICHERANFORDERUNG
	1887	if [[ $do_batch = true ]]
	1888	then
	1889	done=false
	1890	while [[ $done = false ]]
	1891	do
	1892	if (( memory == 0 ))
	1893	then
	1894	printf "\n +++ memory demand is undefined"
	1895	printf "\n >>> Please type memory in MByte per process as INTEGER:"
	1896	printf "\n >>> "
	1897	read memory 1>/dev/null 2>&1
	1898	else
	1899	done=true
	1900	fi
	1901	done
	1902	fi
	1903
	1904
	1905	# PRUEFEN, OB FUER REMOTE-RECHNUNGEN EIN BENUTZERNAME ANGEGEBEN WURDE
	1906	if [[ $do_remote = true && -z $remote_username ]]
	1907	then
	1908	while [[ -z $remote_username ]]
	1909	do
	1910	printf "\n +++ username on remote host \"$host\" is undefined"
	1911	printf "\n >>> Please type username:"
	1912	printf "\n >>> "
	1913	read remote_username
	1914	done
	1915	mc="$mc -u$remote_username"
	1916	fi
	1917
	1918	###########################################################################
	1919	# HEADER-AUSGABE
	1920	###########################################################################
	1921
	1922
	1923	calltime=$(date)
	1924	printf "\n"
	1925	# [[ $silent = false ]] && clear
	1926	printf "#--------------------------------------------------------------# \n"
	1927	printf "\| $version$calltime \| \n"
	1928	printf "\| \| \n"
	1929	spalte1="called on:"; spalte2=$localhost_realname
	1930	printf "\| $spalte1$spalte2 \| \n"
	1931	if [[ $local_compile = false ]]
	1932	then
	1933	if [[ $do_remote = true ]]
	1934	then
	1935	spalte1="execution on:"; spalte2="$host (username: $remote_username)"
	1936	else
	1937	spalte1="execution on:"; spalte2="$host ($localhost_realname)"
	1938	fi
	1939	else
	1940	spalte1="compiling test only!"; spalte2=""
	1941	fi
	1942	printf "\| $spalte1$spalte2 \| \n"
	1943	if [[ -n $numprocs ]]
	1944	then
	1945	spalte1="number of PEs:"; spalte2=$numprocs
	1946	printf "\| $spalte1$spalte2 \| \n"
	1947	fi
	1948	if [[ -n $tasks_per_node ]]
	1949	then
	1950	spalte1="tasks per node:"; spalte2="$tasks_per_node (number of nodes: $nodes)"
	1951	printf "\| $spalte1$spalte2 \| \n"
	1952	fi
	1953	if [[ $threads_per_task != 1 ]]
	1954	then
	1955	spalte1="threads per task:"; spalte2="$threads_per_task"
	1956	printf "\| $spalte1$spalte2 \| \n"
	1957	fi
	1958	printf "\| \| \n"
	1959	if [[ $do_compile = true ]]
	1960	then
	1961	spalte1=cpp-directives:; spalte2=$C_PPOPTS
	1962	printf "\| $spalte1$spalte2 \| \n"
	1963	zeile=$(echo "$C_PPOPTS" \| cut -c41-)
	1964	while [[ "$zeile" != "" ]]
	1965	do
	1966	spalte1=""
	1967	spalte2=$zeile
	1968	printf "\| $spalte1$spalte2 \| \n"
	1969	zeile=$(echo "$zeile" \| cut -c41-)
	1970	done
	1971
	1972	spalte1=compiler-options:; spalte2="$FOPTS $MODULE_OPTIONS"
	1973	printf "\| $spalte1$spalte2 \| \n"
	1974	zeile=$(echo "$FOPTS $MODULE_OPTIONS" \| cut -c41-)
	1975	while [[ "$zeile" != "" ]]
	1976	do
	1977	spalte1=""
	1978	spalte2=$zeile
	1979	printf "\| $spalte1$spalte2 \| \n"
	1980	zeile=$(echo "$zeile" \| cut -c41-)
	1981	done
	1982
	1983	spalte1=linker-options:; spalte2=$LOPTS
	1984	printf "\| $spalte1$spalte2 \| \n"
	1985	zeile=$(echo "$LOPTS" \| cut -c41-)
	1986	while [[ "$zeile" != "" ]]
	1987	do
	1988	spalte1=""
	1989	spalte2=$zeile
	1990	printf "\| $spalte1$spalte2 \| \n"
	1991	zeile=$(echo "$zeile" \| cut -c41-)
	1992	done
	1993
	1994	spalte1="main program":; spalte2=$mainprog
	1995	printf "\| $spalte1$spalte2 \| \n"
	1996	if [[ "$use_makefile" = true ]]
	1997	then
	1998	spalte1="using \"make\" for com"; spalte2="piling and linking"
	1999	printf "\| $spalte1$spalte2 \| \n"
	2000	fi
	2001	# if [[ $vtk_graphics = true ]]
	2002	# then
	2003	# spalte1=""; spalte2="wird in C++ - Programm eingebunden (vtk)"
	2004	# printf "\| $spalte1$spalte2 \| \n"
	2005	# fi
	2006	else
	2007	spalte1=executable:; spalte2=$executable
	2008	printf "\| $spalte1$spalte2 \| \n"
	2009	fi
	2010	printf "\| \| \n"
	2011	spalte1="base name of files":; spalte2=$fname
	2012	printf "\| $spalte1$spalte2 \| \n"
	2013	if [[ $fname != $afname ]]
	2014	then
	2015	spalte1="base name of input files":; spalte2=$afname
	2016	printf "\| $spalte1$spalte2 \| \n"
	2017	fi
	2018	spalte1="INPUT control list":; spalte2=$input_list
	2019	printf "\| $spalte1$spalte2 \| \n"
	2020	spalte1="OUTPUT control list":; spalte2=$output_list
	2021	printf "\| $spalte1$spalte2 \| \n"
	2022
	2023	if [[ $do_batch = true \|\| "$LOADLBATCH" = yes ]]
	2024	then
	2025	spalte1="memory demand / PE":; spalte2="$memory MB"
	2026	printf "\| $spalte1$spalte2 \| \n"
	2027	spalte1=CPU-time:; spalte2="$minuten:$sekunden"
	2028	printf "\| $spalte1$spalte2 \| \n"
	2029	fi
	2030
	2031	if [[ $do_compile = true ]]
	2032	then
	2033	printf "\| \| \n"
	2034	printf "\| Files to be compiled: \| \n"
	2035	zeile=$source_list
	2036	while [[ "$zeile" != "" ]]
	2037	do
	2038	spalte3=$zeile
	2039	printf "\| $spalte3 \| \n"
	2040	zeile=$(echo "$zeile" \| cut -c61-)
	2041	done
	2042	fi
	2043	printf "#--------------------------------------------------------------#"
	2044
	2045
	2046
	2047	# BEDINGTE AUSGABE DER DATEIVERBINDUNGEN
	2048	if [[ $do_trace = true ]]
	2049	then
	2050	(( i = 0 ))
	2051	while (( i < iin ))
	2052	do
	2053	(( i = i + 1 ))
	2054	if (( i == 1 ))
	2055	then
	2056	printf "\n\n >>> INPUT-file assignments:\n"
	2057	fi
	2058	printf "\n ${localin[$i]} : ${absnamein[$i]}"
	2059	done
	2060	(( i = 0 ))
	2061	while (( i < iout ))
	2062	do
	2063	(( i = i + 1 ))
	2064	if (( i == 1 ))
	2065	then
	2066	printf "\n\n >>> OUTPUT-file assignments:\n"
	2067	fi
	2068	printf "\n ${localout[$i]} : ${pathout[$i]}"
	2069	done
	2070	(( i = 0 ))
	2071	while (( i < iic ))
	2072	do
	2073	(( i = i + 1 ))
	2074	if (( i == 1 ))
	2075	then
	2076	printf "\n\n >>> INPUT-commands:\n"
	2077	fi
	2078	printf "\n ${in_command[$i]}"
	2079	done
	2080	(( i = 0 ))
	2081	while (( i < ioc ))
	2082	do
	2083	(( i = i + 1 ))
	2084	if (( i == 1 ))
	2085	then
	2086	printf "\n\n >>> OUTPUT-commands:\n"
	2087	fi
	2088	printf "\n ${out_command[$i]}"
	2089	done
	2090	fi
	2091
	2092
	2093	# ABFRAGEN BEI AUFRUF AUF LOKALER MASCHINE
	2094	if [[ $remotecall = false && $silent = false && $jobmo != BATCH ]]
	2095	then
	2096	antwort=dummy
	2097	printf "\n\n"
	2098	while [[ "$antwort" != y && "$antwort" != Y && "$antwort" != n && "$antwort" != N ]]
	2099	do
	2100	printf " >>> everything o.k. (y/n) ? "
	2101	read antwort
	2102	done
	2103	if [[ $antwort = n \|\| $antwort = N ]]
	2104	then
	2105	locat=user_abort; (( iec = 0 )); exit
	2106	fi
	2107	if [[ $do_batch = true ]]
	2108	then
	2109	printf " >>> batch-job will be created and submitted"
	2110	else
	2111	if [[ $local_compile = false ]]
	2112	then
	2113	printf " >>> MRUN will now continue to execute on this machine"
	2114	else
	2115	printf " >>> a test compilation will now be carried out on this machine"
	2116	fi
	2117	fi
	2118	fi
	2119
	2120
	2121
	2122
	2123	# FALLS AUF DIESER MASCHINE GERECHNET WERDEN SOLL, WERDEN JETZT ENTSPRE-
	2124	# CHENDE AKTIONEN DURCHGEFUEHRT
	2125	if [[ $do_batch = false ]]
	2126	then
	2127
	2128
	2129	# TEMPORAEREN KATALOG ERZEUGEN
	2130	mkdir -p $TEMPDIR
	2131	chmod go+rx $TEMPDIR
	2132	tmpcreate=true
	2133
	2134
	2135	# SAEMTLICHE QUELLTEXT-DATEIEN BZW. AUSFUEHRBARES PROGRAMM IN
	2136	# TEMPORAERES VERZEICHNIS KOPIEREN
	2137	if [[ $do_compile = true ]]
	2138	then
	2139
	2140	# ON NEC, COMPILATION IS DONE ON HOST CROSS VIA CROSS COMPILING
	2141	# CREATE A TEMPORARY DIRECTORY ON THAT MACHINE (HOME MOUNTED VIA NFS)
	2142	if [[ $localhost = nech ]]
	2143	then
	2144	TEMPDIR_COMPILE=$HOME/work/${usern}.$kennung
	2145	if mkdir -p $TEMPDIR_COMPILE
	2146	then
	2147	printf "\n *** \"$TEMPDIR_COMPILE\" "
	2148	printf "\n is generated as temporary directory for cross compiling\n"
	2149	else
	2150	printf "\n +++ creating directory \"$TEMPDIR_COMPILE\" "
	2151	printf "\n needed for cross compilation failed"
	2152	locat=compile
	2153	exit
	2154	fi
	2155	else
	2156	TEMPDIR_COMPILE=$TEMPDIR
	2157	fi
	2158
	2159	if [[ "$use_makefile" = true ]]
	2160	then
	2161
	2162	# PFADNAMEN FUER DAS MAKE-DEPOSITORY ERMITTELN
	2163	line=""
	2164	grep "%depository_path" $config_file \| while read line
	2165	do
	2166	if [[ "$line" != "" && $(echo $line \| cut -c1) != "#" ]]
	2167	then
	2168	if [[ "$(echo $line \| cut -d" " -s -f3)" = "" ]]
	2169	then
	2170	global_depository_path=`echo $line \| cut -d" " -s -f2`
	2171	fi
	2172	fi
	2173	done
	2174
	2175	line=""
	2176	grep " $localhost" $config_file \| grep "%depository_path" \| while read line
	2177	do
	2178	if [[ "$line" != "" && $(echo $line \| cut -c1) != "#" ]]
	2179	then
	2180	local_depository_path=`echo $line \| cut -d" " -s -f2`
	2181	fi
	2182	done
	2183
	2184	if [[ "$local_depository_path" = "" ]]
	2185	then
	2186	if [[ "$global_depository_path" != "" ]]
	2187	then
	2188	local_depository_path=$global_depository_path
	2189	else
	2190	printf "\n\n +++ no depository path found in configuration file"
	2191	printf "\n for local host \"$localhost\" "
	2192	printf "\n please set \"\%depository_path\" in configuration file\n"
	2193	locat=config_file; exit
	2194	fi
	2195	fi
	2196	eval local_depository_path=$local_depository_path
	2197
	2198	basename=`print $mainprog \| cut -f1 -d"."`
	2199	eval make_depository=${local_depository_path}/${basename}_current_version.tar
	2200	if [[ ! -f $make_depository ]]
	2201	then
	2202	printf "\n"
	2203	printf "\n *** WARNING: make depository \"$make_depository\" not found"
	2204	printf "\n \"make\" will fail, if the Makefile or other source files are missing\n"
	2205	else
	2206	cp $make_depository $TEMPDIR_COMPILE
	2207	cd $TEMPDIR_COMPILE
[5]	2208	tar -xf $make_depository > /dev/null 2>&1
[1]	2209	cd - > /dev/null
	2210	fi
	2211
	2212	cp SOURCES_FOR_RUN_$fname/* $TEMPDIR_COMPILE
	2213
	2214	# FALLS USER-INTERFACE VORHANDEN UND AUF DATEI MIT ANDEREM NAMEN
	2215	# ALS user_interface.f90 LIEGT, DIESEN GEAENDERTEN NAMEN INS
	2216	# MAKEFILE EINTRAGEN
	2217	cd $TEMPDIR_COMPILE
	2218	interface_file=`grep -l "END SUBROUTINE user_parin" $source_list`
	2219	if [[ "$interface_file" != "" ]]
	2220	then
	2221	interface_file=`echo $interface_file \| cut -d"." -f1`
	2222	mv Makefile Makefile_old
	2223	sed "s/user_interface/$interface_file/g" Makefile_old > Makefile
	2224	fi
	2225	cd - > /dev/null
	2226	fi
	2227	else
	2228	cp $executable ${TEMPDIR}/a.out
	2229	fi
	2230
	2231
	2232	# WECHSEL IN TEMPORAEREN KATALOG
	2233	cd $TEMPDIR
	2234	printf "\n *** changed to temporary directory: $TEMPDIR"
	2235
	2236
	2237	# OUTPUT-DATEI-VERBINDUNGEN AUF TEMPORAERER DATEI ABLEGEN
	2238	# DIESE DATEI KANN VON SPAETER AUFZURUFENDEN BENUTZERPROZEDUREN GELESEN
	2239	# WERDEN, UM ZU LOKALEN DATEINAMEN GEHOERENDE PERMANENTE NAMEN ZU
	2240	# ERMITTELN
	2241	(( i = 0 ))
	2242	while (( i < iout ))
	2243	do
	2244	(( i = i + 1 ))
	2245	if [[ "${actionout[$i]}" = tr \|\| "${actionout[$i]}" = tra \|\| "${actionout[$i]}" = trpe ]]
	2246	then
	2247	printf "${localout[$i]} ${actionout[$i]}\n${pathout[$i]}\n${localhost}_${fname}${endout[$i]}\n" >> OUTPUT_FILE_CONNECTIONS
	2248	else
	2249	printf "${localout[$i]} ${actionout[$i]}\n${pathout[$i]}\n${frelout[$i]}\n" >> OUTPUT_FILE_CONNECTIONS
	2250	fi
	2251	done
	2252
	2253
	2254	# EVTL. UEBERSETZUNGSAKTIONEN STARTEN
	2255	if [[ $do_compile = true ]]
	2256	then
	2257
	2258
	2259	# COMPILING WITH MAKE (ON NEC COMPILER IS CALLED ON HOST CROSS)
	2260	if [[ "$use_makefile" = true ]]
	2261	then
	2262	printf "\n\n\n *** compilation starts \n$striche\n"
	2263	printf " *** compilation with make using following options:\n"
	2264	printf " compilername: $compiler_name\n"
	2265	printf " options: $FOPTS\n"
	2266	printf " preprocessor-directives: $C_PPOPTS \n"
	2267	printf " linker-options: $LOPTS \n"
	2268	printf " source code files: $source_list \n"
	2269
	2270	if [[ $localhost = nech ]]
	2271	then
	2272	ssh 136.172.44.192 -l $usern ". /SX/opt/etc/initsx.sh; cd \$HOME/work/${usern}.$kennung; sxmake -f Makefile PROG=a.out F90=$compiler_name COPT=\"$C_PPOPTS\" F90FLAGS=\"$FOPTS\" LDFLAGS=\"$LOPTS\" "
	2273	cp $TEMPDIR_COMPILE/a.out .
	2274	[[ $? != 0 ]] && compile_error=true
	2275	rm -rf $TEMPDIR_COMPILE
	2276	else
	2277	make -f Makefile PROG=a.out F90=$compiler_name COPT="$C_PPOPTS" F90FLAGS="$FOPTS" LDFLAGS="$LOPTS"
	2278	fi
	2279
	2280	# UEBERSETZUNG "VON HAND" (OHNE MAKE)
	2281	else
	2282
	2283
	2284	# UEBERSETZUNG DER QUELLTEXT-DATEIEN UND DES HAUPTPROGRAMMS
	2285	# $CPPOPTS FUER CPP-PRAEPROZESSOR WURDEN OBEN BESTIMMT
	2286	printf "\n\n\n *** compilation starts \n$striche\n"
	2287
	2288
	2289	# VORWEG EVTL. UEBERSETZUNG VON PROGRAMMTEILEN MIT SPEZIELLEN
	2290	# COMPILERN UND/ODER COMPILEROPTIONEN. DIESE WERDEN BEI DER SPAETEREN
	2291	# EIGENTLICHEN UEBERSETZUNG ALS OBJECT-FILES EINGEBUNDEN
	2292	new_source_list=""
	2293	for filename in $source_list
	2294	do
	2295
	2296	basename=`print $filename \| cut -f1 -d"."`
	2297
	2298	# EVTL. SPEZIELLEN COMPILER FUER UP BESTIMMEN
	2299	comp_var=$(eval echo \$comp_$basename) # \$ liefert evtl. fuer source alternativ vereinbarten Compiler
	2300	comp_opt_var=$(eval echo \$$basename) # \$$ liefert evtl. fuer source vereinbarte Compiler-Environment-Variable
	2301
	2302	if [[ "$comp_var" = "" && "$comp_opt_var" = "" ]]
	2303	then
	2304	new_source_list="$new_source_list "$filename
	2305	else
	2306
	2307	if [[ "$comp_var" = "" ]]
	2308	then
	2309	SPECIAL_COMPILER=$compiler_name
	2310	else
	2311	SPECIAL_COMPILER=$comp_var
	2312	fi
	2313
	2314	if [[ "$comp_opt_var" = "" ]]
	2315	then
	2316	SPECIAL_FOPTS=$FOPTS
	2317	else
	2318	SPECIAL_FOPTS="$parallel_options $(eval echo \$$comp_opt_var)"
	2319	fi
	2320
	2321	printf " *** compiling of $filename with special compiler call: $SPECIAL_COMPILER $CPPOPTS -c $SPECIAL_FOPTS \n"
	2322	$SPECIAL_COMPILER $C_PPOPTS -c $SPECIAL_FOPTS $MODULE_OPTIONS $filename
	2323	if [[ $? != 0 ]]
	2324	then
	2325	printf "\n +++ compiling error occured"
	2326	locat=compile
	2327	exit
	2328	fi
	2329	new_source_list="$new_source_list "${basename}.o
	2330	fi
	2331	done
	2332	source_list="$new_source_list"
	2333
	2334
	2335	# EIGENTLICHE UEBERSETZUNG MIT DEN NORMALEN OPTIONEN
	2336	printf " *** compilation with standard options: $compiler_name $FOPTS $MODULE_OPTIONS \n"
	2337	printf " preprocessor-directives: $C_PPOPTS \n"
	2338	printf " linker-options: $LOPTS \n"
	2339	printf " source code files: $source_list \n"
[5]	2340
	2341	$compiler_name $C_PPOPTS $FOPTS $MODULE_OPTIONS $LOPTS $source_list
	2342
[1]	2343	fi
	2344	if [[ $? != 0 \|\| "$compile_error" = true \|\| "$module_compile_error" = true ]]
	2345	then
	2346	printf "\n +++ error occured while compiling or linking"
	2347	locat=compile
	2348	exit
	2349	else
	2350	printf "$striche\n *** compilation finished \n"
	2351	fi
	2352	fi
	2353
	2354
	2355	# FALLS NUR TESTWEISE KOMPILIERT WERDEN SOLLTE, IST MRUN JETZT FERTIG
	2356	if [[ $local_compile = true ]]
	2357	then
	2358	cd $HOME
	2359	rm -rf $TEMPDIR
	2360	locat=local_compile; exit
	2361	fi
	2362
	2363
	2364	# BEREITSTELLEN DER INPUT-DATEIEN
	2365	# SCHLEIFE UEBER ALLE VOM BENUTZER ANGEGEBENEN DATEIEN
	2366	(( i = 0 ))
	2367	while (( i < iin ))
	2368	do
	2369	(( i = i + 1 ))
	2370	if (( i == 1 ))
	2371	then
	2372	printf "\n\n *** providing INPUT-files:\n$striche"
	2373	fi
	2374
	2375
	2376	# OPTIONALE DATEIEN BEI NICHTVORHANDENSEIN UEBERGEHEN
	2377	if [[ "${transin[$i]}" = unavailable ]]
	2378	then
	2379	if [[ "${extin[$i]}" = "" \|\| "${extin[$i]}" = " " ]]
	2380	then
	2381	printf "\n +++ WARNING: input file \"${pathin[$i]}/${afname}${endin[$i]}\" "
	2382	printf "\n is not available!"
	2383	else
	2384	printf "\n +++ WARNING: input file \"${pathin[$i]}/${afname}${endin[$i]}.${extin[$i]}\" "
	2385	printf "\n is not available!"
	2386	fi
	2387	continue
	2388	fi
	2389
	2390	# PRUEFEN, OB EINZELDATEI ODER DATEI PRO PROZESSOR
	2391	files_for_pes=false; datentyp=file
	2392	if [[ "${actionin[$i]}" = pe && -n $numprocs ]]
	2393	then
	2394	files_for_pes=true; datentyp=directory
	2395	actionin[$i]=""
	2396	elif [[ "${actionin[$i]}" = pe && ! -n $numprocs ]]
	2397	then
	2398	actionin[$i]=""
	2399	elif [[ "${actionin[$i]}" = arpe && -n $numprocs ]]
	2400	then
	2401	files_for_pes=true; datentyp=directory
	2402	actionin[$i]="ar"
	2403	elif [[ "${actionin[$i]}" = arpe && ! -n $numprocs ]]
	2404	then
	2405	actionin[$i]="ar"
	2406	elif [[ "${actionin[$i]}" = flpe && -n $numprocs ]]
	2407	then
	2408	files_for_pes=true; datentyp=directory
	2409	actionin[$i]="fl"
	2410	elif [[ "${actionin[$i]}" = flpe && ! -n $numprocs ]]
	2411	then
	2412	actionin[$i]="fl"
	2413	fi
	2414
	2415	if [[ $files_for_pes = true ]]
	2416	then
	2417	printf "\n >>> INPUT: ${absnamein[$i]}/.... to ${localin[$i]}"
	2418	else
	2419	printf "\n >>> INPUT: ${absnamein[$i]} to ${localin[$i]}"
	2420	fi
	2421
	2422	# INPUT-DATEI FUER EINEN FORTSETZUNGSLAUF. ES WIRD GEPRUEFT,
	2423	# OB DIESE DATEI NOCH AUF DEM TEMPORAEREN DATENKATALOG VORHANDEN
	2424	# IST. FALLS NICHT, WIRD VERSUCHT, SIE ANSCHLIESSEND VOM ARCHIV-
	2425	# SERVER ZU HOLEN
	2426	if [[ "${actionin[$i]}" = fl ]]
	2427	then
	2428	printf "\n $datentyp will be fetched from temporary directory \"${tmp_data_catalog}\" !"
	2429	if [[ $files_for_pes = false ]]
	2430	then
	2431	if [[ -f "$tmp_data_catalog/${frelin[$i]}" ]]
	2432	then
	2433	ln $tmp_data_catalog/${frelin[$i]} ${localin[$i]}
	2434	got_tmp[$i]=true
	2435	elif [[ -f "$WORK/${frelin[$i]}" && $ignore_archive_error = true ]]
	2436	then
	2437	printf "\n +++ $datentyp not found in \"$tmp_data_catalog\" !"
	2438	printf "\n *** trying to use backup copy in \"$WORK\" "
	2439	cp $WORK/${frelin[$i]} ${localin[$i]}
	2440	else
	2441	printf "\n +++ $datentyp not found in \"$tmp_data_catalog\" "
	2442	printf "\n or \"$tmp_data_catalog\" does not exist!"
	2443	printf "\n *** trying to get copy from archive"
	2444	actionin[$i]=ar
	2445	fi
	2446	else
	2447	if [[ -d "$tmp_data_catalog/${frelin[$i]}" ]]
	2448	then
	2449	mkdir ${localin[$i]}
	2450	ln $tmp_data_catalog/${frelin[$i]}/* ${localin[$i]}
	2451	got_tmp[$i]=true
	2452	elif [[ -d "$WORK/${frelin[$i]}" && $ignore_archive_error = true ]]
	2453	then
	2454	printf "\n +++ $datentyp not found in \"$tmp_data_catalog\" !"
	2455	printf "\n *** trying to use backup copy in \"$WORK\" "
	2456	cp -r $WORK/${frelin[$i]} ${localin[$i]}
	2457	else
	2458	printf "\n +++ $datentyp not found in \"$tmp_data_catalog\" "
	2459	printf "\n or \"$tmp_data_catalog\" does not exist!"
	2460	printf "\n *** trying to get copy from archive"
	2461	actionin[$i]=ar
	2462	fi
	2463	fi
	2464	fi
	2465
	2466
	2467	# DATEI LIEGT AUF ARCHIV-SERVER
	2468	if [[ "${actionin[$i]}" = ar ]]
	2469	then
	2470
	2471	if [[ $files_for_pes = false ]]
	2472	then
	2473	printf "\n file will be restored from archive-system ($archive_system)!"
	2474	else
	2475	printf "\n directory will be restored from archive-system ($archive_system)!"
	2476	fi
	2477
	2478	file_restored=false
	2479
	2480	if [[ $archive_system = asterix ]]
	2481	then
	2482	do_stagein=true
	2483	(( stagein_anz = 0 ))
	2484	while [[ $do_stagein = true ]]
	2485	do
	2486	if [[ $files_for_pes = false ]]
	2487	then
	2488	stagein -O ${frelin[$i]} > STAGEIN_OUTPUT
	2489	else
	2490	stagein -t -O ${frelin[$i]} > STAGEIN_OUTPUT
	2491	fi
	2492	cat STAGEIN_OUTPUT
	2493	if [[ $(grep -c "st.msg:i24" STAGEIN_OUTPUT) != 0 ]]
	2494	then
	2495	file_restored=true
	2496	do_stagein=false
	2497	else
	2498	(( stagein_anz = stagein_anz + 1 ))
	2499	if (( stagein_anz == 10 ))
	2500	then
	2501	printf "\n +++ stagein stoped after 10 tries"
	2502	locat=stage
	2503	exit
	2504	fi
	2505	printf "\n +++ restoring from archive failed, trying again:"
	2506	sleep 900
	2507	fi
	2508	done
	2509	elif [[ $archive_system = DMF ]]
	2510	then
	2511	if [[ $files_for_pes = false ]]
	2512	then
	2513	printf "\n +++ restoring of single files impossible with $archive_system !\n"
	2514	locat=DMF
	2515	exit
	2516	else
	2517	find $ARCHIVE/${frelin[$i]} -type m -print \| dmget
	2518	cp -r $ARCHIVE/${frelin[$i]} $PWD
	2519	file_restored=true
	2520	fi
	2521	elif [[ $archive_system = tivoli ]]
	2522	then
	2523	if [[ $files_for_pes = false ]]
	2524	then
	2525	if [[ $localhost = ibmh ]]
	2526	then
	2527	ssh $usern@hdata.hlrn.de "cp $PERM/${frelin[$i]} $PWD"
	2528	else
	2529	ssh $usern@bdata.hlrn.de "cp $PERM/${frelin[$i]} $PWD"
	2530	fi
	2531	else
	2532	(( inode = 0 ))
	2533	while (( inode < nodes ))
	2534	do
	2535	if [[ $localhost = ibmh ]]
	2536	then
	2537	ssh $usern@hdata.hlrn.de "cd $PWD; tar xf $PERM/${frelin[$i]}/${frelin[$i]}.node_$inode.tar"
	2538	else
	2539	ssh $usern@bdata.hlrn.de "cd $PWD; tar xf $PERM/${frelin[$i]}/${frelin[$i]}.node_$inode.tar"
	2540	fi
	2541	(( inode = inode + 1 ))
	2542	done
	2543	fi
	2544	file_restored=true
	2545	elif [[ $archive_system = ut ]]
	2546	then
	2547	if [[ $files_for_pes = false ]]
	2548	then
	2549	cp $UT/${frelin[$i]} .
	2550	else
	2551	(( inode = 0 ))
	2552	while (( inode < nodes ))
	2553	do
	2554	tar xf $UT/${frelin[$i]}/${frelin[$i]}.node_$inode.tar
	2555	(( inode = inode + 1 ))
	2556	done
	2557	fi
	2558	file_restored=true
	2559	else
	2560	printf "\n +++ archive_system=\"$archive_system\" restore impossible!"
	2561	locat=rearchive
	2562	exit
	2563	fi
	2564
	2565	if [[ $file_restored = true ]]
	2566	then
	2567
	2568	# DATEI AUCH AUF TEMPORAERES DATENVERZEICHNIS LEGEN, DAMIT
	2569	# SIE BEI WEITEREN ZUGRIFFEN NOCH VORHANDEN IST
	2570	[[ ! -d $tmp_data_catalog ]] && mkdir -p $tmp_data_catalog; chmod g+rx $tmp_data_catalog
	2571	if [[ $files_for_pes = false ]]
	2572	then
	2573	ln -f ${frelin[$i]} $tmp_data_catalog/${frelin[$i]}
	2574	else
	2575	mkdir $tmp_data_catalog/${frelin[$i]}
	2576	ln -f ${frelin[$i]}/* $tmp_data_catalog/${frelin[$i]}
	2577	fi
	2578	got_tmp[$i]=true
	2579
	2580	# DATEI UNTER LOKALEM NAMEN ZUR VERFUEGUNG STELLEN
	2581	mv ${frelin[$i]} ${localin[$i]}
	2582
	2583	fi
	2584	fi
	2585
	2586
	2587	# DATEI LIEGT IM VOM BENUTZER ANGEGEBENEN VERZEICHNIS
	2588	if [[ "${actionin[$i]}" = "" \|\| "${actionin[$i]}" = "di" \|\| "${actionin[$i]}" = "npe" ]]
	2589	then
	2590
	2591	if [[ "${actionin[$i]}" = "npe" && -n $numprocs ]]
	2592	then
	2593
	2594	# DATEI WIRD FUER DIE PROZESSOREN EINES PARALLERECHNERS BEREITGESTELLT
	2595	printf "\n file will be provided for $numprocs processors"
	2596	mkdir ${localin[$i]}
	2597	ival=$numprocs
	2598	(( ii = 0 ))
	2599	while (( ii <= ival-1 ))
	2600	do
	2601	if (( ii < 10 ))
	2602	then
	2603	cp ${absnamein[$i]} ${localin[$i]}/_000$ii
	2604	elif (( ii < 100 ))
	2605	then
	2606	cp ${absnamein[$i]} ${localin[$i]}/_00$ii
	2607	elif (( ii < 1000 ))
	2608	then
	2609	cp ${absnamein[$i]} ${localin[$i]}/_0$ii
	2610	else
	2611	cp ${absnamein[$i]} ${localin[$i]}/_$ii
	2612	fi
	2613	(( ii = ii + 1 ))
	2614	done
	2615
	2616	else
	2617
	2618	if [[ $files_for_pes = true ]]
	2619	then
	2620
	2621	# DIE DEN PROZESSOREN EINES PARALLELRECHNERS ZUGEHOERIGEN
	2622	# DATEIEN WERDEN BEREITGESTELLT, INDEM ZUERST DER GESAMTE
	2623	# KATALOGINHALT KOPIERT UND DANN DIE EINZELNEN DATEIEN
	2624	# PER MOVE UMBENANNT WERDEN
	2625	printf "\n providing $numprocs files for the respective processors"
	2626	mkdir ${localin[$i]}
	2627	cp -r ${absnamein[$i]}/* ${localin[$i]}
	2628
	2629	else
	2630	# BEREITSTELLUNG AUF EINPROZESSORRECHNERN
	2631	cp ${absnamein[$i]} ${localin[$i]}
	2632	fi
	2633	fi
	2634	fi
	2635
	2636	done
	2637	if (( i != 0 ))
	2638	then
	2639	printf "\n$striche\n *** all INPUT-files provided \n"
	2640	fi
	2641
	2642
	2643	# NAMELIST-DATEI MIT WERTEN VON ENVIRONMENT-VARIABLEN ERZEUGEN (ZU
	2644	# LESEN VON PALM)
	2645	cat > ENVPAR << %%END%%
	2646	&envpar run_identifier = '$fname', host = '$localhost',
	2647	write_binary = '$write_binary', tasks_per_node = $tasks_per_node,
	2648	maximum_cpu_time_allowed = ${cpumax}. /
	2649
	2650	%%END%%
	2651
	2652
	2653	# EVENTUELLE INPUT-KOMMANDOS ABARBEITEN
	2654	(( i = 0 ))
	2655	while (( i < iic ))
	2656	do
	2657	(( i = i + 1 ))
	2658	if (( i == 1 ))
	2659	then
	2660	printf "\n\n *** execution of INPUT-commands:\n$striche"
	2661	fi
	2662	printf "\n >>> ${in_command[$i]}"
	2663	eval ${in_command[$i]}
	2664	if (( i == iic ))
	2665	then
	2666	printf "\n$striche\n"
	2667	fi
	2668	done
	2669
	2670
	2671	# VERBLEIBENDE CPU-ZEIT BERECHNEN
	2672	cpurest=${cpumax}.
	2673
	2674
	2675	# PROGRAMMSTART
	2676	printf "\n\n *** execution starts in directory\n \"`pwd`\"\n$striche\n"
	2677	PATH=$PATH:$TEMPDIR
	2678
	2679	if [[ "$cond1" = debug \|\| "$cond2" = debug ]]
	2680	then
	2681	if [[ "$ENVIRONMENT" = BATCH ]]
	2682	then
	2683	printf "\n +++ debug is allowed in interactive mode only"
	2684	locat=debug
	2685	exit
	2686	fi
[5]	2687	if [[ $localhost = decalpha ]]
[1]	2688	then
	2689	dxladebug a.out
	2690	elif [[ $localhost = ibmb \|\| $localhost = ibmh ]]
	2691	then
	2692
	2693	# SETUP THE IBM MPI ENVIRONMENT
	2694	export MP_SHARED_MEMORY=yes
	2695	export AIXTHREADS_SCOPE=S
	2696	export OMP_NUM_THREADS=$threads_per_task
	2697	export AUTHSTATE=files
	2698	export XLFRTEOPTS="nlwidth=132:err_recovery=no" # RECORD-LENGTH OF NAMELIST-OUTPUT
	2699
	2700	# FOLLOWING OPTIONS ARE MANDATORY FOR TOTALVIEW
	2701	export MP_ADAPTER_USE=shared
	2702	export MP_CPU_USE=multiple
	2703	export MP_TIMEOUT=1200
	2704
	2705	unset MP_TASK_AFFINITY
	2706
	2707	# SO FAR, TOTALVIEW NEEDS HOSTFILE MECHANISM FOR EXECUTION
	2708	#(( ii = 1 ))
	2709	#while (( ii <= $numprocs ))
	2710	#do
	2711	# echo $localhost_realname >> hostfile
	2712	# (( ii = ii + 1 ))
	2713	#done
	2714	#export MP_HOSTFILE=hostfile
	2715
	2716	if [[ "$LOADLBATCH" = yes ]]
	2717	then
	2718	totalview poe a.out $ROPTS
	2719	else
	2720	echo totalview poe -a a.out -procs $numprocs -rmpool 0 -nodes 1 $ROPTS
	2721	export TVDSVRLAUNCHCMD=ssh
	2722	totalview poe -a a.out -procs $numprocs -rmpool 0 -nodes 1 $ROPTS
	2723	fi
	2724	else
	2725	printf "\n +++ no debug available on \"$localhost\" "
	2726	printf "\n or not implemented in mrun so far"
	2727	locat=debug
	2728	exit
	2729	fi
	2730	else
	2731	if [[ -n $numprocs ]]
	2732	then
	2733
	2734	# RUNNING THE PROGRAM ON PARALLEL MACHINES
	2735	if [[ $(echo $host \| cut -c1-3) = ibm ]]
	2736	then
	2737	# SETUP THE IBM MPI ENVIRONMENT
	2738	export MP_SHARED_MEMORY=yes
	2739	export AIXTHREAD_SCOPE=S
	2740	export OMP_NUM_THREADS=$threads_per_task
	2741	export XLSMPOPTS="spins=0:yields=0:stack=20000000"
	2742	export AUTHSTATE=files
	2743	export XLFRTEOPTS="nlwidth=132:err_recovery=no" # RECORD-LENGTH OF NAMELIST-OUTPUT
	2744	# export MP_PRINTENV=yes
	2745
	2746	# TUNING-VARIABLEN ZUR VERBESSERUNG DER KOMMUNIKATION
	2747	# ZEIGEN ABER DERZEIT (SEP 04, FEDERATION) KAUM WIRKUNG
	2748	export MP_WAIT_MODE=poll
	2749	[[ $node_usage = not_shared ]] && export MP_SINGLE_THREAD=yes
	2750	# export MP_EAGER_LIMIT=65535
	2751
	2752	# # TESTWEISE FUER TURBOMPI (JAN 05)
	2753	# export MPJ_MTAB=128
	2754	# export MPJ_ALLTOALL=1
	2755	# export MPJ_ALLTOALLV=1
	2756
	2757	if [[ "$LOADLBATCH" = yes ]]
	2758	then
	2759	printf "\n--- Control: OMP_NUM_THREADS = \"$OMP_NUM_THREADS\" \n"
	2760	if [[ "$cond1" = hpmcount \|\| "$cond2" = hpmcount ]]
	2761	then
	2762	/opt/optibm/HPM_2_4_1/bin/hpmcount a.out $ROPTS > aout_output 2>&1
	2763	else
	2764	poe ./a.out $ROPTS > aout_output 2>&1
	2765	fi
	2766	else
	2767	if [[ $localhost = ibmb \|\| $localhost = ibmh \|\| $localhost = ibms ]]
	2768	then
	2769	poe a.out -procs $numprocs -nodes 1 -rmpool 0 $ROPTS > aout_output 2>&1
	2770	elif [[ $localhost = ibmy ]]
	2771	then
	2772	if [[ -f $hostfile ]]
	2773	then
	2774	cp $hostfile hostfile
	2775	else
	2776	(( ii = 1 ))
	2777	while (( ii <= $numprocs ))
	2778	do
	2779	echo $localhost_realname >> hostfile
	2780	(( ii = ii + 1 ))
	2781	done
	2782	fi
	2783	export MP_HOSTFILE=hostfile
	2784	./a.out -procs $tasks_per_node $ROPTS > aout_output 2>&1
	2785	else
	2786	if [[ "$host_file" = "" ]]
	2787	then
	2788	printf "\n +++ no hostfile given in configuration file"
	2789	locat=config_file
	2790	exit
	2791	else
	2792	eval host_file=$host_file
	2793	fi
	2794	export MP_HOSTFILE=$host_file
	2795	poe a.out -procs $numprocs -tasks_per_node $numprocs $ROPTS > aout_output 2>&1
	2796	fi
	2797	fi
	2798	elif [[ $host = nech \|\| $host = neck ]]
	2799	then
	2800	(( ii = nodes ))
	2801	if [[ $ii = 1 ]]
	2802	then
	2803	export F_ERRCNT=0 # acceptable number of errors before program is stopped
	2804	export MPIPROGINF=YES
	2805	echo "*** execution on single node with mpirun"
	2806	mpirun -np $numprocs ./a.out $ROPTS > aout_output 2>&1
	2807	else
	2808	(( i = 0 ))
	2809	while (( i < ii ))
	2810	do
	2811	echo "-h $i -p $tasks_per_node -e ./mpi_exec_shell" >> multinode_config
	2812	(( i = i + 1 ))
	2813	done
	2814
	2815	echo "#!/bin/sh" > mpi_exec_shell
	2816	echo " " >> mpi_exec_shell
	2817	echo "set -u" >> mpi_exec_shell
	2818	echo "F_ERRCNT=0" >> mpi_exec_shell
	2819	echo "MPIPROGINV=YES" >> mpi_exec_shell
	2820	echo "OMP_NUM_THREADS=$threads_per_task" >> mpi_exec_shell
	2821	echo "cpurest=$cpurest" >> mpi_exec_shell
	2822	echo "fname=$fname" >> mpi_exec_shell
	2823	echo "localhost=$localhost" >> mpi_exec_shell
	2824	echo "return_addres=$return_addres" >> mpi_exec_shell
	2825	echo "return_username=$return_username" >> mpi_exec_shell
	2826	echo "tasks_per_node=$tasks_per_node" >> mpi_exec_shell
	2827	echo "write_binary=$write_binary" >> mpi_exec_shell
	2828	echo "use_seperate_pe_for_dvrp_output=$use_seperate_pe_for_dvrp_output" >> mpi_exec_shell
	2829	echo " " >> mpi_exec_shell
	2830	echo "export F_ERRCNT" >> mpi_exec_shell
	2831	echo "export MPIPROGINV" >> mpi_exec_shell
	2832	echo "export OMP_NUM_THREADS" >> mpi_exec_shell
	2833	echo "export cpurest" >> mpi_exec_shell
	2834	echo "export fname" >> mpi_exec_shell
	2835	echo "export localhost" >> mpi_exec_shell
	2836	echo "export return_addres" >> mpi_exec_shell
	2837	echo "export return_username" >> mpi_exec_shell
	2838	echo "export tasks_per_node" >> mpi_exec_shell
	2839	echo "export write_binary" >> mpi_exec_shell
	2840	echo "export use_seperate_pe_for_dvrp_output" >> mpi_exec_shell
	2841	echo " " >> mpi_exec_shell
	2842	echo "exec ./a.out > aout_output 2>&1" >> mpi_exec_shell
	2843
	2844	chmod u+x mpi_exec_shell
	2845	export MPIPROGINF=YES
	2846	mpirun -f multinode_config &
	2847	wait
	2848
	2849	fi
	2850	elif [[ $host = lcmuk ]]
	2851	then
	2852
	2853	# COPY HOSTFILE FROM SOURCE DIRECTORY OR CREATE IT, IF IT
	2854	# DOES NOT EXIST
	2855	if [[ -f $hostfile ]]
	2856	then
	2857	cp $hostfile hostfile
	2858	else
	2859	(( ii = 1 ))
	2860	while (( ii <= $numprocs / $threads_per_task ))
	2861	do
	2862	echo $localhost_realname >> hostfile
	2863	(( ii = ii + 1 ))
	2864	done
	2865	fi
	2866	(( ii = $numprocs / $threads_per_task ))
	2867	eval zeile=\"`head -n $ii hostfile`\"
	2868	printf "\n *** running on: $zeile"
	2869	export OMP_NUM_THREADS=$threads_per_task
	2870	if [[ $threads_per_task != 1 ]]
	2871	then
	2872	printf "\n threads per task: $threads_per_task"
	2873	fi
	2874	printf "\n\n"
	2875	if [[ $scirocco = true ]]
	2876	then
	2877	mpiexec -machinefile hostfile -n $ii a.out $ROPTS > aout_output 2>&1
	2878	else
	2879	mpirun -machinefile hostfile -np $ii a.out $ROPTS > aout_output 2>&1
	2880	fi
	2881	elif [[ $host = decalpha ]]
	2882	then
	2883	dmpirun -np $numprocs a.out $ROPTS > aout_output 2>&1
	2884	elif [[ $host = lctit ]]
	2885	then
	2886	export runfile=runfile.$kennung
	2887
	2888	echo "cd $PWD" > $runfile
	2889	echo "export OMP_NUM_THREADS=$OMP_NUM_THREADS" >> $runfile
	2890	echo "export cpurest=$cpurest" >> $runfile
	2891	echo "export fname=$fname" >> $runfile
	2892	echo "export localhost=$localhost" >> $runfile
	2893	echo "export return_addres=$return_addres" >> $runfile
	2894	echo "export return_username=$return_username" >> $runfile
	2895	echo "export tasks_per_node=$tasks_per_node" >> $runfile
	2896	echo "export write_binary=$write_binary" >> $runfile
	2897	echo "export use_seperate_pe_for_dvrp_output=$use_seperate_pe_for_dvrp_output" >> $runfile
	2898	echo "./a.out" >> $runfile
	2899	chmod u+x $runfile
	2900
	2901	if [[ "$QUEUE" = interactive ]]
	2902	then
	2903	mpirun -np $numprocs $runfile $ROPTS > aout_output 2>&1
	2904	else
	2905	n1ge -fore -mpi $numprocs -q $queue $runfile $ROPTS > aout_output 2>&1
	2906	fi
	2907
	2908	rm $runfile
	2909	else
	2910	mpprun -n $numprocs a.out $ROPTS > aout_output 2>&1
	2911	fi
	2912	[[ $? != 0 ]] && execution_error=true
	2913
	2914
	2915	# PERFORMANCE-AUSWERTUNG MIT APPRENTICE
	2916	if [[ "$cond1" = apprentice \|\| "$cond2" = apprentice ]]
	2917	then
	2918	apprentice
	2919	fi
	2920	else
	2921	a.out $ROPTS > aout_output 2>&1
	2922	fi
	2923	fi
	2924	if [[ $? != 0 \|\| $execution_error = true ]]
	2925	then
	2926
	2927	# ABBRUCH BEI LAUFZEITFEHLER
	2928	[[ ! ( "$cond1" = debug \|\| "$cond2" = debug ) ]] && cat aout_output
	2929	printf "\n +++ runtime error occured"
	2930	locat=execution
	2931	exit
	2932	else
	2933	[[ ! ( "$cond1" = debug \|\| "$cond2" = debug ) ]] && cat aout_output
	2934	printf "\n$striche\n *** execution finished \n"
	2935	fi
	2936
	2937
	2938
	2939	# EVENTUELLE OUTPUT-KOMMANDOS ABARBEITEN
	2940	(( i = 0 ))
	2941	while (( i < ioc ))
	2942	do
	2943	(( i = i + 1 ))
	2944	if (( i == 1 ))
	2945	then
	2946	printf "\n\n *** execution of OUTPUT-commands:\n$striche"
	2947	fi
	2948	printf "\n >>> ${out_command[$i]}"
	2949	eval ${out_command[$i]}
	2950	if (( i == ioc ))
	2951	then
	2952	printf "\n$striche\n"
	2953	fi
	2954	done
	2955
	2956
	2957	# EVTL. INHALT DES AKTUELLEN VERZEICHNISSES AUSGEBEN
	2958	if [[ $do_trace = true ]]
	2959	then
	2960	printf "\n\n"
	2961	ls -al
	2962	fi
	2963
	2964
	2965
	2966	# OUTPUT-DATEIEN AN IHRE ZIELORTE KOPIEREN
	2967	(( i = 0 ))
	2968	while (( i < iout ))
	2969	do
	2970	(( i = i + 1 ))
	2971	if (( i == 1 ))
	2972	then
	2973	printf "\n\n *** saving OUTPUT-files:\n$striche"
	2974	fi
	2975
	2976	# PRUEFEN, OB EINZELDATEI ODER DATEI PRO PROZESSOR
	2977	files_for_pes=false; filetyp=file
	2978	if [[ "${actionout[$i]}" = pe && -n $numprocs ]]
	2979	then
	2980	files_for_pes=true; filetyp=directory
	2981	actionout[$i]=""
	2982	elif [[ "${actionout[$i]}" = pe && ! -n $numprocs ]]
	2983	then
	2984	actionout[$i]=""
	2985	elif [[ "${actionout[$i]}" = arpe && -n $numprocs ]]
	2986	then
	2987	files_for_pes=true; filetyp=directory
	2988	actionout[$i]="ar"
	2989	elif [[ "${actionout[$i]}" = arpe && ! -n $numprocs ]]
	2990	then
	2991	actionout[$i]="ar"
	2992	elif [[ "${actionout[$i]}" = flpe && -n $numprocs ]]
	2993	then
	2994	files_for_pes=true; filetyp=directory
	2995	actionout[$i]="fl"
	2996	elif [[ "${actionout[$i]}" = flpe && ! -n $numprocs ]]
	2997	then
	2998	actionout[$i]="fl"
	2999	elif [[ "${actionout[$i]}" = trpe && -n $numprocs ]]
	3000	then
	3001	files_for_pes=true; filetyp=directory
	3002	actionout[$i]="tr"
	3003	elif [[ "${actionout[$i]}" = trpe && ! -n $numprocs ]]
	3004	then
	3005	actionout[$i]="tr"
	3006	fi
	3007
	3008	if [[ ! -f ${localout[$i]} && $files_for_pes = false ]]
	3009	then
	3010	printf "\n +++ temporary OUTPUT-file ${localout[$i]} does not exist\n"
	3011	elif [[ ! -f ${localout[$i]}/_0000 && $files_for_pes = true ]]
	3012	then
	3013	printf "\n +++ temporary OUTPUT-file ${localout[$i]}/.... does not exist\n"
	3014	else
	3015
	3016
	3017	# KOPIEREN PER FTP/SCP (IMMER IM BINAERMODUS, -M: FALLS ZIELKATALOG
	3018	# NICHT VORHANDEN, WIRD VERSUCHT, IHN ANZULEGEN), ABER NUR BEI
	3019	# REMOTE-RECHNUNGEN
	3020	if [[ "${actionout[$i]}" = tr ]]
	3021	then
	3022	if [[ $localhost != $fromhost ]]
	3023	then
	3024	if [[ $files_for_pes = false ]]
	3025	then
	3026	cps=""
	3027	cst=""
	3028	else
	3029	cps=-c
	3030	cst="/"
	3031	fi
	3032	if [[ $localhost = ibmh \|\| $localhost = ibmb \|\| $localhost = nech ]]
	3033	then
	3034
	3035	# TRANSFER IN EIGENSTAENDIGEM JOB
	3036	# ZUERST KOPIE DER DATEI INS TEMPORAERE DATENVERZEICHNIS
	3037	[[ ! -d $tmp_data_catalog/TRANSFER ]] && mkdir -p $tmp_data_catalog/TRANSFER
	3038	file_to_transfer=${fname}_${localout[$i]}_to_transfer_$kennung
	3039	if [[ $files_for_pes = false ]]
	3040	then
	3041	ln -f ${localout[$i]} $tmp_data_catalog/TRANSFER/$file_to_transfer
	3042	else
	3043	mkdir $tmp_data_catalog/TRANSFER/$file_to_transfer
	3044	ln ${localout[$i]}/* $tmp_data_catalog/TRANSFER/$file_to_transfer
	3045	fi
	3046
	3047	echo "set -x" > transfer_${localout[$i]}
	3048	echo "cd $tmp_data_catalog/TRANSFER" >> transfer_${localout[$i]}
	3049	if [[ $filetransfer_protocol = ftp ]]
	3050	then
	3051	if [[ $files_for_pes = true ]]
	3052	then
	3053	printf "\n +++ WARNING: no transfer of catalogs via ftpcopy\n"
	3054	fi
	3055	printf "\n >>> OUTPUT: ${localout[$i]} by FTP in seperate job to"
	3056	printf "\n ${pathout[$i]}/${localhost}_${fname}${endout[$i]}"
	3057	printf "\n or higher cycle\n"
	3058	echo "ftpcopy -b -m $return_addres $file_to_transfer \"${pathout[$i]}\" ${localhost}_${fname}${endout[$i]} ${extout[$i]}" >> transfer_${localout[$i]}
	3059	else
	3060	printf "\n >>> OUTPUT: ${localout[$i]}$cst by SCP in seperate job to"
	3061	printf "\n ${pathout[$i]}/${localhost}_${fname}${endout[$i]}$cst"
	3062	printf "\n or higher cycle\n"
	3063	echo "batch_scp $cps -b -m -u $return_username $return_addres $file_to_transfer \"${pathout[$i]}\" ${localhost}_${fname}${endout[$i]} ${extout[$i]}" >> transfer_${localout[$i]}
	3064	fi
	3065	echo "[[ \$? = 0 ]] && rm $file_to_transfer" >> transfer_${localout[$i]}
	3066
	3067	if [[ $localhost = nech ]]
	3068	then
	3069	subjob -d -c /pf/b/$usern/job_queue -v -q pp -X 0 -m 1000 -t 900 -f $filetransfer_protocol transfer_${localout[$i]}
	3070	else
	3071	if [[ "$LOGNAME" = b323013 ]]
	3072	then
	3073	subjob -v -q c1 -X 0 -m 1000 -t 900 -f $filetransfer_protocol transfer_${localout[$i]}
	3074	else
	3075	subjob -d -v -q c1 -X 0 -m 1000 -t 900 -f $filetransfer_protocol transfer_${localout[$i]}
	3076	fi
	3077	fi
	3078
	3079	else
	3080
	3081	# TRANSFER INNERHALB DIESES JOBS
	3082	transfer_failed=false
	3083	if [[ $filetransfer_protocol = ftp ]]
	3084	then
	3085	if [[ $files_for_pes = true ]]
	3086	then
	3087	printf "\n +++ WARNING: no transfer of catalogs via ftpcopy\n"
	3088	fi
	3089	printf "\n >>> OUTPUT: ${localout[$i]} by FTP to"
	3090	printf "\n ${pathout[$i]}/${localhost}_${fname}${endout[$i]}\n"
	3091	ftpcopy -b -m $return_addres ${localout[$i]} "${pathout[$i]}" ${localhost}_${fname}${endout[$i]} ${extout[$i]}
	3092	[[ $? != 0 ]] && transfer_failed=true
	3093	else
	3094	printf "\n >>> OUTPUT: ${localout[$i]}$cst by SCP to"
	3095	printf "\n ${pathout[$i]}/${localhost}_${fname}${endout[$i]}$cst\n"
	3096	batch_scp $cps -b -m -u $return_username $return_addres ${localout[$i]} "${pathout[$i]}" ${localhost}_${fname}${endout[$i]} ${extout[$i]}
	3097	[[ $? != 0 ]] && transfer_failed=true
	3098	fi
	3099
	3100	# BEI FEHLGESCHLAGENEM TRANSFER SICHERUNGSKOPIE AUF
	3101	# LOKALER MASCHINE ANLEGEN
	3102	if [[ $transfer_failed = true ]]
	3103	then
	3104	printf " +++ transfer failed. Trying to save a copy on the local host under:\n"
	3105	printf " ${pathout[$i]}/${localhost}_${fname}${endout[$i]}_$kennung\n"
	3106
	3107	# ERSTMAL PRUEFEN, OB VERZEICHNIS EXISTIERT. GEGEBENENFALLS
	3108	# ANLEGEN.
	3109	eval local_catalog=${pathout[$i]}
	3110	if [[ ! -d $local_catalog ]]
	3111	then
	3112	printf " *** local directory does not exist. Trying to create:\n"
	3113	printf " $local_catalog \n"
	3114	mkdir -p $local_catalog
	3115	fi
	3116	eval cp ${localout[$i]} ${pathout[$i]}/${localhost}_${fname}${endout[$i]}_$kennung
	3117	transfer_problems=true
	3118	fi
	3119	fi
	3120	else
	3121
	3122	# WERTZUWEISUNG, SO DASS WEITER UNTEN NUR KOPIERT WIRD
	3123	actionout[$i]=""
	3124	fi
	3125	fi
	3126
	3127
	3128	# APPEND PER FTP/SCP (IMMER IM BINAERMODUS, -M: FALLS ZIELKATALOG
	3129	# NICHT VORHANDEN, WIRD VERSUCHT, IHN ANZULEGEN), ABER NUR BEI
	3130	# REMOTE-RECHNUNGEN
	3131	if [[ "${actionout[$i]}" = tra ]]
	3132	then
	3133	if [[ $localhost != $fromhost ]]
	3134	then
	3135	if [[ $localhost = ibmh \|\| $localhost = ibmb \|\| $localhost = nech ]]
	3136	then
	3137
	3138	# TRANSFER IN EIGENSTAENDIGEM JOB
	3139	# ZUERST KOPIE DER DATEI INS TEMPORAERE DATENVERZEICHNIS
	3140	[[ ! -d $tmp_data_catalog/TRANSFER ]] && mkdir -p $tmp_data_catalog/TRANSFER
	3141	file_to_transfer=${fname}_${localout[$i]}_to_transfer_$kennung
	3142	ln -f ${localout[$i]} $tmp_data_catalog/TRANSFER/$file_to_transfer
	3143
	3144	echo "set -x" > transfer_${localout[$i]}
	3145	echo "cd $tmp_data_catalog/TRANSFER" >> transfer_${localout[$i]}
	3146	if [[ $filetransfer_protocol = ftp ]]
	3147	then
	3148	printf "\n >>> OUTPUT: ${localout[$i]} append by FTP in seperate job to"
	3149	printf "\n ${pathout[$i]}/${localhost}_${fname}${endout[$i]}"
	3150	printf "\n or higher cycle\n"
	3151	echo "ftpcopy -A -b -m $return_addres $file_to_transfer \"${pathout[$i]}\" ${localhost}_${fname}${endout[$i]} ${extout[$i]}" >> transfer_${localout[$i]}
	3152	else
	3153	printf "\n >>> OUTPUT: ${localout[$i]} append by SCP in seperate job to"
	3154	printf "\n ${pathout[$i]}/${localhost}_${fname}${endout[$i]}"
	3155	printf "\n or higher cycle\n"
	3156	echo "batch_scp -A -b -m -u $return_username $return_addres $file_to_transfer \"${pathout[$i]}\" ${localhost}_${fname}${endout[$i]} ${extout[$i]}" >> transfer_${localout[$i]}
	3157	fi
	3158	echo "[[ \$? = 0 ]] && rm $file_to_transfer" >> transfer_${localout[$i]}
	3159
	3160	if [[ $localhost = nech ]]
	3161	then
	3162	subjob -d -c /pf/b/$usern/job_queue -v -q pp -X 0 -m 1000 -t 900 -f $filetransfer_protocol transfer_${localout[$i]}
	3163	else
	3164	if [[ $LOGNAME = b323013 ]]
	3165	then
	3166	subjob -v -q c1 -X 0 -m 1000 -t 900 -f $filetransfer_protocol transfer_${localout[$i]}
	3167	else
	3168	subjob -d -v -q c1 -X 0 -m 1000 -t 900 -f $filetransfer_protocol transfer_${localout[$i]}
	3169	fi
	3170	fi
	3171
	3172	else
	3173
	3174	# TRANSFER INNERHALB DIESES JOBS
	3175	transfer_failed=false
	3176	if [[ $filetransfer_protocol = ftp ]]
	3177	then
	3178	printf "\n >>> OUTPUT: ${localout[$i]} append by FTP to"
	3179	printf "\n ${pathout[$i]}/${localhost}_${fname}${endout[$i]}\n"
	3180	ftpcopy -A -b -m $return_addres ${localout[$i]} "${pathout[$i]}" ${localhost}_${fname}${endout[$i]} ${extout[$i]}
	3181	[[ $? != 0 ]] && transfer_failed=true
	3182	else
	3183	printf "\n >>> OUTPUT: ${localout[$i]} append by SCP to"
	3184	printf "\n ${pathout[$i]}/${localhost}_${fname}${endout[$i]}\n"
	3185	batch_scp -A -b -m -u $return_username $return_addres ${localout[$i]} "${pathout[$i]}" ${localhost}_${fname}${endout[$i]} ${extout[$i]}
	3186	[[ $? != 0 ]] && transfer_failed=true
	3187	fi
	3188
	3189	# BEI FEHLGESCHLAGENEM TRANSFER SICHERUNGSKOPIE AUF
	3190	# LOKALER MASCHINE ANLEGEN
	3191	if [[ $transfer_failed = true ]]
	3192	then
	3193	printf " +++ transfer failed. Trying to save a copy on the local host under:\n"
	3194	printf " ${pathout[$i]}/${localhost}_${fname}${endout[$i]}_$kennung\n"
	3195
	3196	# ERSTMAL PRUEFEN, OB VERZEICHNIS EXISTIERT. GEGEBENENFALLS
	3197	# ANLEGEN
	3198	eval local_catalog=${pathout[$i]}
	3199	if [[ ! -d $local_catalog ]]
	3200	then
	3201	printf " *** local directory does not exist. Trying to create:\n"
	3202	printf " $local_catalog \n"
	3203	mkdir -p $local_catalog
	3204	fi
	3205
	3206	eval cp ${localout[$i]} ${pathout[$i]}/${localhost}_${fname}${endout[$i]}_$kennung
	3207	transfer_problems=true
	3208	fi
	3209	fi
	3210	else
	3211
	3212	# WERTZUWEISUNG, SO DASS WEITER UNTEN NUR APPEND AUF
	3213	# LOKALER MASCHINE DURCHGEFUEHRT WIRD
	3214	actionout[$i]=a
	3215	fi
	3216	fi
	3217
	3218
	3219	# OUTPUT-DATEI FUER EINEN FORTSETZUNGSLAUF. DATEI WIRD PER
	3220	# LINK AUF DEN TEMPORAEREN DATENKATALOG GELEGT. OPTION -f WIRD
	3221	# VERWENDET, FALLS DORT NOCH EINE DATEI GLEICHEN NAMENS VORHANDEN
	3222	# SEIN SOLLTE. ANSCHLIESSEND WIRD SEINE ARCHIVIERUNG ANGESTOSSEN
	3223	if [[ "${actionout[$i]}" = fl ]]
	3224	then
	3225	[[ ! -d $tmp_data_catalog ]] && mkdir -p $tmp_data_catalog
	3226	chmod g+rx $tmp_data_catalog
	3227	if [[ $files_for_pes = false ]]
	3228	then
	3229	printf "\n >>> OUTPUT: ${localout[$i]} to"
	3230	printf "\n $tmp_data_catalog/${frelout[$i]} (temporary data catalog)\n"
	3231	ln -f ${localout[$i]} $tmp_data_catalog/${frelout[$i]}
	3232	else
	3233	printf "\n >>> OUTPUT: ${localout[$i]}/.... to"
	3234	printf "\n $tmp_data_catalog/${frelout[$i]} (temporary data catalog)\n"
	3235	mkdir $tmp_data_catalog/${frelout[$i]}
	3236	ln -f ${localout[$i]}/* $tmp_data_catalog/${frelout[$i]}
	3237	fi
	3238
	3239
	3240	# ARCHIVIERUNGSJOB WIRD ERZEUGT UND ABGESCHICKT
	3241	if [[ $store_on_archive_system = true ]]
	3242	then
	3243
	3244	if [[ $archive_system = asterix ]]
	3245	then
	3246	echo "cd $tmp_data_catalog" >> archive_${frelout[$i]}
	3247	if [[ $files_for_pes = false ]]
	3248	then
	3249	echo "stageout ${frelout[$i]} > STAGE_OUTPUT${i}_$kennung" >> archive_${frelout[$i]}
	3250	else
	3251	echo "stageout -t ${frelout[$i]} > STAGE_OUTPUT${i}_$kennung" >> archive_${frelout[$i]}
	3252	fi
	3253	echo "cat STAGE_OUTPUT${i}_$kennung" >> archive_${frelout[$i]}
	3254	echo "if [[ \$(grep -c \"st.msg:150\" STAGE_OUTPUT${i}_$kennung) != 0 ]]" >> archive_${frelout[$i]}
	3255	echo "then" >> archive_${frelout[$i]}
	3256	echo " do_stageout=false" >> archive_${frelout[$i]}
	3257	echo "else" >> archive_${frelout[$i]}
	3258	echo " echo \" +++ $filetyp ${frelout[$i]} could not be stored on archive-system\" " >> archive_${frelout[$i]}
	3259	echo " cat /dev/null > ~/job_queue/ARCHIVE_ERROR_$fname" >> archive_${frelout[$i]}
	3260	echo " cat STAGE_OUTPUT${i}_$kennung > ~/job_queue/archive_${frelout[$i]}_error" >> archive_${frelout[$i]}
	3261	echo " echo \" *** $filetyp ${frelout[$i]} will be copied to \$WORK as backup\" " >> archive_${frelout[$i]}
	3262	if [[ $files_for_pes = false ]]
	3263	then
	3264	echo " cp ${frelout[$i]} \$WORK" >> archive_${frelout[$i]}
	3265	else
	3266	echo " cp -r ${frelout[$i]} \$WORK/${frelout[$i]}" >> archive_${frelout[$i]}
	3267	fi
	3268	echo " echo \" *** $filetyp ${frelout[$i]} saved\" " >> archive_${frelout[$i]}
	3269	echo "fi" >> archive_${frelout[$i]}
	3270	echo "rm STAGE_OUTPUT${i}_$kennung" >> archive_${frelout[$i]}
	3271	elif [[ $archive_system = DMF ]]
	3272	then
	3273	echo "cd $tmp_data_catalog" >> archive_${frelout[$i]}
	3274	if [[ $files_for_pes = false ]]
	3275	then
	3276	printf "\n +++ archiving of single files impossible with $archive_system !\n"
	3277	locat=DMF
	3278	exit
	3279	else
	3280	# FUER RECHNER IN JUELICH. DORT KOENNTEN AUCH
	3281	# EINZELNE DATEIEN GESICHERT WERDEN (SPAETER KORR.)
	3282	echo "rm -rf \$ARCHIVE/${frelout[$i]}" >> archive_${frelout[$i]}
	3283	echo "cp -r ${frelout[$i]} \$ARCHIVE" >> archive_${frelout[$i]}
	3284	fi
	3285	elif [[ $archive_system = tivoli ]]
	3286	then
	3287	echo "cd $tmp_data_catalog" >> archive_${frelout[$i]}
	3288	if [[ $files_for_pes = false ]]
	3289	then
	3290	# EVENTUELL NOCH VORHANDENE DATEI IM ARCHIV LOSCHEN
	3291	echo "set -x" >> archive_${frelout[$i]}
	3292	echo "rm -rf \$PERM/${frelout[$i]}" >> archive_${frelout[$i]}
	3293	echo "cp ${frelout[$i]} \$PERM/${frelout[$i]}" >> archive_${frelout[$i]}
	3294	else
	3295
	3296	echo "set -x" >> archive_${frelout[$i]}
	3297	echo "rm -rf \$PERM/${frelout[$i]}/*" >> archive_${frelout[$i]}
	3298	echo "[[ ! -d \$PERM/${frelout[$i]} ]] && mkdir $PERM/${frelout[$i]}" >> archive_${frelout[$i]}
	3299	cd $tmp_data_catalog
	3300	all_files=`ls -1 ${frelout[$i]}/*`
	3301	cd - > /dev/null
	3302	(( inode = 0 ))
	3303	(( tp1 = tasks_per_node + 1 ))
	3304	while (( inode < nodes ))
	3305	do
	3306	# echo "*** all_files = \"$all_files\" "
	3307	files=`echo $all_files \| cut -d" " -f1-$tasks_per_node`
	3308	all_files=`echo $all_files \| cut -d" " -f$tp1-`
	3309	# echo "*** tasks_per_node = \"$tasks_per_node\" "
	3310	# echo "*** files = \"$files\" "
	3311	# echo "*** all_files = \"$all_files\" "
	3312	echo "tar cvf \$PERM/${frelout[$i]}/${frelout[$i]}.node_$inode.tar $files" >> archive_${frelout[$i]}
	3313	echo "dsmc incremental \$PERM/${frelout[$i]}/${frelout[$i]}.node_$inode.tar" >> archive_${frelout[$i]}
	3314	echo "dsmmigrate \$PERM/${frelout[$i]}/${frelout[$i]}.node_$inode.tar" >> archive_${frelout[$i]}
	3315	(( inode = inode + 1 ))
	3316	done
	3317	# echo "rm -rf \$PERM/${frelout[$i]}.tar" >> archive_${frelout[$i]}
	3318	# echo "tar cvf \$PERM/${frelout[$i]}.tar ${frelout[$i]}" >> archive_${frelout[$i]}
	3319	fi
	3320	elif [[ $archive_system = ut ]]
	3321	then
	3322	echo "cd $tmp_data_catalog" >> archive_${frelout[$i]}
	3323	if [[ $files_for_pes = false ]]
	3324	then
	3325	# EVENTUELL NOCH VORHANDENE DATEI IM ARCHIV LOSCHEN
	3326	echo "set -x" >> archive_${frelout[$i]}
	3327	echo "rm -rf \$UT/${frelout[$i]}" >> archive_${frelout[$i]}
	3328	echo "cp ${frelout[$i]} \$UT/${frelout[$i]}" >> archive_${frelout[$i]}
	3329	else
	3330
	3331	echo "set -x" >> archive_${frelout[$i]}
	3332	echo "rm -rf \$UT/${frelout[$i]}/*" >> archive_${frelout[$i]}
	3333	echo "[[ ! -d \$UT/${frelout[$i]} ]] && mkdir $UT/${frelout[$i]}" >> archive_${frelout[$i]}
	3334	cd $tmp_data_catalog
	3335	all_files=`ls -1 ${frelout[$i]}/*`
	3336	cd - > /dev/null
	3337	(( inode = 0 ))
	3338	(( tp1 = tasks_per_node + 1 ))
	3339	while (( inode < nodes ))
	3340	do
	3341	files=`echo $all_files \| cut -d" " -f1-$tasks_per_node`
	3342	all_files=`echo $all_files \| cut -d" " -f$tp1-`
	3343	echo "tar cvf \$UT/${frelout[$i]}/${frelout[$i]}.node_$inode.tar $files" >> archive_${frelout[$i]}
	3344	(( inode = inode + 1 ))
	3345	done
	3346	fi
	3347	elif [[ $archive_system = none ]]
	3348	then
	3349	printf " +++ archiving on $localhost not available!\n"
	3350	fi
	3351
	3352	if [[ $archive_system != none ]]
	3353	then
	3354	if [[ $localhost = ibmh \|\| $localhost = ibmb ]]
	3355	then
	3356	subjob -d -v -q cdata -X 0 -m 1000 -t 7200 -f $filetransfer_protocol archive_${frelout[$i]}
	3357	elif [[ $localhost = nech ]]
	3358	then
	3359	subjob -d -c /pf/b/$usern/job_queue -v -q pp -X 0 -m 1000 -t 7200 -f $filetransfer_protocol archive_${frelout[$i]}
	3360	fi
	3361	printf " Archiving of $tmp_data_catalog/${frelout[$i]} initiated (batch job submitted)\n"
	3362	fi
	3363	else
	3364	printf " +++ caution: option -A is switched on. No archiving on $archive_system!\n"
	3365	fi
	3366
	3367
	3368	# LEERDATEI IM BENUTZERVERZEICHNIS ANLEGEN, DAMIT BEKANNT IST,
	3369	# WIE DIE HOECHSTE ZYKLUSNUMMER AUF DEM ARCHIV-SYSTEM LAUTET
	3370	if [[ $files_for_pes = false ]]
	3371	then
	3372	cat /dev/null > ${pathout[$i]}
	3373	else
	3374	mkdir -p ${pathout[$i]}
	3375	fi
	3376
	3377	fi
	3378
	3379
	3380	# KOPIEREN AUF LOKALER MASCHINE ZUM ARCHIV-SYSTEM
	3381	# AUSSERDEM MUSS LEERDATEI ANGELEGT WERDEN, DAMIT BEKANNT IST,
	3382	# WIE DIE HOECHSTE ZYKLUSNUMMER AUF DEM ARCHIV-SYSTEM LAUTET
	3383	# FALLS IRGENDEINE (VON MEHREREN) ARCHIVIERUNGEN SCHIEF GEHT,
	3384	# WIRD FLAG GESETZT UND BLEIBT AUCH BEI WEITEREN ORDNUNGS-
	3385	# GEMAESSEN ARCHIVIERUNGEN GESETZT
	3386	if [[ "${actionout[$i]}" = ar ]]
	3387	then
	3388	if [[ $files_for_pes = false ]]
	3389	then
	3390	printf "\n >>> OUTPUT: ${localout[$i]} to"
	3391	printf "\n ${pathout[$i]}"
	3392	printf "\n File will be copied to archive-system ($archive_system) !\n"
	3393	else
	3394	printf "\n >>> OUTPUT: ${localout[$i]}/_.... to"
	3395	printf "\n ${pathout[$i]}"
	3396	printf "\n Directory will be copied to archive-system ($archive_system) !\n"
	3397	fi
	3398	mv ${localout[$i]} ${frelout[$i]}
	3399
	3400	file_saved=false
	3401
	3402	if [[ $archive_system = asterix ]]
	3403	then
	3404	do_stageout=true
	3405	(( stageout_anz = 0 ))
	3406	while [[ $do_stageout = true ]]
	3407	do
	3408	if [[ $files_for_pes = false ]]
	3409	then
	3410	stageout ${frelout[$i]} > STAGE_OUTPUT
	3411	else
	3412	stageout -t ${frelout[$i]} > STAGE_OUTPUT
	3413	fi
	3414	cat STAGE_OUTPUT
	3415	if [[ $(grep -c "st.msg:150" STAGE_OUTPUT) != 0 ]]
	3416	then
	3417	file_saved=true
	3418	do_stageout=false
	3419	else
	3420	if [[ $files_for_pes = false ]]
	3421	then
	3422	printf "\n +++ file ${frelout[$i]} could not be saved on archive-system"
	3423	else
	3424	printf "\n +++ directory ${frelout[$i]} could not be saved on archive-system"
	3425	fi
	3426	(( stageout_anz = stageout_anz + 1 ))
	3427	if (( stageout_anz == 10 ))
	3428	then
	3429	printf "\n +++ stoped after 10 unsuccessful tries!"
	3430	archive_save=false
	3431	do_stageout=false
	3432	else
	3433	printf "\n *** new try to store on archive after 15 min:"
	3434	sleep 900
	3435	fi
	3436	fi
	3437	done
	3438	elif [[ $archive_system = DMF ]]
	3439	then
	3440	if [[ $files_for_pes = false ]]
	3441	then
	3442	printf "\n +++ archiving of single files impossible on $archive_system!\n"
	3443	locat=DMF
	3444	exit
	3445	else
	3446	rm -rf $ARCHIVE/${frelout[$i]}
	3447	cp -r ${frelout[$i]} $ARCHIVE
	3448	fi
	3449	file_saved=true
	3450	elif [[ $archive_system = tivoli ]]
	3451	then
	3452	# ARCHIVIERUNG NUR PER BATCH-JOB MOEGLICH
	3453	# DATEI MUSS ZWISCHENZEITLICH INS TEMPORAERE DATENVERZEICHNIS
	3454	# GELEGT WERDEN
	3455	[[ ! -d $tmp_data_catalog ]] && mkdir -p $tmp_data_catalog
	3456	chmod g+rx $tmp_data_catalog
	3457	if [[ $files_for_pes = false ]]
	3458	then
	3459	ln -f ${frelout[$i]} $tmp_data_catalog/${frelout[$i]}
	3460	else
	3461	mkdir $tmp_data_catalog/${frelout[$i]}
	3462	ln -f ${frelout[$i]}/* $tmp_data_catalog/${frelout[$i]}
	3463	fi
	3464
	3465	# BATCH JOB GENERIEREN UND ABSCHICKEN; DATEI MUSS WIEDER
	3466	# AUS TEMPORAEREM DATENVERZEICHNIS ENTFERNT WERDEN
	3467	echo "cd $tmp_data_catalog" > archive_${frelout[$i]}
	3468	if [[ $files_for_pes = false ]]
	3469	then
	3470	# EVENTUELL NOCH VORHANDENE DATEI IM ARCHIV LOSCHEN
	3471	echo "rm -rf \$PERM/${frelout[$i]}" >> archive_${frelout[$i]}
	3472	echo "cp ${frelout[$i]} \$PERM/${frelout[$i]}" >> archive_${frelout[$i]}
	3473	echo "rm -rf ${frelout[$i]}" >> archive_${frelout[$i]}
	3474	else
	3475	echo "rm -rf \$PERM/${frelout[$i]}.tar" >> archive_${frelout[$i]}
	3476	echo "tar cvf \$PERM/${frelout[$i]}.tar ${frelout[$i]}" >> archive_${frelout[$i]}
	3477	echo "rm -rf ${frelout[$i]}" >> archive_${frelout[$i]}
	3478	fi
	3479
	3480	subjob -v -d -q cdata -X 0 -m 1000 -t 7200 -f $filetransfer_protocol archive_${frelout[$i]}
	3481	printf " Archiving of $tmp_data_catalog/${frelout[$i]} initiated (batch job submitted)\n"
	3482	file_saved=true
	3483
	3484	elif [[ $archive_system = ut ]]
	3485	then
	3486	# ARCHIVIERUNG NUR PER BATCH-JOB MOEGLICH
	3487	# DATEI MUSS ZWISCHENZEITLICH INS TEMPORAERE DATENVERZEICHNIS
	3488	# GELEGT WERDEN
	3489	[[ ! -d $tmp_data_catalog ]] && mkdir -p $tmp_data_catalog
	3490	chmod g+rx $tmp_data_catalog
	3491	if [[ $files_for_pes = false ]]
	3492	then
	3493	ln -f ${frelout[$i]} $tmp_data_catalog/${frelout[$i]}
	3494	else
	3495	mkdir $tmp_data_catalog/${frelout[$i]}
	3496	ln -f ${frelout[$i]}/* $tmp_data_catalog/${frelout[$i]}
	3497	fi
	3498
	3499	# BATCH JOB GENERIEREN UND ABSCHICKEN; DATEI MUSS WIEDER
	3500	# AUS TEMPORAEREM DATENVERZEICHNIS ENTFERNT WERDEN
	3501	echo "cd $tmp_data_catalog" > archive_${frelout[$i]}
	3502	if [[ $files_for_pes = false ]]
	3503	then
	3504	# EVENTUELL NOCH VORHANDENE DATEI IM ARCHIV LOSCHEN
	3505	echo "rm -rf \$UT/${frelout[$i]}" >> archive_${frelout[$i]}
	3506	echo "cp ${frelout[$i]} \$UT/${frelout[$i]}" >> archive_${frelout[$i]}
	3507	echo "rm -rf ${frelout[$i]}" >> archive_${frelout[$i]}
	3508	else
	3509	echo "rm -rf \$UT/${frelout[$i]}.tar" >> archive_${frelout[$i]}
	3510	echo "tar cvf \$UT/${frelout[$i]}.tar ${frelout[$i]}" >> archive_${frelout[$i]}
	3511	echo "rm -rf ${frelout[$i]}" >> archive_${frelout[$i]}
	3512	fi
	3513
	3514	subjob -v -c /pf/b/$usern/job_queue -d -q pp -X 0 -m 1000 -t 7200 -f $filetransfer_protocol archive_${frelout[$i]}
	3515	printf " Archiving of $tmp_data_catalog/${frelout[$i]} initiated (batch job submitted)\n"
	3516	file_saved=true
	3517
	3518	else
	3519	printf "\n +++ archive_system=\"$archive_system\" archiving impossible!"
	3520	archive_save=false
	3521	fi
	3522	if [[ $file_saved = true ]]
	3523	then
	3524	if [[ $files_for_pes = false ]]
	3525	then
	3526	cat /dev/null > ${pathout[$i]}
	3527	else
	3528	mkdir -p ${pathout[$i]}
	3529	fi
	3530	fi
	3531	fi
	3532
	3533	# APPEND AUF LOKALER MASCHINE
	3534	if [[ "${actionout[$i]}" = "a" ]]
	3535	then
	3536	printf "\n >>> OUTPUT: ${localout[$i]} append to ${pathout[$i]}\n"
	3537	cat ${localout[$i]} >> ${pathout[$i]}
	3538	fi
	3539
	3540	# KOPIEREN AUF LOKALER MASCHINE
	3541	# ES MUSS KOPIERT WERDEN, DA MOVE NICHT UEBER FILESYSTEM HINAUS MOEGLICH
	3542	if [[ "${actionout[$i]}" = "" && $files_for_pes = false ]]
	3543	then
	3544
	3545	# KOPIEREN AUF EINPROZESSORMASCHINE
[5]	3546	if [[ "${extout[$i]}" != " " && "${extout[$i]}" != "" ]]
[1]	3547	then
	3548	printf "\n >>> OUTPUT: ${localout[$i]} to ${pathout[$i]}.${extout[$i]}\n"
	3549	cp ${localout[$i]} ${pathout[$i]}.${extout[$i]}
	3550	else
	3551	printf "\n >>> OUTPUT: ${localout[$i]} to ${pathout[$i]}\n"
	3552	cp ${localout[$i]} ${pathout[$i]}
	3553	fi
	3554
	3555	elif [[ "${actionout[$i]}" = "" && $files_for_pes = true ]]
	3556	then
	3557
	3558	# DIE DEN PROZESSOREN EINES PARALLELRECHNERS ZUGEHOERIGEN
	3559	# DATEIEN WERDEN ERST IN EINEM KATALOG GESAMMELT UND DIESER
	3560	# WIRD DANN KOPIERT
	3561	# PER MOVE UMBENANNT WERDEN
	3562	printf "\n >>> OUTPUT: ${localout[$i]}/_.... to ${pathout[$i]}\n"
	3563	cp -r ${localout[$i]} ${pathout[$i]}
	3564
	3565	fi
	3566	fi
	3567	done
	3568	if (( i != 0 ))
	3569	then
	3570	if [[ $transfer_problems = true ]]
	3571	then
	3572	printf "\n$striche\n *** OUTPUT-files saved"
	3573	printf "\n +++ WARNING: some data transfers failed! \n"
	3574	else
	3575	printf "\n$striche\n *** all OUTPUT-files saved \n"
	3576	fi
	3577	fi
	3578
	3579
	3580	# EVENTUELL FOLGEJOB STARTEN
	3581	# DATEI CONTINUE_RUN MUSS VOM BENUTZERPROGRAMM AUS ERZEUGT WERDEN
	3582	if [[ -f CONTINUE_RUN ]]
	3583	then
	3584
	3585	if [[ $archive_save = true ]]
	3586	then
	3587
	3588	# ZUERST IN MRUN-AUFRUF OPTIONEN FUER FORTSETZUNGSLAUF, FUER
	3589	# STILLES ABARBEITEN (OHNE INTERAKTIVE RUECKFAGEN) UND FUER
	3590	# BATCH-BETRIEB (NUR WICHTIG, FALLS BATCH AUF LOKALER MASCHINE
	3591	# DURCHGEFUEHRT WERDEN SOLL) EINFUEGEN, FALLS NICHT BEREITS VOR-
	3592	# HANDEN
	3593	[[ $(echo $mc \| grep -c "\-C") = 0 ]] && mc="$mc -C"
	3594	[[ $(echo $mc \| grep -c "\-v") = 0 ]] && mc="$mc -v"
	3595	[[ $(echo $mc \| grep -c "\-b") = 0 ]] && mc="$mc -b"
	3596	if [[ $(echo $mc \| grep -c "#") != 0 ]]
	3597	then
	3598	mc=`echo $mc \| sed 's/#/f/g'`
	3599	fi
	3600
	3601
	3602	# JOB STARTEN
	3603	printf "\n\n *** initiating restart-run on \"$return_addres\" using command:\n"
	3604	echo " $mc"
	3605	printf "\n$striche\n"
	3606	if [[ $localhost != $fromhost ]]
	3607	then
	3608
	3609	if [[ $localhost = nech \|\| $localhost = ibmb \|\| $localhost = ibmh \|\| $localhost = ibms \|\| $localhost = lctit ]]
	3610	then
	3611	echo "*** ssh will be used to initiate restart-runs!"
	3612	echo " return_addres=\"$return_addres\" "
	3613	echo " return_username=\"$return_username\" "
	3614	if [[ $return_addres = 172.20.25.41 ]]
	3615	then
	3616	# WORKAROUND AUF SCIROCCO AM TIT
[33]	3617	print "PATH=\$PATH:$LOCAL_MRUN_PATH;export PALM_BIN=$LOCAL_MRUN_PATH;cd $LOCAL_PWD; $mc " \| ssh $return_addres -l $return_username
[1]	3618	else
[33]	3619	ssh $return_addres -l $return_username "PATH=\$PATH:$LOCAL_MRUN_PATH;export PALM_BIN=$LOCAL_MRUN_PATH;cd $LOCAL_PWD; $mc "
[1]	3620	fi
	3621	else
	3622	printf "\n +++ no restart mechanism available for host \"$localhost\" "
	3623	locat=restart; exit
	3624	fi
	3625
	3626	# WARTEN, DAMIT SICH RESTART JOB IN QUEUE EINREIHEN KANN, BEVOR
	3627	# DER AKTUELLE JOB ENDET
	3628	sleep 30
	3629
	3630	else
	3631
	3632	# BEI RECHNUNGEN AUF LOKALER MASCHINE KANN MRUN DIREKT AUFGE-
	3633	# RUFEN WERDEN
	3634	cd $LOCAL_PWD
	3635	eval $mc # ' MUESSEN AUSGEWERTET WERDEN
	3636	cd - > /dev/null
	3637	fi
	3638	printf "\n$striche\n *** restart-run initiated \n"
	3639
	3640
	3641	# EVENTUELL INPUT-DATEIEN, DIE VON TEMPORAEREM DATENVERZEICHNIS
	3642	# GEHOLT WORDEN SIND, LOESCHEN
	3643	(( i = 0 ))
	3644	while (( i < iin ))
	3645	do
	3646	(( i = i + 1 ))
	3647	if [[ "${got_tmp[$i]}" = true && $keep_data_from_previous_run = false ]]
	3648	then
	3649	rm -r $tmp_data_catalog/${frelin[$i]}
	3650	fi
	3651	done
	3652
	3653	else
	3654
	3655	printf "\n +++ no restart-run possible, since errors occured"
	3656	printf "\n during the archive process"
	3657	fi
	3658
	3659	fi
	3660
	3661
	3662
	3663	# EVTL. EMAIL-BENACHRICHTIGUNG UEBER ABGESCHLOSSENEN LAUF
	3664	if [[ "$email_notification" != "" ]]
	3665	then
	3666
	3667	if [[ $localhost != $fromhost ]]
	3668	then
	3669	if [[ -f CONTINUE_RUN ]]
	3670	then
	3671	echo "PALM restart run necessary" > email_text
	3672	echo "description header of actual run:" >> email_text
	3673	cat CONTINUE_RUN >> email_text
	3674	echo "mrun-command to restart:" >> email_text
	3675	echo "$mc" >> email_text
	3676	else
	3677	echo "PALM run with base filename \"$fname\" on host \"$localhost\" finished" > email_text
	3678	fi
	3679	mail $email_notification < email_text
	3680	printf "\n *** email notification sent to \"$email_notification\" "
	3681	fi
	3682	fi
	3683
	3684
	3685
	3686	# ALLE ARBEITEN BEENDET. TEMPORAERER KATALOG KANN GELOESCHT WERDEN
	3687	cd $HOME
	3688	[[ $delete_temporary_catalog = true ]] && rm -rf $TEMPDIR
	3689
	3690	else
	3691
	3692
	3693	# FALLS AUF REMOTE-MASCHINE GERECHNET WERDEN SOLL, WERDEN JETZT ENTSPRE-
	3694	# CHENDE AKTIONEN DURCHGEFUEHRT
	3695
	3696	# MRUN-BEFEHL FUER REMOTE-MASCHINE ZUSAMMENSTELLEN
	3697	mrun_com="$mrun_script_name -a $afname -c $config_file -d $fname -f $filetransfer_protocol -h $host -H $fromhost -m $memory -t $cpumax -q $queue -R $return_addres -U $return_username -u $remote_username"
	3698	[[ "$cpp_opts" != "" ]] && mrun_com=${mrun_com}" -D \"$cpp_opts\""
	3699	[[ $do_compile = true ]] && mrun_com=${mrun_com}" -s \"$source_list\""
	3700	[[ "$input_list" != "" ]] && mrun_com=${mrun_com}" -i \"$input_list\""
	3701	[[ $ignore_archive_error = true ]] && mrun_com=${mrun_com}" -I"
	3702	[[ $keep_data_from_previous_run = true ]] && mrun_com=${mrun_com}" -k"
	3703	[[ "$additional_conditions" != "" ]] && mrun_com=${mrun_com}" -K \"$additional_conditions\""
	3704	# [[ "$makefile" != "$source_path/Makefile" ]] && mrun_com=${mrun_com}" -M \"$makefile\""
	3705	[[ "$output_list" != "" ]] && mrun_com=${mrun_com}" -o \"$output_list\""
	3706	[[ "$read_from_config" = false ]] && mrun_com=${mrun_com}" -S"
	3707	[[ $do_trace = true ]] && mrun_com=${mrun_com}" -x"
	3708	[[ "$numprocs" != "" ]] && mrun_com=${mrun_com}" -X $numprocs"
	3709	if [[ $use_openmp = true ]]
	3710	then
	3711	mrun_com=${mrun_com}" -O"
	3712	[[ "$tasks_per_node" != "" ]] && mrun_com=${mrun_com}" -T $threads_per_task"
	3713	else
	3714	[[ "$tasks_per_node" != "" ]] && mrun_com=${mrun_com}" -T $tasks_per_node"
	3715	fi
	3716	[[ $store_on_archive_system = false ]] && mrun_com=${mrun_com}" -A"
	3717	[[ $package_list != "" ]] && mrun_com=${mrun_com}" -p \"$package_list\""
	3718	[[ $return_password != "" ]] && mrun_com=${mrun_com}" -P $return_password"
	3719	[[ $delete_temporary_catalog = false ]] && mrun_com=${mrun_com}" -B"
	3720	[[ $node_usage != default ]] && mrun_com=${mrun_com}" -n $node_usage"
	3721	if [[ $do_remote = true ]]
	3722	then
	3723	printf "\n>>>> MRUN-command on execution host:\n>>>> $mrun_com \n"
	3724	fi
	3725
	3726
	3727	# ZUSAMMENSTELLUNG DES JOBSCRIPTS AUF DATEI jobfile
	3728	jobfile=jobfile.$RANDOM
	3729
	3730
	3731	# TEMPORAERES VERZEICHNIS GENERIEREN UND NACH DORT WECHSELN
	3732	echo "mkdir $TEMPDIR" >> $jobfile
	3733	echo "cd $TEMPDIR" >> $jobfile
	3734
	3735
	3736	# EVENTUELL FEHLERVERFOLGUNG AKTIVIEREN
	3737	if [[ $do_trace = true ]]
	3738	then
	3739	echo "set -x" >> $jobfile
	3740	else
	3741	echo "set +vx" >> $jobfile
	3742	fi
	3743
	3744
	3745	# ABSPEICHERN DER QUELLTEXTE (NUR FALLS UEBERSETZT WERDEN SOLL)
	3746	# SOWIE GEGEBENENFALLS DES MAKEFILES
	3747	if [[ $do_compile = true ]]
	3748	then
	3749
	3750	source_catalog=SOURCES_FOR_RUN_$fname
	3751
	3752	# UNTERVERZEICHNIS FUER QUELLTEXTE UND MAKEFILE ANLEGEN
	3753	# MRUN WIRD DIESES VRZEICHNIS UEBER ENVIRONMENT VARIABLE
	3754	# MITGETEILT (UEBERSTEUERT ANGABE IN KONFIGURATIONSDATEI)
	3755	echo "mkdir SOURCES_FOR_RUN_$fname" >> $jobfile
	3756	echo "export SOURCES_COMPLETE=true" >> $jobfile
	3757	echo "cd SOURCES_FOR_RUN_$fname" >> $jobfile
	3758
	3759	for filename in $source_list
	3760	do
	3761	# ABDECKZEICHEN VERHINDERN, DASS ERSETZUNGEN ERFOLGEN
	3762	echo "cat > $filename << \"%END%\"" >> $jobfile
	3763	cat $source_catalog/$filename >> $jobfile
	3764	echo " " >> $jobfile
	3765	echo "%END%" >> $jobfile
	3766	echo " " >> $jobfile
	3767	done
	3768
	3769	if [[ "$use_makefile" = true ]]
	3770	then
	3771	# ABDECKZEICHEN VERHINDERN, DASS ERSETZUNGEN ERFOLGEN
	3772	echo "cat > Makefile << \"%END%\"" >> $jobfile
	3773	cat $source_catalog/Makefile >> $jobfile
	3774	echo " " >> $jobfile
	3775	echo "%END%" >> $jobfile
	3776	echo " " >> $jobfile
	3777	fi
	3778
	3779	echo "cd - > /dev/null" >> $jobfile
	3780
	3781	fi
	3782
	3783
	3784	# ABSPEICHERN DER KONFIGURATIONSDATEI
	3785	# ABDECKZEICHEN VERHINDERN, DASS ERSETZUNGEN ERFOLGEN
	3786	echo "cat > $config_file << \"%END%\"" >> $jobfile
	3787	cat $config_file >> $jobfile
	3788	echo "%END%" >> $jobfile
	3789	echo " " >> $jobfile
	3790
	3791
	3792	# ABSPEICHERN DER AKTUELLEN MRUN-VERSION
	3793	# ABDECKZEICHEN VERHINDERN, DASS ERSETZUNGEN ERFOLGEN
	3794	echo "cat > $mrun_script_name <<\"%END%\"" >> $jobfile
[22]	3795	cat ${PALM_BIN}/$mrun_script_name >> $jobfile
[1]	3796	echo "%END%" >> $jobfile
	3797	if [[ $host = lctit ]]
	3798	then
	3799	echo "sed 's/bin\/ksh/home2\/usr5\/mkanda\/pub\/ksh/' < $mrun_script_name > mrun_new" >> $jobfile
	3800	echo "mv mrun_new $mrun_script_name" >> $jobfile
	3801	fi
	3802	echo "chmod u+x $mrun_script_name" >> $jobfile
	3803	echo "execute_mrun=true" >> $jobfile
	3804	echo " " >> $jobfile
	3805
	3806	# EVTL. BENOETIGTE INPUT-DATEIEN PER FTP HOLEN ODER DEM JOB DIREKT
	3807	# MITGEBEN UND AUF DEM REMOTE-RECHNER IM BENUTZERVERZEICHNIS ABLEGEN
	3808	# FALLS DIESES NICHT EXISTIERT, WIRD VERSUCHT, DAS JEWEILS LETZTE
	3809	# UNTERVERZEICHNIS DES PFADNAMENS ANZULEGEN
	3810	if [[ $do_remote = true ]]
	3811	then
	3812	(( i = 0 ))
	3813	while (( i < iin ))
	3814	do
	3815	(( i = i + 1 ))
	3816	echo "[[ ! -d ${pathin[$i]} ]] && mkdir -p ${pathin[$i]}" >> $jobfile
	3817	if [[ "${transin[$i]}" = job ]]
	3818	then
	3819	echo "cat > ${remotepathin[$i]} <<\"%END%\"" >> $jobfile
	3820	eval cat ${pathin[$i]}/${frelin[$i]} >> $jobfile
	3821	echo " " >> $jobfile
	3822	echo "%END%" >> $jobfile
	3823	else
	3824	if [[ $filetransfer_protocol = ftp ]]
	3825	then
	3826	echo "ftpcopy -b -o -g -s $return_addres ${remotepathin[$i]} \"${pathin[$i]}\" ${frelin[$i]}" >> $jobfile
	3827	else
	3828	echo "batch_scp -b -o -g -s -u $return_username $return_addres ${remotepathin[$i]} \"${pathin[$i]}\" ${frelin[$i]}" >> $jobfile
	3829	fi
	3830	fi
	3831
	3832	# UEBERPRUEFEN, OB DATEI ANGELEGT WERDEN KONNTE
	3833	echo "if [[ \$? = 1 ]]" >> $jobfile
	3834	echo "then" >> $jobfile
	3835	echo " echo \" \" " >> $jobfile
	3836	echo " echo \"+++ file ${remotepathin[$i]} could not be created\" " >> $jobfile
	3837	echo " echo \" please check, if directory exists on $host!\" " >> $jobfile
	3838	echo " echo \"+++ MRUN will not be continued\" " >> $jobfile
	3839	echo " execute_mrun=false" >> $jobfile
	3840	echo "fi" >> $jobfile
	3841	done
	3842	fi
	3843
	3844	# ARBEITSKATALOG AUF DER LOKALEN MASCHINE FUER EVENTUELLE
	3845	# FORTSETZUNGSLAUEFE PER ENV-VARIABLE UEBERGEBEN
	3846	echo "LOCAL_PWD=$working_directory" >> $jobfile
	3847	echo "export LOCAL_PWD" >> $jobfile
	3848
	3849	# EBENSO LOKALEN MRUN-KATALOG UEBERGEBEN
[22]	3850	echo "LOCAL_MRUN_PATH=$PALM_BIN" >> $jobfile
[1]	3851	echo "export LOCAL_MRUN_PATH" >> $jobfile
	3852
	3853	# echo "for_subjob_to_do=\"rm -rf $TEMPDIR\" " >> $jobfile
	3854
	3855	# MRUN AUF ZIELRECHNER AUFRUFEN
	3856	echo "set -x" >> $jobfile
	3857	echo "[[ \$execute_mrun = true ]] && ./$mrun_com" >> $jobfile
	3858
	3859	echo 'ls -al; echo `pwd`' >> $jobfile
	3860	echo "cd \$HOME" >> $jobfile
	3861	echo "rm -rf $TEMPDIR" >> $jobfile
	3862
	3863
	3864
	3865
	3866	# JOB PER SUBJOB STARTEN
	3867	if [[ $silent = false ]]
	3868	then
	3869	printf "\n "
	3870	else
	3871	printf "\n\n"
	3872	fi
	3873
	3874	subjob $job_on_file -h $host -u $remote_username -q $queue -m $memory -N $node_usage -t $cpumax $XOPT $TOPT $OOPT -n $fname -f $filetransfer_protocol -v $jobfile
	3875	rm -rf $jobfile
	3876
	3877
	3878	fi # ENDE REMOTE-TEIL
	3879
	3880
	3881
	3882	# ENDE DER PROZEDUR

Note: See TracBrowser for help on using the repository browser.

Download in other formats:

| Impressum | ©Leibniz Universität Hannover |