Home

Context Navigation

source: palm/trunk/SCRIPTS/mrun @ 746

Last change on this file since 746 was 746, checked in by letzel, 13 years ago
'wiki' inserted into weblink path to error message database (message) support for Linux OS with German locale (mrun) properly report hosts on general Linux clusters (mrun)
Property svn:keywords set to `Id Rev`
File size: 175.9 KB

Rev	Line
[71]	1	#!/bin/ksh
[503]	2	# mrun - script for running PALM jobs
[169]	3	# $Id: mrun 746 2011-08-18 21:14:48Z letzel $
[1]	4
[503]	5	# Procedure for interactive/batch PALM runs (initial runs and automatic
	6	# restart runs)
[1]	7
[503]	8	# Last changes:
[1]	9	# 03/03/94 - Siggi - Entwicklungsbeginn
	10	# 21/03/94 - Siggi - Entwicklungsabschluss (Version 1.0)
	11	# 09/12/94 - Siggi - allexport unterbunden; statt dessen werden jetzt ein-
	12	# zelne Variable exportiert. Dies ist noetig, weil an-
	13	# sonsten irgendein Speicher ueberlaeuft und von mrun
	14	# aufgerufene Programme (z.B. stageout) mit Fehlern
	15	# abbrechen (too many arguments)
	16	# 20/06/95 - Siggi - noclobber muss explizit abgeschaltet, da jetzt
	17	# defaultmaessig im RRZN-Profile eingeschaltet
	18	# 10/06/97 - Siggi - Zusaetzliche Option -C bei cpp, damit // nicht
	19	# wegfallen
	20	# 25/04/00 - Siggi - Version 1.5
	21	# remote_addres und remote_user in return_addres bzw.
	22	# return_username umbenannt, return_username wird dem
	23	# mrun-Aufruf im Job ueber Option -U mitgegeben.
	24	# Neue Variable remote_username, die beim interaktiven
	25	# mrun-Aufruf zwingend ueber die neue Option -u oder
	26	# in der Konfigurationsdatei angegeben werden muss.
	27	# 08/02/01 - Siggi - Alle mrun-Meldungen ins englische uebersetzt,
	28	# Version 1.7
	29	# 05/05/02 - Siggi - Uebersetzungen mittels make-Mechanismus moeglich
	30	# 21/11/02 - Siggi - ENV-Variable XLFRTEOPTS wird vor Ausfuehrung auf
	31	# IBM gesetzt, um Record-Laenge der NAMELIST-Dateien,
	32	# zu bestimmen; Praeprozessoroptionen haben nun alle
	33	# die Form -Dtext=text, weil auf IBM sonst text durch
	34	# die leere Zeichenkette ersetzt wird
	35	# 01/08/03 - Siggi - Test of implementing dvrp+1PE on ibm
	36	# 07/01/04 - Siggi - additional preprocessor directive for ibm included
	37	# (-D$OMP=OMP) in order to avoid problems with
	38	# OMP_NUM_THREADS
	39	# 04/01/05 - Siggi - archiving on hanni and berni is done in separate
	40	# tar files, one for each node used
	41	# 07/01/04 - Siggi - old code for t3e, vpp and hpcs eliminated
	42	# 09/01/05 - Siggi - workaround for getting the IP address on gfdl3
	43	# 17/01/05 - Siggi - job for data transfer to local machine now within
	44	# job class c1 (HLRN)
	45	# 27/01/05 - Siggi - IP address workaround for gfdl3 removed
	46	# 28/01/05 - Siggi - tar-filenames on $PERM (HLRN) now include the
	47	# base filename and cycle number
	48	# 16/02/05 - Gerald - hababai validated
	49	# 14/03/05 - Siggi - abort on NEC after first runtime error (F_ERRCNT)
	50	# 29/03/05 - Marcus - berni*-en0 validated
	51	# 21/04/05 - Siggi - transfer-job-protocol for avs-data is no more stored
	52	# 24/04/05 - Siggi - netcdf support on lcmuk
	53	# 25/04/05 - Siggi - netcdf support on gfld3 (decalpha)
	54	# 11/05/05 - Siggi - files with non-numeric extensions (.xxx after cycle
	55	# number) can now be used within interactive runs
	56	# 12/05/05 - Siggi - netcdf support on ibm
	57	# 13/05/05 - Siggi - error in tar-filenames on $PERM (HLRN) removed
	58	# 18/05/05 - Siggi - netcdf support on nec
	59	# 24/05/05 - Siggi - netcdf support on ibms
	60	# 14/06/05 - Siggi - sleep interval after submit of restart job increased
	61	# to 30 seconds
	62	# 28/06/05 - Siggi - bora adjusted to new intel compiler (LD_LIBRARY_
	63	# PATH is temporarily set)
	64	# 07/09/05 - Siggi - setting of MP_EAGER_LIMIT switched off because it
	65	# caused warnings in the job protocol
	66	# 20/10/05 - Siggi - update of netcdf-version on decalpha (gfdl3)
	67	# 25/10/05 - Siggi - error in listing files for getting the cycle number
	68	# removed
	69	# 26/10/05 - Siggi - new paths for dvrp-library, transfer of catalogs
	70	# realized by file attribute trpe
	71	# 28/10/05 - Siggi - if existing, the file extension is output in case
	72	# of missing input file - no job abort in case of
	73	# missing input files with file extensions
	74	# (preliminary solution)
	75	# 31/10/05 - Siggi - data transfer from nech now within seperate job
	76	# 04/11/05 - Siggi - netcdf 3.6.0-p1 on ibmh/ibmb
	77	# 07/12/05 - Siggi - gallego and elephanta admitted
	78	# 30/12/05 - Siggi - gfdl5 (ibmy) admitted
	79	# 10/01/06 - Siggi - cpp directive for NetCDF 64bit support
	80	# 20/01/06 - Siggi - cpp directive for ibmy
	81	# 09/02/06 - Siggi - ibmy admitted for batch mode
	82	# 13/04/06 - Siggi - ostria admitted
	83	# 18/04/06 - Siggi - usage of OpenMP implemented (additional option -O)
	84	# 10/05/06 - Siggi - environment variable XLSMPOPTS set on IBM for
	85	# optimization of OpenMP runs (option stacksize is
	86	# necessary for large gridpoint numbers because
	87	# otherwise a segmentation fault occurs)
	88	# 23/05/05 - Siggi - lctit (SUN Fire X4600) admitted
	89	# 21/08/06 - Siggi - standard mrun path on scirocco is as on bora
	90	# 23/08/06 - Siggi - netcdf support for scirocco (notebook)
	91	# 20/09/06 - Marcus - add DVRP library on nech
	92	# 24/10/06 - Siggi - Environment variables for PALM steering are written
	93	# on local NAMELIST file ENVPAR
	94	# 24/11/06 - Siggi - levanto admitted, maestro switched to ifc 9.1
	95	# 28/11/06 - Siggi - call to interpret_config.x_levanto because of SuSe 10
	96	# 06/02/07 - Siggi - add DVRP library on lcmuk
	97	# 07/02/07 - Siggi - revision of source code handling, source code is
	98	# now expected in directory given by source_path and
	99	# not in the current working directory, source code
	100	# files provided in add_source_path are allways
	101	# compiled, source code to be compiled is allways
	102	# sampled in SOURCES_FOR_RUN_$fname which is a
	103	# subdirectory of the current working directory,
[5]	104	# -s TEST changed to -s WRITE_PERMIT or -s WP,
[1]	105	# adapted for RIAM (neck)
[5]	106	# bugfix for cycle number of output file (extout)
	107	# all hpmuk-related code removed
[14]	108	# 14/02/07 - Siggi - dvrp support for neck added
[21]	109	# 28/02/07 - Siggi - empty lines in configuration file are accepted
[26]	110	# mrun_path replaced by PALM_BIN,
	111	# all machines are calling interpret_config.x
[66]	112	# 14/03/07 - Siggi - fimm admitted, revision number added to terminal
	113	# output
[69]	114	# 16/03/07 - Siggi - adjustments for lctit
[78]	115	# 29/03/07 - Siggi - global revision transfered to batch job by new
	116	# option -G
[82]	117	# 30/03/07 - Siggi - compilation "by hand" removed, cpp-directives/options
	118	# + netcdf/dvrp-options are read from configuration
	119	# file, host identifier (local_host) is read from
	120	# config file, code related to ftp filetransfer
	121	# removed (incl. option -f)
[97]	122	# 19/06/07 - Siggi - time limit for cdata jobs increased
[102]	123	# 25/07/07 - Siggi - two sets of executables can be started on lcmuk,
	124	# if new option -Y (coupling) is given,
	125	# output of executables to aout_output removed,
	126	# messages are immediately written to stdout instead
[108]	127	# 03/08/07 - Marcus - add XOPT="-X $numprocs" for lcfimm
	128	# 09/08/07 - Marcus - workaround on lcfimm to propagate environment
	129	# variables out to the nodes in coupled mode -disabled-
	130	# 13/08/07 - Marcus - start local restart jobs per ssh on lcfimm
[109]	131	# 28/08/07 - Marcus - completely remove workaround on lcfimm to propagate
	132	# environment variables out to the nodes in coupled mode
[118]	133	# 15/10/07 - Siggi - Preliminary adjustments for lctit, based on Jin's
	134	# suggestions
[122]	135	# 19/10/07 - Marcus - further adjustments for lctit: add new optional
	136	# argument -g group_number, admit all sla* node_usage
	137	# queues
[129]	138	# 30/10/07 - Marcus - further adjustments for queues on lctit
[149]	139	# 14/02/08 - Bjoern - link of restart files with long file name possible
	140	# 28/02/08 - Marcus - If environment variable link_local_output is set to
	141	# true, mrun tries "ln -f" on local output and resorts
	142	# to "cp" or "cp -r" on error
[164]	143	# 15/04/08 - Siggi - argument -c introduced to most of the subjob calls,
	144	# which allows the user to choose his own job catalog
	145	# by setting job_catalog in the configuration file
	146	# (default is ~/job_queue),
[172]	147	# workaround for mpiexec with -env option,
[164]	148	# adjustments for lcxt4 (Bergen Center for Computational
	149	# Science)
[183]	150	# 22/05/08 - Marcus - If environment variable link_local_input is set to
	151	# true, mrun tries "ln -f" on local input and resorts
	152	# to "cp" or "cp -r" on error
[180]	153	# 27/05/08 - Siggi - PATH is set to PALM_BIN everywhere (missing so far)
[179]	154	# 14/07/08 - Siggi - adjustments for lcsgih
[188]	155	# 08/08/08 - Marcus - typo removed in lcxt4 branch
[204]	156	# 17/09/08 - Siggi - restart mechanism adjusted for lcsgi
[210]	157	# 02/10/08 - BjornM - argument "-Y" modified, adjustments for coupled runs
	158	# 21/10/08 - Siggi - bugfix for the case that -K has more than one
	159	# argument
[211]	160	# 08/11/08 - Siggi - bugfix for depository path settings in configuration
	161	# file
	162	# 11/11/08 - Siggi - rules for using user code files changed: user
	163	# code splitted into one file per subroutine,
	164	# user can provide his/her own makefile, only
	165	# default source-code filenames are allowed
[215]	166	# 14/11/08 - Siggi - cond1 and cond2 are part of the depository name
[221]	167	# 08/01/09 - Siggi - mpi module on sgi machines to be used can be set
	168	# with environment variable mpilib
[223]	169	# 13/01/09 - Siggi - totalview on hice/bice can be used in debug mode
	170	# using environment variable totalview
[246]	171	# 25/02/09 - Siggi - cputime-option (-t) is allowed to be omitted for
	172	# interactive runs
[253]	173	# 05/03/09 - Siggi - adjustments for new NEC-SX9 at RIAM (necriam)
[260]	174	# 16/03/09 - Siggi - dvrp_lib, dvrp_inc replaced by dvr_lib, dvr_inc,
	175	# dvr streaming server is automatically started using
	176	# new variable dvr_server and a configuration file
	177	# .dvrserver.config,
[261]	178	# processing of dvr output files using new script
	179	# process_dvr_output implemented, in order to make
	180	# dvr application more user friendly
[265]	181	# 20/03/09 - Marcus - update of n1ge command for lctit
[291]	182	# 16/04/09 - Siggi - new option -y for precursor (uncoupled) ocean runs
	183	# to be followed by a coupled atmosphere-ocean run
[343]	184	# 21/04/09 - Siggi - adjustments for new IBM at DKRZ, which is now ibmh
	185	# 24/06/09 - BjornM - adjustments for coupled/precursor runs on ibmy
[352]	186	# 08/07/09 - Siggi - default value for email_notification is none,
	187	# option -e added to subjob
[359]	188	# 19/08/09 - Marcus - MPI debug option
[362]	189	# 21/08/09 - Marcus - totalview on hice/bice also available for mvapich2
[367]	190	# 25/08/09 - BjornM - adpated for lck
[369]	191	# 26/08/09 - Siggi - additional environment variables for mpt on ice2
[372]	192	# 26/08/09 - Marcus - adjustment for ice2 on lcsgib
[397]	193	# 29/09/09 - Siggi - default value for archiving is false, option "-A"
	194	# switches on archiving, archiving on SGI-ICE enabled
[416]	195	# 16/10/09 - Carolin- adjustments for archiving on SGI-ICE of binary files;
	196	# adjustment for special1q
	197	# 18/12/09 - Carolin- modification of archiving (permq)
[440]	198	# 01/02/10 - Siggi - adapted for lcxt5m and lckyoto (Fujitsu HX600)
[475]	199	# 03/02/10 - Siggi - make options (mopts) to be set by configuration file
	200	# implemented
[493]	201	# 08/02/10 - Siggi - loading of modules now controlled via configuration
	202	# file (modules)
[503]	203	# 02/03/10 - Siggi - for each block in the configuration file, separate
	204	# utility programs are used, combine_plot_fields is
	205	# directly called and must not be given by an output
	206	# command in the configuration file any more
[540]	207	# 15/06/10 - Rieke - ssh from compute nodes to login nodes on hosts
	208	# lcsgih/lcsgib is done using the names
	209	# hicegate0/bicegate0 instead of the IP addresses
[544]	210	# 17/08/10 - BjornM - adjustments for interactive runs on lcxt4
[555]	211	# 07/09/10 - Siggi - bugfix for wrong netcdf/3.6.3 module on lcsgi
[622]	212	# 08/12/10 - Siggi - new handling of openmp/hybrid runs, option -O
	213	# has now argument threads_per_task
	214	# adjustments for Kyushu Univ. (lcrte, ibmku)
[635]	215	# 14/12/10 - Siggi - adjustments for new Tsubame system at Tokyo
	216	# institute of technology (lctit)
[669]	217	# 23/12/10 - Micha - different number of processors in ocean and atmosphere
	218	# is now allowed
[678]	219	# 02/02/10 - Siggi - further adjustments on Tsubame and concerning openMP
	220	# usage
[696]	221	# 09/03/10 - Siggi - adjustments for ibmkisti, mpt bugfix for netCDF4
[693]	222	# usage, totalview usage extended
[721]	223	# 17/03/11 - Siggi - adjustments for openmp usage on ibmkisti
	224	# 03/04/11 - Micha - added lckordi
	225	# 06/04/11 - BjornM - bugfix for runs with mpt on lcsgi
[738]	226	# 17/08/11 - Siggi - extensions for impi library
[741]	227	# 18/08/11 - Siggi - bugfix for local append of output files with suffix (.nc)
[746]	228	# 18/08/11 - Marcus - support for Linux OS with German locale
	229	# - properly report hosts on general Linux clusters
[437]	230
	231
[1]	232
	233	# VARIABLENVEREINBARUNGEN + DEFAULTWERTE
	234
	235	set +o allexport # SICHERHEITSHALBER UNTERBINDEN, DA SONST EVTL. STAGEOUT
	236	# NICHT LAUEFT (TOO MANY ARGUMENTS - PROBLEM)
	237	set +o noclobber # EXISTIERENDE DATEIEN DUERFEN UEBERSCHRIEBEN WERDEN
	238
	239	AddFilenames=""
	240	additional_conditions=""
	241	add_source_path=""
	242	afname=""
	243	archive_save=true
	244	archive_system=none
	245	compiler_name=""
	246	cond1=""
	247	cond2=""
	248	config_file=.mrun.config
[206]	249	coupled_dist=""
	250	coupled_mode="mpi1"
[1]	251	cpp_opts=""
[82]	252	cpp_options=""
[1]	253	cpumax=0
	254	cpurest=0
	255	delete_temporary_catalog=true
	256	do_batch=false
	257	do_compile=true
	258	do_remote=false
	259	do_stagein=true
	260	do_stageout=true
	261	do_trace=false
[352]	262	email_notification="none"
[1]	263	exclude=""
	264	executable=""
	265	execution_error=false
[67]	266	fimm=false
[1]	267	fname=test
	268	fromhost=""
[78]	269	global_revision=""
[122]	270	group_number=none
[1]	271	host=""
	272	host_file=""
	273	hp=""
	274	ignore_archive_error=false
	275	input_list=""
	276	interpreted_config_file=""
[164]	277	job_catalog="~/job_queue"
[1]	278	job_on_file=""
	279	keep_data_from_previous_run=false
[183]	280	link_local_input=false
[149]	281	link_local_output=false
[1]	282	localhost_realname=$(hostname)
	283	local_compile=false
[260]	284	local_dvrserver_running=.FALSE.
[1]	285	locat=normal
	286	mainprog=""
	287	makefile=""
	288	mc=$0
	289	while [[ $(echo $mc \| grep -c "/") != 0 ]]
	290	do
	291	mc=`echo $mc \| cut -f2- -d"/"`
	292	done
[493]	293	module_calls=""
[1]	294	mrun_script_name=$mc
[82]	295	netcdf_inc=""
	296	netcdf_lib=""
[1]	297	netcdf_support=false
	298	node_usage=default
	299	numprocs=""
[206]	300	numprocs_atmos=0
	301	numprocs_ocean=0
[1]	302	OOPT=""
	303	openmp=false
	304	output_list=""
	305	package_list=""
	306	punkte="..........................................................."
	307	queue=none
	308	read_from_config=""
[5]	309	restart_run=false
[622]	310	if [[ `hostname` = rte10 ]]
[1]	311	then
[622]	312	return_addres=133.5.185.60
[693]	313	echo "+++ WARNING: fixed return_addres = $return_addres is used !!!!!"
	314	elif [[ `hostname` = climate0 ]]
	315	then
	316	return_addres=165.132.26.68
	317	echo "+++ WARNING: fixed return_addres = $return_addres is used !!!!!"
[622]	318	else
	319	return_addres=$(nslookup `hostname` 2>&1 \| grep "Address:" \| tail -1 \| awk '{print $2}')
[1]	320	fi
	321	return_password=""
	322	return_username=$LOGNAME
	323	remotecall=false
	324	remote_username=""
[102]	325	run_coupled_model=false
[1]	326	run_mode=""
	327	scirocco=false
[397]	328	store_on_archive_system=false
[1]	329	striche=" ----------------------------------------------------------------------------"
	330	silent=false
	331	source_list=""
	332	source_path=SOURCE
	333	tasks_per_node=""
	334	threads_per_task=1
	335	tmpcreate=false
	336	tmp_data_catalog=""
	337	transfer_problems=false
	338	usern=$LOGNAME
[377]	339	use_openmp=false
[1]	340	working_directory=`pwd`
	341	TOPT=""
	342	XOPT=""
	343	zeit=$( date \| cut -c 12-19 )
	344
	345	typeset -i iec=0 iic=0 iin=0 ioc=0 iout=0 memory=0 stagein_anz=0 stageout_anz=0
[206]	346	typeset -i cputime i ii iia iii iio icycle inode ival jobges jobsek maxcycle minuten nodes pes sekunden tp1
[1]	347
[66]	348	typeset -R30 calltime
[1]	349	typeset -L20 spalte1
	350	typeset -L40 spalte2
	351	typeset -L60 spalte3
	352	typeset -L35 string1=`date`
	353	typeset -L12 string2=$usern
	354	typeset -L12 string3=$localhost_realname
	355	typeset -L12 string4
	356	typeset -L12 string5
[201]	357	typeset -L30 version="MRUN 2.0 Rev$Rev: 746 $"
[1]	358
	359
	360
	361	# EINZELNE VARIABLE FUER HAUPTPROGRAMM EXPORTIEREN
	362	export cpurest fname host localhost return_addres return_username remotecall tasks_per_node
	363
	364	# FOLGENDE VARIABLEN MUESSEN FUER DIE INTERPRETATION DER KONFIGURATIONSDATEI
	365	# EXPORTIERT WERDEN
	366	export afname config_file cpp_opts cpumax do_batch do_trace fname fromhost
[127]	367	export group_number input_list memory numprocs output_list queue run_mode
[1]	368
	369
	370
	371	# FEHLERBEHANDLUNG
	372	# BEI EXIT:
[83]	373	trap 'rm -rf $working_directory/tmp_mrun
[69]	374	if [[ $locat != localhost ]]
[1]	375	then
[22]	376	# if [[ ! -f ${mrun_path}/statistik/mrun_statistik ]]
	377	# then
	378	# cat > ${mrun_path}/statistik/mrun_statistik << %STATEND%
	379	#MRUN-calls on $localhost
	380	#
	381	#date and time user localhost remotehost termination mrun-command
	382	#--------------------------------------------------------------------------------------------------------------------
	383	#%STATEND%
	384	# chmod 666 ${mrun_path}/statistik/mrun_statistik
	385	# fi
	386	#
	387	# # EINTRAG IN DIE STATISTIK-DATEI
	388	# string4=$host
	389	# string5=$locat
	390	# if [[ "$job_on_file" = "" && $locat != control_c && $locat != user_abort ]]
	391	# then
	392	# if [[ $do_batch = true ]]
	393	# then
	394	# printf "$string1$string2$string3$string4$string5$mrun_com \n" >> ${mrun_path}/statistik/mrun_statistik
	395	# else
	396	# printf "$string1$string2$string3$string4$string5$mc \n" >> ${mrun_path}/statistik/mrun_statistik
	397	# fi
	398	# fi
[69]	399	echo " " > /dev/null
[1]	400	fi
	401
	402	if [[ $locat != normal && $locat != control_c && $locat != local_compile ]]
	403	then
	404
	405	# EVENTUELLE ERROR-KOMMANDOS ABARBEITEN
	406	(( i = 0 ))
	407	while (( i < iec ))
	408	do
	409	(( i = i + 1 ))
	410	printf "\n *** Execution of ERROR-command:\n"
	411	printf " >>> ${err_command[$i]}\n"
	412	eval ${err_command[$i]}
	413	done
	414	if [[ -n $interpreted_config_file ]] then
	415	rm -rf $interpreted_config_file
	416	fi
[71]	417	if [[ -n .mrun_environment ]] then
	418	rm -rf .mrun_environment
	419	fi
[1]	420	if [[ $tmpcreate = true ]]
	421	then
	422	printf "\n *** Contents of \"$TEMPDIR\":\n"
	423	ls -al; cd
	424	[[ $delete_temporary_catalog = true ]] && rm -rf $TEMPDIR
	425	fi
[260]	426	if [[ "$dvrserver_id" != "" ]]
	427	then
	428	echo "+++ killing dvrserver_id=$dvrserver_id"
	429	kill $dvrserver_id
	430	fi
[1]	431	if [[ -f ~/job_queue/JOBINFO.$QSUB_REQID ]]
	432	then
	433	rm -rf ~/job_queue/JOBINFO.$QSUB_REQID
	434	fi
	435	printf "\n\n+++ MRUN killed \n\n"
	436	elif [[ $locat != control_c ]]
	437	then
	438	printf "\n\n --> all actions finished\n\n"
	439	printf " Bye, bye $usern !!\n\n"
	440	fi' exit
	441
	442
	443	# BEI TERMINAL-BREAK:
[83]	444	trap 'rm -rf $working_directory/tmp_mrun
[69]	445	[[ $tmpcreate = true ]] && (cd; rm -rf $TEMPDIR)
[1]	446	if [[ -f ~/job_queue/JOBINFO.$QSUB_REQID ]]
	447	then
	448	rm -rf ~/job_queue/JOBINFO.$QSUB_REQID
	449	fi
[260]	450	if [[ "$dvrserver_id" != "" ]]
	451	then
	452	echo "+++ killing dvrserver_id=$dvrserver_id"
	453	kill $dvrserver_id
	454	fi
[1]	455	printf "\n+++ MRUN killed by \"^C\" \n\n"
	456	locat=control_c
	457	exit
	458	' 2
	459
	460
[22]	461	# CHECK IF THE PATH FOR THE PALM BINARIES (SCRIPTS+UTILITY-PROGRAMS) HAS
	462	# BEEN SET
	463	if [[ "$PALM_BIN" = "" ]]
	464	then
	465	printf "\n +++ environment variable PALM_BIN has not been set"
	466	printf "\n please set it to the directory where the PALM scripts are located"
	467	locat=palm_bin; exit
	468	fi
[180]	469	export PATH=$PALM_BIN:$PATH
[22]	470
	471
[1]	472
	473	# SHELLSCRIPT-OPTIONEN EINLESEN UND KOMMANDO NEU ZUSAMMENSETZEN, FALLS ES
	474	# FUER FOLGEJOBS BENOETIGT WIRD
[622]	475	while getopts :a:AbBc:Cd:D:Fg:G:h:H:i:IkK:m:M:n:o:O:p:P:q:r:R:s:St:T:u:U:vxX:yY: option
[1]	476	do
	477	case $option in
	478	(a) afname=$OPTARG;;
[397]	479	(A) store_on_archive_system=true; mc="$mc -A";;
[1]	480	(b) do_batch=true; mc="$mc -b";;
	481	(B) delete_temporary_catalog=false; mc="$mc -B";;
	482	(c) config_file=$OPTARG; mc="$mc -c$OPTARG";;
[5]	483	(C) restart_run=true; mc="$mc -C";;
[1]	484	(d) fname=$OPTARG; mc="$mc -d$OPTARG";;
[210]	485	(D) cpp_opts="$cpp_opts $OPTARG"; mc="$mc -D'$OPTARG'";;
[79]	486	(F) job_on_file="-D"; mc="$mc -F";;
[122]	487	(g) group_number=$OPTARG; mc="$mc -g$OPTARG";;
[78]	488	(G) global_revision=$OPTARG; mc="$mc -G'$OPTARG'";;
[1]	489	(h) host=$OPTARG; mc="$mc -h$OPTARG";;
	490	(H) fromhost=$OPTARG; mc="$mc -H$OPTARG";;
	491	(i) input_list=$OPTARG; mc="$mc -i'$OPTARG'";;
	492	(I) ignore_archive_error=true; mc="$mc -I";;
	493	(k) keep_data_from_previous_run=true; mc="$mc -k";;
[210]	494	(K) additional_conditions="$OPTARG"; mc="$mc -K'$OPTARG'";;
[1]	495	(m) memory=$OPTARG; mc="$mc -m$OPTARG";;
	496	(M) makefile=$OPTARG; mc="$mc -M$OPTARG";;
	497	(n) node_usage=$OPTARG; mc="$mc -n$OPTARG";;
	498	(o) output_list=$OPTARG; mc="$mc -o'$OPTARG'";;
[622]	499	(O) use_openmp=true; threads_per_task=$OPTARG; mc="$mc -O$OPTARG";;
[1]	500	(p) package_list=$OPTARG; mc="$mc -p'$OPTARG'";;
	501	(P) return_password=$OPTARG; mc="$mc -P$OPTARG";;
	502	(q) queue=$OPTARG; mc="$mc -q$OPTARG";;
	503	(r) run_mode=$OPTARG; mc="$mc -r'$OPTARG'";;
	504	(R) remotecall=true;return_addres=$OPTARG; mc="$mc -R$OPTARG";;
	505	(s) source_list=$OPTARG; mc="$mc -s'$OPTARG'";;
	506	(S) read_from_config=false; mc="$mc -S";;
	507	(t) cpumax=$OPTARG; mc="$mc -t$OPTARG";;
	508	(T) tasks_per_node=$OPTARG; mc="$mc -T$OPTARG";;
	509	(u) remote_username=$OPTARG; mc="$mc -u$OPTARG";;
	510	(U) return_username=$OPTARG; mc="$mc -U$OPTARG";;
	511	(v) silent=true; mc="$mc -v";;
	512	(x) do_trace=true;set -x; mc="$mc -x";;
	513	(X) numprocs=$OPTARG; mc="$mc -X$OPTARG";;
[291]	514	(y) ocean_file_appendix=true; mc="$mc -y";;
[206]	515	(Y) run_coupled_model=true; coupled_dist=$OPTARG; mc="$mc -Y'$OPTARG'";;
[1]	516	(\?) printf "\n +++ unknown option $OPTARG \n"
	517	printf "\n --> type \"$0 ?\" for available options \n"
	518	locat=parameter;exit;;
	519	esac
	520	done
	521
	522
	523	# EVTL. POSITIONSPARAMETER EINLESEN
	524	# ZUR ZEIT GIBT ES NUR DEN PARAMETER ? (=KURZINFO)
	525	shift OPTIND-1
	526
	527
	528	# KURZE AUFRUFBESCHREIBUNG WIRD HIER AUSGEGEBEN
	529	if [[ "$1" = "?" ]]
	530	then
	531	(printf "\n *** mrun can be called as follows:\n"
[291]	532	printf "\n $mrun_script_name -b -c.. -d.. -D.. -f.. -F -h.. -i.. -I -K.. -m.. -o.. -p.. -r.. -R -s.. -t.. -T.. -v -x -X.. -y -Y.. <modus>\n"
[1]	533	printf "\n Description of available options:\n"
	534	printf "\n Option Description Default-Value"
	535	printf "\n -a base name of input files equiv. -d"
[400]	536	printf "\n -A archiving when using file-attribute fl"
[1]	537	printf "\n -b batch-job on local machine ---"
	538	printf "\n -B do not delete temporary directory at end ---"
	539	printf "\n -c configuration file .mrun.config"
	540	printf "\n -d base name of files attached to program test"
	541	printf "\n -D preprocessor(cpp)-directives \"\" "
	542	printf "\n -F create remote job file only ---"
	543	printf "\n -h execution host $localhost_realname"
	544	printf "\n -i INPUT control list \"\" "
	545	printf "\n -I archiving errors of previous batch-jobs"
	546	printf "\n will be ignored"
	547	printf "\n -k keep data from previous run"
	548	printf "\n -K additional conditions for controling"
	549	printf "\n usage of conditional code and"
	550	printf "\n env-variables in configuration file \"\" "
	551	printf "\n -m memory demand in MB (batch-jobs) 0 MB"
	552	printf "\n -M Makefile name Makefile"
	553	printf "\n -n node usage (shared/not_shared) depending on -h"
	554	printf "\n -o OUTPUT control list \"\" "
	555	printf "\n -O use OpenMP ---"
	556	printf "\n -p software package list \"\" "
	557	printf "\n -q queue \"$queue\" "
	558	printf "\n -r run control list (combines -i -o) \"\" "
	559	printf "\n -s filenames of routines to be compiled \"\" "
	560	printf "\n must end with .f, .f90, .F, or .c !"
	561	printf "\n use \"..\" for more than one file and wildcards"
	562	printf "\n -s TEST compiles all files with w-permit"
	563	printf "\n -S config file interpreted by shellscript ---"
	564	printf "\n -t allowed cpu-time in seconds (batch) 0"
	565	printf "\n -T tasks per node depending on -h"
	566	printf "\n -u username on remote machine \"\" "
	567	printf "\n -v no prompt for confirmation ---"
	568	printf "\n -x tracing of mrun for debug purposes ---"
	569	printf "\n -X # of processors (on parallel machines) 1"
[291]	570	printf "\n -y add appendix \"_O\" to all local output"
	571	printf "\n files (ocean precursor runs followed by"
	572	printf "\n coupled atmosphere-ocean runs) ---"
[206]	573	printf "\n -Y run coupled model, \"#1 #2\" with"
	574	printf "\n #1 atmosphere and #2 ocean processors \"#/2 #/2\" depending on -X"
[1]	575	printf "\n "
	576	printf "\n Possible values of positional parameter <modus>:"
	577	printf "\n \"?\" - this outline \n\n") \| more
	578	exit
	579	elif [[ "$1" != "" ]]
	580	then
	581	printf "\n +++ positional parameter $1 unknown \n"
	582	locat=parameter; exit
	583	fi
	584
	585
	586
	587	# KURZE STARTMELDUNG
	588	printf "\n*** $version "
	589	printf "\n will be executed. Please wait ..."
	590
	591
[82]	592
	593	# PRUEFEN, OB KONFIGURATIONS-DATEI VORHANDEN
	594	if [[ ! -f $config_file ]]
	595	then
	596	printf "\n\n +++ configuration file: "
	597	printf "\n $config_file"
	598	printf "\n does not exist"
	599	locat=connect; exit
	600	fi
	601
	602
	603	# HOST-IDENTIFIER (local_host) AUS KONFIGURATIONSDATEI BESTIMMEN
	604	line=""
	605	grep "%host_identifier" $config_file > tmp_mrun
	606	while read line
	607	do
[206]	608	if [[ "$line" != "" && $(echo $line \| cut -c1) != "#" ]]
[82]	609	then
	610	HOSTNAME=`echo $line \| cut -d" " -s -f2`
	611	host_identifier=`echo $line \| cut -d" " -s -f3`
[83]	612	if [[ $localhost_realname = $HOSTNAME ]]
[82]	613	then
	614	localhost=$host_identifier
	615	break
	616	fi
	617	fi
	618	done < tmp_mrun
	619
	620	if [[ "$localhost" = "" ]]
	621	then
	622	printf "\n\n +++ no host identifier found in configuration file \"$config_file\""
	623	printf "\n for local host \"$localhost_realname\"."
	624	printf "\n Please add line"
	625	printf "\n \"\%host_identifier $localhost_realname <identifier>\""
	626	printf "\n to the configuration file."
	627	locat=localhost; exit
	628	fi
	629
	630
	631	# HOSTSPEZIFISCHE VARIABLEN SETZEN
	632	case $localhost_realname in
[397]	633	(r1\|r2\|h01\|b01) archive_system=tivoli;;
[82]	634	(cs*) archive_system=ut;;
	635	(fimm.bccs.uib.no) fimm=true;;
	636	(gate\|n-sx) PATH=$PALM_BIN:$PATH:/usr/bin/nqsII;;
	637	(scirocco) scirocco=true;;
	638	esac
	639
	640
	641
[1]	642	# BASISNAME DER INPUT-DATEIEN GLEICH ALLGEMEINEM BASISNAMEN SETZEN,
	643	# WENN NICHT VOM BENUTZER ANDERS BESTIMMT
	644	[[ "$afname" = "" ]] && afname=$fname
	645
	646
	647	# EVTL. RUN-MODUS DEN I/O-LISTEN HINZUFUEGEN
	648	if [[ "$run_mode" != "" ]]
	649	then
	650	input_list="$input_list $run_mode"
	651	output_list="$output_list $run_mode"
	652	fi
	653
	654
	655	# RECHNERNAMEN ABSPEICHERN, VON DEM AUS JOB GESTARTET WIRD,
	656	# ALLERDINGS NUR DANN, WENN NICHT PER OPTION -H BEREITS EIN WERT
	657	# ZUGEWIESEN WURDE (MRUN MACHT DIES IMMER, WENN ES SELBST BATCH-JOBS
	658	# STARTET)
	659	if [[ "$fromhost" = "" ]]
	660	then
	661	fromhost=$localhost
	662	fi
	663
	664
	665	# PRUEFEN, OB MRUN NUR TESTWEISE AUF DER LOKALEN MASCHINE KOMPILIEREN SOLL
	666	if [[ "$source_list" = LOCAL_COMPILE_TEST ]]
	667	then
	668	source_list=TEST
	669	local_compile=true
	670	host=$localhost
	671	fi
	672
	673
	674	# PRUEFEN, OB AUF REMOTE-MASCHINE GERECHNET WERDEN SOLL
	675	# WERT VON do_remote WIRD FUER DATEIVERBINDUNGEN BENOETIGT.
	676	# WENN AUF REMOTE-MASCHINE GERECHNET WIRD, IST GLEICHZEITIG KLAR,
	677	# DASS EIN BATCH-JOB GESTARTET WERDEN MUSS
	678	if [[ -n $host && "$host" != $localhost ]]
	679	then
	680	do_batch=true
	681	do_remote=true
	682	case $host in
[717]	683	(ibm\|ibmh\|ibmkisti\|ibmku\|ibms\|ibmy\|nech\|necriam\|lckyoto\|lcsgib\|lcsgih\|unics\|lcxt4\|lcxt5m\|lck\|lckordi) true;;
[1]	684	(*) printf "\n"
[83]	685	printf "\n +++ sorry: execution of batch jobs on remote host \"$host\""
	686	printf "\n is not available"
[1]	687	locat=nqs; (( iec = 0 )); exit;;
	688	esac
	689	else
	690	host=$localhost
	691	fi
	692
	693
	694	# ZUSATZBEDINGUNGEN (OPTION -K) AUSWERTEN
	695	if [[ -n $additional_conditions ]]
	696	then
[69]	697	# echo $additional_conditions \| cut -d" " -f1-3 \| read cond1 cond2 dummy
	698	cond1=`echo $additional_conditions \| cut -d" " -f1`
	699	cond2=`echo $additional_conditions \| cut -d" " -s -f2`
	700	dummy=`echo $additional_conditions \| cut -d" " -s -f3`
[1]	701	if [[ -n $dummy ]]
	702	then
	703	printf "\n +++ more than 2 additional conditions given for Option \"-K\""
	704	locat=options; exit
	705	fi
[503]	706	block=_$cond1
	707	[[ -n $cond2 ]] && block=${block}_$cond2
[1]	708	fi
	709
	710
[206]	711	# KOPPLUNGSEIGENSCHAFTEN (-Y) AUSWERTEN UND coupled_mode BESTIMMEN
	712	if [[ $run_coupled_model = true ]]
	713	then
	714
	715	if [[ -n $coupled_dist ]]
	716	then
	717
	718	numprocs_atmos=`echo $coupled_dist \| cut -d" " -s -f1`
	719	numprocs_ocean=`echo $coupled_dist \| cut -d" " -s -f2`
	720
	721	if (( $numprocs_ocean + $numprocs_atmos != $numprocs ))
	722	then
	723
	724	printf "\n +++ number of processors does not fit to specification by \"-Y\"."
	725	printf "\n PEs (total) : $numprocs"
	726	printf "\n PEs (atmosphere): $numprocs_atmos"
	727	printf "\n PEs (ocean) : $numprocs_ocean"
	728	locat=coupling; exit
	729
	730	fi
	731
	732	else
	733
	734	(( numprocs_ocean = $numprocs / 2 ))
	735	(( numprocs_atmos = $numprocs / 2 ))
	736
	737	fi
	738	coupled_dist=`echo "$numprocs_atmos $numprocs_ocean"`
	739
	740	# GET coupled_mode FROM THE CONFIG FILE
	741	line=""
	742	grep "%cpp_options.-D__mpi2.$host" $config_file > tmp_mrun
	743	while read line
	744	do
	745	if [[ "$line" != "" && $(echo $line \| cut -c1) != "#" && ( $(echo $line \| cut -d" " -s -f4) = $cond1 \|\| $(echo $line \| cut -d" " -s -f4) = $cond2 ) ]]
	746	then
	747	coupled_mode="mpi2"
	748	fi
	749	done < tmp_mrun
	750
	751	fi
	752
	753
[1]	754	# PRUEFEN, OB EVTL. BEI VORHERGEHENDEM LAUF (KETTENJOB) EINE
	755	# ARCHIVIERUNG FEHLGESCHLAGEN IST
	756	if [[ -f ~/job_queue/ARCHIVE_ERROR_$fname ]]
	757	then
	758	if [[ $ignore_archive_error = false ]]
	759	then
	760	printf "\n +++ data archiving of previous run failed"
	761	printf "\n see directory \~/job_queue on remote machine"
	762	locat=archive; exit
	763	else
	764	printf "\n +++ warning: data archiving in a previous run failed"
	765	printf "\n MRUN continues, trying to get backup copy"
	766	fi
	767	fi
	768
	769
	770
	771	# LESEN UND INTERPRETIEREN DER KONFIGURATIONS-DATEI VOM SHELLSCRIPT AUS
	772	# VORUEBERGEHEND ZWINGEND AUF LINUX-RECHNERN
	773	if [[ "$read_from_config" = false ]]
	774	then
	775
	776	[[ $silent = false ]] && printf "\n Reading the configuration file... "
	777	while read zeile
	778	do
	779	[[ $silent = false ]] && printf "."
	780
	781
	782	# ZUERST EVENTUELL VORKOMMENDE ENVIRONMENT-VARIABLEN DURCH IHRE WERTE
	783	# ERSETZEN
	784	eval zeile=\"$zeile\"
	785
	786
	787	# INTERPRETATION DER ZEILE
[21]	788	if [[ "$(echo $zeile)" = "" ]]
[1]	789	then
[21]	790	# LEERZEILE, KEINE AKTION
	791	continue
[1]	792
[21]	793	elif [[ "$(echo $zeile \| cut -c1)" = "#" ]]
	794	then
	795
[1]	796	# ZEILE IST KOMMENTARZEILE
	797	true
	798
	799	elif [[ "$(echo $zeile \| cut -c1)" = "%" ]]
	800	then
	801
	802	# ZEILE DEFINIERT ENVIRONMENT-VARIABLE
	803	zeile=$(echo $zeile \| cut -c2-)
[69]	804	# echo $zeile \| cut -d" " -f1-5 \| read var value for_host for_cond1 for_cond2
	805	var=`echo $zeile \| cut -d" " -f1`
	806	value=`echo $zeile \| cut -d" " -s -f2`
	807	for_host=`echo $zeile \| cut -d" " -s -f3`
	808	for_cond1=`echo $zeile \| cut -d" " -s -f4`
	809	for_cond2=`echo $zeile \| cut -d" " -s -f5`
[1]	810
	811	if [[ "$for_host" = "" \|\| ( "$for_host" = $host && "$for_cond1" = "$cond1" && "$for_cond2" = "$cond2" ) \|\| $(echo "$input_list$output_list"\|grep -c "$for_host") != 0 ]]
	812	then
	813
	814	# BEI COMPILER- CPP- ODER LINKEROPTIONEN EVTL ":" DURCH " "
	815	# ERSETZEN. "::" WIRD DURCH ":" ERSETZT.
	816	value=`echo $value \| sed 's/::/%DUM%/g' \| sed 's/:/ /g' \| sed 's/%DUM%/:/g'`
	817
	818
	819	# ENVIRONMENT-VARIABLE WIRD WERT AUS KONFIGURATIONSDATEI
	820	# ZUGEWIESEN, WENN SIE SELBST NOCH KEINEN WERT UEBER DIE
	821	# ENTSPRECHENDE SCRIPT-OPTION ERHALTEN HAT. SOLCHE
	822	# VARIABLEN HAETTEN DANN DEN WERT "" ODER IM INTEGER-FALL DEN
	823	# WERT 0. ALLGEMEINE REGEL ALSO: SCRIPT-OPTION GEHT UEBER
	824	# KONFIGURATIONSDATEI
	825	if [[ "$(eval echo \$$var)" = "" \|\| "$(eval echo \$$var)" = "0" ]]
	826	then
	827	eval $var=\$value
	828
	829	# EVTL. BILDSCHIRMAUSGABEN DER ENVIRONMENT-VARIABLEN
	830	if [[ $do_trace = true ]]
	831	then
	832	printf "\n*** ENVIRONMENT-VARIABLE $var = $value"
	833	fi
	834	fi
	835
	836	# WENN ENVIRONMENT-VARIABLE HOST VEREINBART, DANN SOFORT AUSWERTEN
	837	# WERT VON do-remote WIRD BEI DATEIVERBINDUNGEN BENOETIGT
	838	# WENN AUF REMOTE-MASCHINE GERECHNET WIRD, IST GLEICHZEITIG KLAR,
	839	# DASS EIN BATCH-JOB GESTARTET WERDEN MUSS
	840	if [[ $var = host ]]
	841	then
	842	if [[ -n $host && "$host" != $localhost ]]
	843	then
	844	do_batch=true
	845	do_remote=true
	846	case $host in
[717]	847	(ibm\|ibmh\|ibmkisti\|ibmku\|ibms\|ibmy\|lckyoto\|lcsgib\|lcsgih\|nech\|necriam\|unics\|lcxt4\|lcxt5m\|lck\|lckordi) true;;
[83]	848	(*) printf "\n +++ sorry: execution of batch jobs on remote host \"$host\""
	849	printf "\n is not available"
[1]	850	locat=nqs; exit;;
	851	esac
	852	else
	853	host=$localhost
	854	fi
	855	fi
	856
	857	# VOM BENUTZER DEFINIERTE ENVIRONMENT VARIABLEN MUESSEN PRINZIPIELL
	858	# EXPORTIERT WERDEN, DA SIE VIELLEICHT IN WEITER UNTEN AUFZURUFEN-
	859	# DEN PROGRAMMEN BENOETIGT WERDEN
	860	export $var
	861	fi
	862
	863
	864
	865
	866	elif [[ "$(echo $zeile \| cut -c1-3)" = "EC:" ]]
	867	then
	868
	869	# ZEILE DEFINIERT ERROR-KOMMANDO
	870	(( iec = iec + 1 ))
	871	zeile=$(echo $zeile \| cut -c4-)
	872	err_command[$iec]="$zeile"
	873
	874	elif [[ "$(echo $zeile \| cut -c1-3)" = "IC:" ]]
	875	then
	876
	877	# ZEILE DEFINIERT INPUT-KOMMANDO
	878	(( iic = iic + 1 ))
	879	zeile=$(echo $zeile \| cut -c4-)
	880	in_command[$iic]="$zeile"
	881
	882	elif [[ "$(echo $zeile \| cut -c1-3)" = "OC:" ]]
	883	then
	884
	885	# ZEILE DEFINIERT OUTPUT-KOMMANDO
	886	(( ioc = ioc + 1 ))
	887	zeile=$(echo $zeile \| cut -c4-)
	888	out_command[$ioc]="$zeile"
	889
	890	else
	891
	892	# ZEILE DEFINIERT DATEIVERBINDUNG. EINLESEN DER DATEIEIGENSCHAFTEN
	893	# s2a: in/out - Feld
	894	# s2b: loc - Feld (optional)
	895	# s2c: tr/ar - Feld (optional)
[69]	896	# echo $zeile \| cut -d" " -f1-2 \| read s1 s2
	897	s1=`echo $zeile \| cut -d" " -f1`
	898	s2=`echo $zeile \| cut -d" " -s -f2`
[1]	899	s2a=$(echo $s2 \| cut -d":" -f1)
	900	if [[ $(echo $s2 \| grep -c ":") = 0 ]]
	901	then
	902	s2b=""
	903	s2c=""
	904	else
[69]	905	# echo $s2 \| cut -d":" -f2-3 \| sed 's/:/ /g' \| read s2b s2c
	906	s2b=`echo $s2 \| cut -d":" -f2 \| sed 's/:/ /g'`
	907	s2c=`echo $s2 \| cut -d":" -s -f3 \| sed 's/:/ /g'`
[1]	908	fi
[69]	909	# echo $zeile \| cut -d" " -f3-6 \| read s3 s4 s5 s6
	910	s3=`echo $zeile \| cut -d" " -f3`
	911	s4=`echo $zeile \| cut -d" " -s -f4`
	912	s5=`echo $zeile \| cut -d" " -s -f5`
	913	s6=`echo $zeile \| cut -d" " -s -f6`
[1]	914
	915
	916	# ABSPEICHERN DER DATEIVERBINDUNG, FALLS IN INPUT- ODER OUTPUT-LIST
	917	# VERMERKT. VARIABLE S3 KANN AUCH LISTE ENTHALTEN (FELDTRENNER ":")
	918	# DATEIVERBINDUNG WIRD DANN NICHT ABGESPEICHERT UND GEPRUEFT, WENN
	919	# PROGRAMMLAUF AUF REMOTE-MASCHINE ERFOLGT UND DATEI NUR LOKAL VOR-
	920	# HANDEN SEIN MUSS (D.H. s2b = loc)
	921	IFSALT="$IFS"; IFS="$IFS:"
	922	if [[ "$s2a" = in && ! ( $do_remote = true && ( "$s2b" = loc \|\| "$s2b" = locopt ) ) ]]
	923	then
	924	found=false
	925	for actual in $input_list
	926	do
	927	for formal in $s3
	928	do
	929	[[ $actual = $formal \|\| "$formal" = "-" ]] && found=true
	930	done
	931	done
	932	if [[ $found = true ]]
	933	then
	934	(( iin = iin + 1 ))
	935	localin[$iin]=$s1; transin[$iin]=$s2b; actionin[$iin]=$s2c;
	936	typein[$iin]=$s3; pathin[$iin]=$s4; endin[$iin]=$s5;
	937	extin[$iin]=$s6
	938	fi
	939	elif [[ "$s2a" = out && ! ( $do_remote = true && "$s2b" = loc ) ]]
	940	then
	941	found=false
	942	for actual in $output_list
	943	do
	944	for formal in $s3
	945	do
	946	[[ $actual = $formal \|\| "$formal" = "-" ]] && found=true
	947	done
	948	done
	949	if [[ $found = true ]]
	950	then
	951	(( iout = iout + 1 ))
	952	localout[$iout]=$s1; actionout[$iout]=$s2c; typeout[$iout]=$s3;
	953	pathout[$iout]=$s4; endout[$iout]=$s5; extout[$iout]=$s6
	954	fi
	955	elif [[ "$s2a" != in && "$s2a" != out ]]
	956	then
	957	printf "\n +++ I/O-attribute in configuration file $config_file has the invalid"
	958	printf "\n value \"$s2\". Only \"in\" and \"out\" are allowed!"
	959	locat=connect; exit
	960	fi
	961	IFS="$IFSALT"
	962	fi
	963	done < $config_file
	964
	965	else
	966
	967
	968	# INTERPRETATION DER KONFIGURATIONSDATEI MITTELS FORTRAN 90 - PROGRAMM
	969	[[ $silent = false ]] && printf "..."
	970	export cond1 cond2 config_file do_remote do_trace input_list localhost output_list
	971	export interpreted_config_file=.icf.$RANDOM
	972
	973
[69]	974
	975	# ENVIRONMENT-VARIABLEN FUER INTERPRET_CONFIG UEBER NAMELIST_DATEI ZUR
	976	# VERFUEGUNG STELLEN
	977	cat > .mrun_environment << %%END%%
	978	&mrun_environment cond1 = '$cond1', cond2 = '$cond2',
	979	config_file = '$config_file', do_remote = '$do_remote',
	980	do_trace = '$do_trace', host = '$host',
	981	input_list = '$input_list', icf = '$interpreted_config_file',
	982	localhost = '$localhost', output_list = '$output_list' /
	983
	984	%%END%%
	985
	986	# WERTE VON MRUN-OPTIONEN SICHERN UND DAMIT GEGEBENENFALLS SPAETER DIE
	987	# IN DER KONFIGURAIONSDATEI ANGEGEBENEN WERTE UEBERSTEUERN
	988	mrun_memory=$memory
[127]	989	mrun_group_number=$group_number
[69]	990	mrun_cpumax=$cpumax
	991	mrun_numprocs=$numprocs
	992
[251]	993	if [[ $localhost_realname = "sx-fep" ]]
[66]	994	then
[251]	995	/home/COAR/NC/raasch/pub/interpret_config_necriam.x
[66]	996	else
[503]	997
	998	if [[ "$host" != $localhost ]]
	999	then
	1000
	1001	# REMOTE JOB FROM LOCAL HOST: JUST TAKE THE FIRST EXECUTABLE FOUND
	1002	interpret_config_executable=`ls -1 ${PALM_BIN}/interpret_config*.x 2>/dev/null`
	1003	if [[ $? != 0 ]]
	1004	then
	1005	printf "\n\n +++ no interpret_config found"
	1006	printf "\n run \"mbuild -u -h ...\" to generate utilities for this host"
	1007	locat=interpret_config; exit
	1008	fi
	1009	interpret_config_executable=`echo $interpret_config_executable \| cut -d" " -f1`
	1010	$interpret_config_executable
	1011
	1012	else
	1013
	1014	# CHECK, IF THERE IS AN EXECUTABLE FOR THE BLOCK
	1015	if [[ ! -f ${PALM_BIN}/interpret_config${block}.x ]]
	1016	then
	1017	printf "\n\n +++ no interpret_config found for given block \"$cond1 $cond2\""
	1018	printf "\n run \"mbuild -u -h ...\" to generate utilities for this block"
	1019	locat=interpret_config; exit
	1020	else
	1021	interpret_config${block}.x
	1022	fi
	1023
	1024	fi
[66]	1025	fi
[71]	1026	rm .mrun_environment
[1]	1027
[71]	1028
[1]	1029	# AUSFUEHRUNG DER GENERIERTEN SHELL-KOMMANDOS IN DIESER SHELL
	1030	chmod u+x $interpreted_config_file
	1031	export PATH=$PATH:.
	1032	. $interpreted_config_file
[71]	1033	rm $interpreted_config_file
[1]	1034
[71]	1035
[69]	1036	# OPTIONSWERTE UEBERSTEUERN KONFIGURATIONSDATEI
[71]	1037	[[ $mrun_memory != 0 ]] && memory=$mrun_memory
[127]	1038	[[ "$mrun_group_number" != "none" ]] && group_number=$mrun_group_number
[71]	1039	[[ $mrun_cpumax != 0 ]] && cpumax=$mrun_cpumax
	1040	[[ "$mrun_numprocs" != "" ]] && numprocs=$mrun_numprocs
[69]	1041
[1]	1042	fi
	1043
	1044
	1045	# QUELLTEXTVERZEICHNIS AUF LOKALER MASCHINE AUS KONFIGURATIONSDATEI
	1046	# BESTIMMEN (WUERDE SONST EVTL. DAS VERZEICHNIS DES JEWEILS UNTER -h
	1047	# ANGEGEBENEN REMOTE-RECHNERS SEIN)
	1048	# BEI BATCH-JOBS SIND DIE ZU UEBERSETZENDEN PROGRAMMTEILE SCHON KOMPLETT
	1049	if [[ "$SOURCES_COMPLETE" = "" ]]
	1050	then
	1051
	1052	# ZUERST PRUEFEN, OB EIN GLOBALER QUELLTEXTPFAD FUER ALLE RECHNER
	1053	# VEREINBART WURDE
	1054	source_path=""
	1055	line=""
	1056	grep "%source_path" $config_file > tmp_mrun
	1057	while read line
	1058	do
	1059	if [[ "$line" != "" && $(echo $line \| cut -c1) != "#" ]]
	1060	then
	1061	if [[ "$(echo $line \| cut -d" " -f3)" = "" ]]
	1062	then
	1063	global_source_path=`echo $line \| cut -d" " -f2`
	1064	fi
	1065	fi
	1066	done < tmp_mrun
	1067
	1068	line=""
	1069	found=false
	1070	grep " $localhost" $config_file \| grep "%source_path" > tmp_mrun
	1071	while read line
	1072	do
	1073	if [[ "$line" != "" && $(echo $line \| cut -c1) != "#" ]]
	1074	then
	1075	if [[ $found = true ]]
	1076	then
	1077	printf "\n\n +++ more than one source path found in configuration file"
	1078	printf "\n for local host \"$localhost\" "
	1079	locat=source_path; exit
	1080	fi
	1081	source_path=`echo $line \| cut -d" " -f2`
	1082	found=true
	1083	fi
	1084	done < tmp_mrun
	1085	rm tmp_mrun
	1086
	1087	if [[ "$source_path" = "" ]]
	1088	then
	1089	if [[ "$global_source_path" != "" ]]
	1090	then
	1091	source_path=$global_source_path
	1092	else
	1093	printf "\n\n +++ no source path found in configuration file"
	1094	printf "\n for local host \"$localhost\" "
	1095	locat=source_path; exit
	1096	fi
	1097	fi
	1098	eval source_path=$source_path
	1099
	1100	if [[ ! -d $source_path ]]
	1101	then
	1102	printf "\n\n +++ source path \"$source_path\" on local host"
	1103	printf "\n \"$localhost\" does not exist"
	1104	locat=source_path; exit
	1105	fi
	1106
	1107	fi
	1108
	1109
[78]	1110	# GLOBALE REVISIONSNUMMER ERMITTELN (FORTSETZUNGSLAEUFEN WIRD DIESE
	1111	# DURCH OPTION -G MITGETEILT)
[693]	1112	if [[ "$global_revision" = "" && $host != "ibmkisti" ]]
[78]	1113	then
	1114	global_revision=`svnversion $source_path 2>/dev/null`
	1115	global_revision="Rev: $global_revision"
	1116	fi
[1]	1117
[78]	1118
[1]	1119	# NOCHMAL PRUEFEN, OB AUF REMOTE-MASCHINE GERECHNET WERDEN SOLL
	1120	# (HOST KANN IN KONFIGURATIONSDATEI ANDERS FESTGELEGT WORDEN SEIN)
	1121	# WERT VON do_remote WIRD FUER DATEIVERBINDUNGEN BENOETIGT.
	1122	# WENN AUF REMOTE-MASCHINE GERECHNET WIRD, IST GLEICHZEITIG KLAR,
	1123	# DASS EIN BATCH-JOB GESTARTET WERDEN MUSS
	1124	if [[ -n $host && "$host" != $localhost ]]
	1125	then
	1126	do_batch=true
	1127	do_remote=true
	1128	case $host in
[717]	1129	(ibm\|ibmh\|ibmkisti\|ibmku\|ibms\|ibmy\|lckyoto\|lcsgib\|lcsgih\|nech\|necriam\|unics\|lcxt4\|lcxt5m\|lck\|lckordi) true;;
[1]	1130	(*) printf "\n"
[83]	1131	printf "\n +++ sorry: execution of batch jobs on remote host \"$host\""
	1132	printf "\n is not available"
[1]	1133	locat=nqs; (( iec = 0 )); exit;;
	1134	esac
	1135	else
	1136	host=$localhost
	1137	fi
	1138
	1139
	1140	# PRUEFUNG EINIGER PROZESSORZAHLANGABEN BEI RECHNUNGEN AUF PARALLELRECHNERN
	1141	if [[ "$cond1" = parallel \|\| "$cond2" = parallel ]]
	1142	then
	1143
	1144	# PRUEFEN, OB DIE ANZAHL DER ZU VERWENDENDEN PES ANGEGEBEN WURDE
	1145	if [[ ! -n $numprocs ]]
	1146	then
	1147	printf "\n"
	1148	printf "\n +++ option \"-K parallel\" requires additional specification"
	1149	printf "\n of the number of processors to be used by"
	1150	printf "\n mrun-option \"-X\" or by environment-variable"
	1151	printf "\n \"numprocs\" in the configuration file"
	1152	locat=numprocs; (( iec = 0 )); exit
	1153	fi
	1154
	1155	# PRUEFEN, OB DIE PROZESSORANZAHL PRO KNOTEN ANGEGEBEN WURDE (GGF.
	1156	# DEFAULT-WERT SETZEN) UND OB SIE EIN GANZZAHLIGER TEILER DER
	1157	# GESAMTPROZESSORANZAHL IST
[693]	1158	if [[ $host = nech \|\| $host = necriam \|\| $host = ibmh \|\| $host = ibmkisti \|\| $host = ibms ]]
[1]	1159	then
	1160	[[ "$tasks_per_node" = "" ]] && tasks_per_node=6
	1161	(( ival = $tasks_per_node ))
	1162	(( pes = numprocs ))
	1163	# if [[ $(echo $package_list \| grep -c dvrp_graphics+1PE) = 1 ]]
	1164	# then
	1165	# (( pes = pes - 1 ))
	1166	# fi
	1167	(( ii = pes / ival ))
	1168	if (( pes - ii * ival > 0 ))
	1169	then
	1170	printf "\n"
	1171	printf "\n +++ tasks per node (option \"-T\") must be an integral"
	1172	printf "\n divisor of the total number of processors (option \"-X\")"
	1173	printf "\n values of this mrun-call: \"-T $tasks_per_node\" \"-X $numprocs\""
	1174	locat=tasks_per_node; (( iec = 0 )); exit
	1175	fi
	1176	fi
	1177
	1178	# IBMY HAT NUR EINEN KNOTEN
	1179	if [[ $host = ibmy ]]
	1180	then
	1181	if [[ "$tasks_per_node" != "" && "$tasks_per_node" != "$numprocs" ]]
	1182	then
	1183	printf "\n"
	1184	printf "\n +++ on ibmy, tasks per node (option \"-T\") must be equal to the"
	1185	printf "\n total number of processors (option \"-X\")"
	1186	printf "\n values of this mrun-call: \"-T $tasks_per_node\" \"-X $numprocs\""
	1187	locat=tasks_per_node; (( iec = 0 )); exit
	1188	fi
	1189	fi
	1190
	1191	# FALLS OPENMP PARALLELISIERUNG VERWENDET WERDEN SOLL, ANZAHL VON THREADS
	1192	# SETZEN UND ZAHL DER TASKS PRO KNOTEN AUF 1 SETZEN
[622]	1193	# if [[ $use_openmp = true ]]
	1194	# then
	1195	# threads_per_task=$tasks_per_node
	1196	# tasks_per_node=1
	1197	# fi
[377]	1198
	1199	# SETTINGS FOR SUBJOB-COMMAND
[622]	1200	TOPT="-T $tasks_per_node"
[1]	1201	OOPT="-O $threads_per_task"
	1202
	1203	# GESAMTZAHL DER KNOTEN BESTIMMEN
	1204	if [[ "$tasks_per_node" != "" ]]
	1205	then
	1206	(( nodes = numprocs / ( tasks_per_node * threads_per_task ) ))
	1207	fi
	1208
	1209	# PRUEFEN, OB NODE USAGE EINEN ERLAUBTEN WERT HAT BZW. DEN DEFAULT
	1210	# SETZEN
	1211	if [[ $node_usage = default ]]
	1212	then
	1213	if [[ $host = ibms ]]
	1214	then
	1215	node_usage=shared
[181]	1216	elif [[ $(echo $host \| cut -c1-5) = lcsgi ]]
	1217	then
	1218	node_usage=singlejob
[1]	1219	else
	1220	node_usage=not_shared
	1221	fi
	1222	fi
[181]	1223
[635]	1224	if [[ $node_usage != shared && $node_usage != not_shared && $node_usage != singlejob && "$(echo $node_usage \| cut -c1-3)" != "sla" ]]
[1]	1225	then
	1226	printf "\n"
	1227	printf "\n +++ node usage (option \"-n\") is only allowed to be set"
	1228	printf "\n \"shared\" or \"not_shared\""
	1229	locat=tasks_per_node; (( iec = 0 )); exit
	1230	fi
	1231
	1232	fi
	1233
	1234	# PRUEFEN, OB HOSTFILE EXISTIERT
	1235	if [[ -n $hostfile ]]
	1236	then
	1237	if [[ ! -f $hostfile ]]
	1238	then
	1239	printf "\n"
	1240	printf "\n +++ hostfile \"$hostfile\" does not exist"
	1241	locat=hostfile; exit
	1242	fi
	1243	fi
	1244
	1245	# PRUEFEN, OB RHOSTS DATEI EXISTIERT. GEGEBENENFALLS ANLEGEN BZW. ERWEITERN
	1246	if [[ $host = ibmy && $do_remote = false ]]
	1247	then
	1248	if [[ ! -f $HOME/.rhosts ]]
	1249	then
	1250	echo "gfdl5.yonsei.ac.kr" > $HOME/.rhosts
	1251	printf "\n\n *** file:"
	1252	printf "\n $HOME/.rhosts"
	1253	printf "\n was created\n"
	1254	fi
	1255	if [[ $(grep -c gfdl5.yonsei.ac.kr $HOME/.rhosts) = 0 ]]
	1256	then
	1257	echo "gfdl5.yonsei.ac.kr" >> $HOME/.rhosts
	1258	printf "\n\n *** file:"
	1259	printf "\n $HOME/.rhosts"
	1260	printf "\n was extended by the name of the current host\n"
	1261	fi
	1262	fi
	1263
	1264
	1265
	1266	# FALLS NICHT VORGEGEBEN, DEFAULT-QUEUE AUF DER ZIELMASCHINE FESTLEGEN
	1267	if [[ $queue = none ]]
	1268	then
	1269	case $host in
[312]	1270	(ibmh) queue=no_class;;
[693]	1271	(ibmkisti) queue=class.32plus;;
[1]	1272	(ibmy) queue=parallel;;
[440]	1273	(lckyoto) queue=ph;;
[191]	1274	(lcsgib) queue=bigq;;
	1275	(lcsgih) queue=bigq;;
[635]	1276	(lctit) queue=S;;
[1]	1277	(nech) queue=none;;
[253]	1278	(necriam) queue=SP;;
[1]	1279	(unics) queue=unics;;
	1280	esac
	1281	fi
	1282
	1283
	1284	# VOLLSTAENDIGE DATEINAMEN DER INPUT-FILES BILDEN,
	1285	# INPUT-DATEIEN AUF VORHANDENSEIN PRUEFEN UND EVTL. HOECHSTE ZYKLUSNUMMER
	1286	# ERMITTELN
	1287	(( i = 0 ))
	1288	while (( i < iin ))
	1289	do
	1290	(( i = i + 1 ))
	1291	(( maxcycle = 0 ))
	1292
	1293	# NAMENSBILDUNG (EVTL. IST FESTER DATEINAME VORGEGEBEN)
	1294	if [[ "${actionin[$i]}" = di ]]
	1295	then
	1296	remotepathin[$i]=${pathin[$i]}/${endin[$i]} # REMOTE-PFAD ERST AUF
	1297	# REM-MASCHINE AUSWERTEN
	1298	eval filename=${pathin[$i]}/${endin[$i]}
	1299	else
	1300	remotepathin[$i]=${pathin[$i]}/${afname}${endin[$i]} # REMOTE-PFAD ERST AUF
	1301	# REM-MASCHINE AUSWERTEN
	1302	eval filename=${pathin[$i]}/${afname}${endin[$i]}
	1303	fi
	1304
	1305	# PRUEFEN AUF VORHANDENSEIN
	1306	if [[ $(ls $filename* 2>&1 \| grep -c "not found") = 1 \|\| \
[746]	1307	$(ls $filename* 2>&1 \| grep -c "nicht gefunden") = 1 \|\| \
[1]	1308	$(ls $filename* 2>&1 \| grep -c "No such file") = 1 \|\| \
	1309	$(ls $filename* 2>&1 \| grep -c "does not exist") = 1 ]]
	1310	then
	1311	# DATEIEN MIT EXTENSION (Z.B. NC) MUESSEN NICHT ZWINGEND VORHANDEN
	1312	# SEIN, DESHALB IN SOLCHEN FAELLEN KEIN ABBRUCH. DIES IST NUR EINE
	1313	# VORUEBERGEHENDE LOESUNG (OKT 05). NICHT ZWINGEND BENOETIGTE
	1314	# EINGABEDATEIEN SOLLTEN EINE SPEZIELLE OPTION IN DER DATEI-
	1315	# VERBINDUNGSANWEISUNG BEKOMMEN (Z.B. inopt?)
	1316	if [[ "${transin[$i]}" != "locopt" ]]
	1317	then
	1318	printf "\n\n +++ INPUT-file: "
	1319	if [[ "${extin[$i]}" = "" ]]
	1320	then
	1321	printf "\n $filename"
	1322	else
	1323	printf "\n $filename.${extin[$i]}"
	1324	fi
	1325	printf "\n does not exist\n"
	1326	locat=input; exit
	1327	else
	1328	transin[$i]="unavailable"
	1329	fi
	1330	else
	1331
	1332	# ZYKLUSNUMMER FESTSTELLEN
	1333	ls -1 -d $filename > filelist 2>/dev/null
	1334	ls -1 -d $filename.* >> filelist 2>/dev/null
	1335	while read zeile
	1336	do
	1337	cycle=$(print $zeile \| cut -f2 -d".")
	1338	if [[ "$cycle" = "$zeile" ]]
	1339	then
	1340	(( icycle = 0 ))
	1341	elif [[ "$cycle" = "${extin[$i]}" ]]
	1342	then
	1343	(( icycle = 0 ))
	1344	else
	1345	(( icycle = $cycle ))
	1346	fi
	1347	if (( icycle > maxcycle ))
	1348	then
	1349	(( maxcycle = icycle ))
	1350	file_to_be_used=$zeile
	1351	fi
	1352	done <filelist
	1353	rm filelist
	1354
	1355	# EVTL. ZYKLUSNUMMER AN DATEINAMEN ANFUEGEN
	1356	if (( maxcycle > 0 ))
	1357	then
	1358	if [[ "${extin[$i]}" != " " && "${extin[$i]}" != "" ]]
	1359	then
	1360	filename=${filename}.$maxcycle.${extin[$i]}
	1361	else
	1362	filename=${filename}.$maxcycle
	1363	fi
	1364	else
	1365	if [[ "${extin[$i]}" != " " && "${extin[$i]}" != "" ]]
	1366	then
	1367	filename=${filename}.${extin[$i]}
	1368	fi
	1369	fi
	1370
	1371	# DATEINAMEN OHNE PFAD ABER MIT ZYKLUSNUMMER ABSPEICHERN,
	1372	# DA SPAETER BEI RUECKSPEICHERN VOM ARCHIVSYSTEM EVTL. BENOETIGT
	1373	absnamein[$i]=$filename
	1374	if (( maxcycle > 0 ))
	1375	then
	1376	if [[ "${actionin[$i]}" = di ]]
	1377	then
	1378	frelin[$i]=${endin[$i]}.$maxcycle
	1379	else
	1380	frelin[$i]=${afname}${endin[$i]}.$maxcycle
	1381	fi
	1382	else
	1383	if [[ "${actionin[$i]}" = di ]]
	1384	then
	1385	frelin[$i]=${endin[$i]}
	1386	else
	1387	frelin[$i]=${afname}${endin[$i]}
	1388	fi
	1389	fi
	1390
	1391	fi
	1392	done
	1393
	1394
	1395	# VOLLSTAENDIGE DATEINAMEN (OHNE $ ODER ~) DER OUTPUT-FILES BILDEN,
	1396	# OUTPUT-DATEIEN AUF VORHANDENSEIN PRUEFEN UND EVTL. HOECHSTE ZYKLUSNUMMER
	1397	# ERMITTELN ODER, FALLS NICHT VORHANDEN, PRUEFEN, OB SIE SICH ANLEGEN LASSEN
	1398	# DIESE AKTIONEN FINDEN NICHT STATT, WENN PROGRAMM AUF REMOTE-MASCHINE
	1399	# RECHNET UND DATEI ANSCHLIESSEND TRANSFERIERT WERDEN SOLL!
	1400	(( i = 0 ))
	1401	while (( i < iout ))
	1402	do
	1403	(( i = i + 1 ))
	1404	if [[ ! ( $fromhost != $localhost && ( "${actionout[$i]}" = tr \|\| "${actionout[$i]}" = tra \|\| "${actionout[$i]}" = trpe ) ) ]]
	1405	then
[260]	1406	if [[ "${actionout[$i]}" = tr ]]
[1]	1407	then
	1408	actionout[$i]=""
[260]	1409	elif [[ "${actionout[$i]}" = trpe ]]
	1410	then
	1411	actionout[$i]=pe
[1]	1412	elif [[ "${actionout[$i]}" = tra ]]
	1413	then
	1414	actionout[$i]=a
	1415	fi
	1416	(( maxcycle = 0 ))
	1417	eval filename=${pathout[$i]}/${fname}${endout[$i]}
	1418	eval catalogname=${pathout[$i]}
	1419	if [[ $(ls $filename* 2>&1 \| grep -c "not found") = 1 \|\| \
[746]	1420	$(ls $filename* 2>&1 \| grep -c "nicht gefunden") = 1 \|\| \
[1]	1421	$(ls $filename* 2>&1 \| grep -c "No such file") = 1 \|\| \
	1422	$(ls $filename* 2>&1 \| grep -c "does not exist") = 1 ]]
	1423	then
	1424
	1425	# OUTPUT-DATEI NICHT VORHANDEN. PRUEFEN, OB ANLEGEN MOEGLICH.
	1426	if cat /dev/null > $filename
	1427	then
	1428	rm $filename
	1429	else
	1430
	1431	# PRUEFEN, OB KATALOG VORHANDEN UND EVTL. DIESEN ANLEGEN
	1432	if [[ ! -d $catalogname ]]
	1433	then
	1434	if mkdir -p $catalogname
	1435	then
	1436	printf "\n\n *** directory:"
	1437	printf "\n $catalogname"
	1438	printf "\n was created\n"
	1439	else
	1440	printf "\n\n +++ OUTPUT-file:"
	1441	printf "\n $filename"
	1442	printf "\n cannot be created, because directory does not exist"
	1443	printf "\n and cannot be created either"
	1444	printf "\n"
	1445	locat=output ; exit
	1446	fi 2>/dev/null
	1447	else
	1448	printf "\n\n +++ OUTPUT-file:"
	1449	printf "\n $filename"
	1450	printf "\n cannot be created, although directory exists"
	1451	printf "\n"
	1452	locat=output ; exit
	1453	fi
	1454	fi 2>/dev/null
	1455	else
	1456
	1457	# ZYKLUSNUMMER FESTSTELLEN
	1458	ls -1 -d $filename > filelist 2>/dev/null
	1459	ls -1 -d $filename.* >> filelist 2>/dev/null
	1460	while read zeile
	1461	do
	1462	cycle=$(print $zeile \| cut -f2 -d".")
	1463	if [[ "$cycle" = "$zeile" \|\| "$cycle" = ${extout[$i]} ]]
	1464	then
	1465	(( icycle = 1 ))
	1466	else
	1467	(( icycle = $cycle + 1 ))
	1468	fi
	1469	if (( icycle > maxcycle ))
	1470	then
	1471	(( maxcycle = icycle ))
	1472	fi
	1473	done <filelist
	1474	rm filelist
	1475	fi
	1476
	1477	# EVTL. ZYKLUSNUMMER AN DATEINAMEN ANFUEGEN UND PRUEFEN, OB SICH
	1478	# DATEI AUCH WIRKLICH ANLEGEN LAESST. IM APPEND-FALL MUSS AN DEN
	1479	# HOECHSTEN VORHANDENEN ZYKLUS-ANGEHAENGT WERDEN. ANLAGEPRUEFUNG
	1480	# IST DANN NATUERLICH NICHT ZULAESSIG BZW. MOEGLICH
	1481	if [[ "${actionout[$i]}" != a ]]
	1482	then
	1483	if (( maxcycle > 0 ))
	1484	then
	1485	filename=${filename}.$maxcycle
	1486	if cat /dev/null > $filename
	1487	then
	1488	rm $filename
	1489	else
	1490	printf "\n +++ OUTPUT-file:"
	1491	printf "\n $filename"
	1492	printf "\n cannot be created"
	1493	locat=output ; exit
	1494	fi
	1495	fi
	1496	else
	1497	(( maxcycle = maxcycle - 1 ))
	1498	if (( maxcycle > 0 ))
	1499	then
	1500	filename=${filename}.$maxcycle
	1501	fi
	1502	fi
	1503
	1504	# DATEINAMEN OHNE PFAD ABER MIT ZYKLUSNUMMER ABSPEICHERN,
	1505	# DA SPAETER BEI ABLAGE AUF ARCHIVSYSTEM BZW. FUER
	1506	# DATEI OUTPUT_FILE_CONNECTIONS EVTL. BENOETIGT
	1507	pathout[$i]=$filename
	1508	if (( maxcycle > 0 ))
	1509	then
	1510	frelout[$i]=${fname}${endout[$i]}.$maxcycle
	1511	else
	1512	frelout[$i]=${fname}${endout[$i]}
	1513	fi
	1514
	1515	fi
	1516	done
	1517
	1518
[260]	1519	# DAS DVR-PAKET ERFORDERT EINE ENTSPRECHENDE BIBLIOTHEK
[83]	1520	if [[ $(echo $package_list \| grep -c dvrp_graphics) != 0 ]]
	1521	then
[260]	1522	if [[ "$dvr_inc" = "" ]]
[83]	1523	then
[260]	1524	printf "\n\n +++ no value for \"dvr_inc\" given in configuration file"
[83]	1525	printf "\n This is required for the dvrp_graphics package.\n"
[260]	1526	locat=dvr; exit
[83]	1527	fi
[260]	1528	if [[ "$dvr_lib" = "" ]]
[83]	1529	then
[260]	1530	printf "\n\n +++ no value for \"dvr_lib\" given in configuration file"
[83]	1531	printf "\n This is required for the dvrp_graphics package.\n"
[260]	1532	locat=dvr; exit
[83]	1533	fi
	1534	fi
	1535
	1536
[1]	1537	# PRUEFEN, OB ENTWEDER HAUPTPROGRAMM ODER NUR EIN AUSFUEHRBARES
	1538	# PROGRAMM VEREINBART WURDE (IN DIESEM FALL BRAUCHT IM WEITEREN NICHT
	1539	# UEBERSETZT ZU WERDEN)
	1540	if [[ "$mainprog" = "" && "$executable" = "" ]]
	1541	then
	1542	printf "\n +++ neither main program nor executable defined"
	1543	locat=source; exit
	1544	elif [[ "$mainprog" != "" && "$executable" != "" ]]
	1545	then
	1546	printf "\n +++ main program as well as executable defined"
	1547	locat=source; exit
	1548	elif [[ "$mainprog" = "" && "$executable" != "" ]]
	1549	then
	1550	do_compile=false
	1551	fi
	1552
	1553
[191]	1554	# SOURCE-VERZEICHNIS ZUM AUFSAMMELN DER ZU UEBERSETZENDEN PROGRAMMTEILE
	1555	# ERZEUGEN. HIERHIN WERDEN SPAETER IM FALL VON BATCH-JOBS AUCH DAS MRUN-
	1556	# SCRIPTS SOWIE DIE KONFIGURATIONSDATEI KOPIERT
[193]	1557	if [[ $restart_run != true && "$SOURCES_COMPLETE" = "" ]]
[191]	1558	then
	1559	rm -rf SOURCES_FOR_RUN_$fname
	1560	mkdir SOURCES_FOR_RUN_$fname
	1561	fi
	1562
	1563
[1]	1564	# ALLE ZU UEBERSETZENDEN PROGRAMMTEILE ZUSAMMENSAMMELN
	1565	# BEI BATCH-JOBS IST DIES NICHT NOETIG, WEIL DIE PROGRAMMTEILE BEREITS DURCH
	1566	# DEN MRUN-AUFRUF ERMITTELT SIND, DER DEN BATCH-JOB GENERIERT HAT, UND
	1567	# IM VERZEICHNIS SOURCES_FOR_RUN_... ABGELEGT SIND
	1568	if [[ $do_compile = true && "$SOURCES_COMPLETE" = "" ]]
	1569	then
	1570
[5]	1571	[[ "$source_list" = LM ]] && source_list=LOCALLY_MODIFIED
	1572	[[ "$source_list" = WP ]] && source_list=WRITE_PERMIT
[1]	1573
	1574
	1575	# EVTL. ZU UEBERSETZENDE PROGRAMMTEILE DADURCH BESTIMMEN, DASS GEPRUEFT
	1576	# WIRD, WELCHE DATEIEN WRITE-PERMIT BESITZEN (DATEIEN SOLLTEN DESHALB
	1577	# UNTER RCS-KONTROLLE STEHEN). MIT DER VARIABLEN EXCLUDE KOENNEN
	1578	# BESTIMMTE DATEIEN GRUNDSAETZLICH VON DER UEBERSETZUNG AUSGESCHLOSSEN
	1579	# WERDEN. DIES IST NUR BEI MANUELLEM START VON MRUN DURCH DEN BENUTZER
	1580	# MOEGLICH, DA BEI AUTOMATISCHEN RESTARTS TEST DURCH DIE TATSAECHLICH
	1581	# ZU UBERSETZENDEN DATEIEN ERSETZT IST.
	1582	# ALLE ERMITTELTEN DATEIEN WERDEN IM VERZEICHNIS SOURCES_FOR_RUN_...
	1583	# GESAMMELT
[5]	1584	if [[ "$source_list" = WRITE_PERMIT ]]
[1]	1585	then
	1586
	1587	source_list=""
	1588	cd $source_path
	1589
	1590
	1591	# ALLE MOEGLICHEN QUELLCODEDATEIEN AUFLISTEN
	1592	Names=$(ls -1 *.f90 2>&1)
	1593	[[ $(echo $Names \| grep -c '*.f90') = 0 ]] && Filenames="$Names"
	1594	Names=$(ls -1 *.F90 2>&1)
	1595	[[ $(echo $Names \| grep -c '*.F90') = 0 ]] && Filenames="$Filenames $Names"
	1596	Names=$(ls -1 *.F 2>&1)
	1597	[[ $(echo $Names \| grep -c '*.F') = 0 ]] && Filenames="$Filenames $Names"
	1598	Names=$(ls -1 *.f 2>&1)
	1599	[[ $(echo $Names \| grep -c '*.f') = 0 ]] && Filenames="$Filenames $Names"
	1600	Names=$(ls -1 *.c 2>&1)
	1601	[[ $(echo $Names \| grep -c '*.c') = 0 ]] && Filenames="$Filenames $Names"
	1602
	1603
	1604	# DATEIEN MIT WRITE-PERMIT NACH SOURCES_FOR_RUN_... KOPIEREN
	1605	for dateiname in $Filenames
	1606	do
	1607	if [[ -w $dateiname ]]
	1608	then
	1609	if [[ "$exclude" = "" \|\| $(echo $exclude \| grep -c $dateiname) = 0 ]]
	1610	then
	1611	cp $dateiname $working_directory/SOURCES_FOR_RUN_$fname
	1612	source_list=$source_list"$dateiname "
	1613	fi
	1614	fi
	1615	done
	1616
	1617	cd - > /dev/null
	1618
	1619
[5]	1620	elif [[ "$source_list" = LOCALLY_MODIFIED ]]
	1621	then
	1622
	1623	# MODIFIZIERTE DATEIEN DER SVN-ARBEITSKOPIE BESTIMMEN
	1624	source_list=""
	1625	cd $source_path
	1626
	1627
	1628	# PRUEFEN, OB VERZEICHNIS UEBERHAUPT UNTER SVN-KONTROLLE STEHT
	1629	if [[ ! -d .svn ]]
	1630	then
	1631	printf "\n\n +++ source directory"
	1632	printf "\n \"$source_path\" "
	1633	printf "\n is not under control of \"subversion\"."
[306]	1634	printf "\n Please do not use mrun-option \"-s LOCALLY_MODIFIED\"\n"
[5]	1635	fi
	1636
	1637
	1638	# ALLE MODIFIZIERTEN QUELLCODEDATEIEN AUFLISTEN
	1639	Filenames=""
[69]	1640	svn status > tmp_mrun
	1641	while read line
[5]	1642	do
	1643	firstc=`echo $line \| cut -c1`
	1644	if [[ $firstc = M \|\| $firstc = "?" ]]
	1645	then
	1646	Name=`echo "$line" \| cut -c8-`
	1647	extension=`echo $Name \| cut -d. -f2`
	1648	if [[ "$extension" = f90 \|\| "$extension" = F90 \|\| "$extension" = f \|\| "$extension" = F \|\| "$extension" = c ]]
	1649	then
	1650	Filenames="$Filenames "$Name
	1651	fi
	1652	fi
[69]	1653	done < tmp_mrun
[5]	1654
	1655
	1656	# DATEIEN NACH SOURCES_FOR_RUN_... KOPIEREN
	1657	for dateiname in $Filenames
	1658	do
	1659	cp $dateiname $working_directory/SOURCES_FOR_RUN_$fname
	1660	source_list=$source_list"$dateiname "
	1661	done
	1662
	1663	cd - > /dev/null
	1664
	1665
[1]	1666	# MITTELS OPTION -s ANGEGEBENE DATEIEN NACH SOURCES_FOR_RUN_... KOPIEREN
	1667	# BEI AUTOMATISCHEN FORTSETZUNGSLAEUFEN SIND DORT SCHON ALLE DATEIEN
	1668	# VORHANDEN
[5]	1669	elif [[ "$source_list" != "" && $restart_run != true ]]
[1]	1670	then
	1671
	1672	cd $source_path
	1673
	1674	for filename in $source_list
	1675	do
	1676
	1677	# QUELLTEXT-DATEI DARF KEINE PFADE BEINHALTEN
	1678	if [[ $(print $filename \| grep -c "/") != 0 ]]
	1679	then
	1680	printf "\n +++ source code file: $filename"
	1681	printf "\n must not contain (\"/\") "
	1682	locat=source; exit
	1683	fi
	1684
	1685	if [[ ! -f $filename ]]
	1686	then
	1687	printf "\n +++ source code file: $filename"
	1688	printf "\n does not exist"
	1689	locat=source; exit
	1690	else
	1691	cp $filename $working_directory/SOURCES_FOR_RUN_$fname
	1692	fi
	1693
	1694	done
	1695
	1696	cd - > /dev/null
	1697
	1698	fi
	1699
	1700
	1701	# PRUEFEN, OB ENTWEDER HAUPTPROGRAMM VORHANDEN UND ES EVTL. IN DER
	1702	# LISTE DER ZU UEBERSETZENDEN PROGRAMMTEILE MIT ENTHALTEN IST (WENN
	1703	# NICHT, WIRD ES DIESER LISTE HINZUGEFUEGT)
[5]	1704	if [[ $restart_run != true ]]
[1]	1705	then
	1706
	1707	if [[ ! -f "$source_path/$mainprog" ]]
	1708	then
	1709	printf "\n\n +++ main program: $mainprog"
	1710	printf "\n does not exist in source directory"
	1711	printf "\n \"$source_path\"\n"
	1712	locat=source; exit
	1713	else
	1714	if [[ $(echo $source_list \| grep -c $mainprog) = 0 ]]
	1715	then
	1716	cp $source_path/$mainprog SOURCES_FOR_RUN_$fname
	1717	source_list=${mainprog}" $source_list"
	1718	fi
	1719	fi
	1720	fi
	1721
	1722
[211]	1723	# MAKEFILE AUF VORHANDENSEIN PRUEFEN UND KOPIEREN
	1724	# BEI RESTART-LAEUFEN LIEGT ES SCHON IM VERZEICHNIS SOURCES_FOR_RUN...
	1725	if [[ "$restart_run" != true ]]
	1726	then
	1727	[[ "$makefile" = "" ]] && makefile=$source_path/Makefile
	1728	if [[ ! -f $makefile ]]
	1729	then
	1730	printf "\n +++ file \"$makefile\" does not exist"
	1731	locat=make; exit
	1732	else
	1733	cp $makefile SOURCES_FOR_RUN_$fname/Makefile
	1734	fi
	1735	fi
	1736
	1737
[1]	1738	# DATEIEN AUS ZUSAETZLICHEM QUELLVERZEICHNIS HINZUFUEGEN
[5]	1739	if [[ $restart_run != true && "$add_source_path" != "" ]]
[1]	1740	then
	1741
	1742	# GIBT ES DAS VERZEICHNIS UEBERHAUPT?
	1743	if [[ ! -d $add_source_path ]]
	1744	then
	1745	printf "\n\n +++ WARNING: additional source code directory"
	1746	printf "\n \"$add_source_path\" "
	1747	printf "\n does not exist or is not a directory."
	1748	printf "\n No source code will be used from this directory!\n"
	1749	add_source_path=""
	1750	sleep 3
	1751	else
	1752
	1753	cd $add_source_path
[211]	1754	found=false
[1]	1755
	1756	Names=$(ls -1 *.f90 2>&1)
	1757	[[ $(echo $Names \| grep -c '*.f90') = 0 ]] && AddFilenames="$Names"
	1758	Names=$(ls -1 *.F90 2>&1)
	1759	[[ $(echo $Names \| grep -c '*.F90') = 0 ]] && AddFilenames="$AddFilenames $Names"
	1760	Names=$(ls -1 *.F 2>&1)
	1761	[[ $(echo $Names \| grep -c '*.F') = 0 ]] && AddFilenames="$AddFilenames $Names"
	1762	Names=$(ls -1 *.f 2>&1)
	1763	[[ $(echo $Names \| grep -c '*.f') = 0 ]] && AddFilenames="$AddFilenames $Names"
	1764	Names=$(ls -1 *.c 2>&1)
	1765	[[ $(echo $Names \| grep -c '*.c') = 0 ]] && AddFilenames="$AddFilenames $Names"
	1766
	1767	cd - > /dev/null
	1768	cd SOURCES_FOR_RUN_$fname
	1769
[211]	1770	# COPY MAKEFILE IF EXISTING
	1771	if [[ -f $add_source_path/Makefile ]]
	1772	then
	1773	printf "\n\n *** user Makefile from directory"
	1774	printf "\n \"$add_source_path\" is used \n"
	1775	sleep 1
	1776	cp $add_source_path/Makefile .
	1777	fi
	1778
[1]	1779	for dateiname in $AddFilenames
	1780	do
	1781	if [[ -f $dateiname ]]
	1782	then
	1783	printf "\n +++ source code file \"$dateiname\" found in additional"
	1784	printf "\n source code directory \"$add_source_path\" "
[211]	1785	printf "\n but was also given with option \"-s\" which means that it should be taken"
	1786	printf "\n from directory \"$source_path\"."
[1]	1787	locat=source; exit
	1788	fi
	1789
	1790	cp $add_source_path/$dateiname .
	1791	source_list="$source_list $dateiname"
	1792
[211]	1793	# CHECK IF FILE IS CONTAINED IN MAKEFILE
	1794	if [[ $(grep -c $dateiname Makefile) = 0 ]]
	1795	then
	1796	printf "\n\n +++ user file \"$dateiname\" "
	1797	printf "\n is not listed in Makefile \n"
	1798	locat=source; exit
	1799	else
	1800
[1]	1801	# Default User-Interface von der Liste entfernen, falls Datei
	1802	# ein User-Interface enthaelt
[211]	1803	# if [[ $( cat $dateiname \| grep -c "END SUBROUTINE user_parin" ) != 0 ]]
	1804	# then
	1805	# if [[ $dateiname != user_interface.f90 && -f user_interface.f90 ]]
	1806	# then
	1807	# rm -rf user_interface.f90
	1808	# source_list=`echo $source_list \| sed -e 's/user_interface.f90//'`
	1809	# printf "\n\n *** default \"user_interface.f90\" removed from the files to be translated"
	1810	# printf "\n since a user-interface is found in file"
	1811	# printf "\n \"$add_source_path/$dateiname\" \n"
	1812	# sleep 3
	1813	# else
	1814	if [[ $found = false ]]
[1]	1815	then
[211]	1816	found=true
	1817	printf "\n\n *** following user file(s) added to the"
	1818	printf " files to be translated:\n "
[1]	1819	fi
[211]	1820	printf "$dateiname "
	1821	sleep 0.5
[1]	1822	fi
	1823	done
[211]	1824	[[ $found = true ]] && printf "\n"
[1]	1825	cd - > /dev/null
	1826	fi
	1827	fi
	1828
	1829
	1830	# ALLE UNTERPROGRAMME, DIE ZU VEREINBARTEN SOFTWAREPAKETEN GEHOEREN,
	1831	# DER LISTE DER ZU UEBERSETZENDEN DATEIEN HINZUFUEGEN
[5]	1832	if [[ $restart_run != true && -n $package_list ]]
[1]	1833	then
	1834
	1835	cd $source_path
	1836
	1837	for package in $package_list
	1838	do
	1839
	1840	[[ $package = "dvrp_graphics+1PE" ]] && package=dvrp_graphics
	1841
	1842	# ERMITTELE ALLE DATEIEN, DIE ZUM PAKET GEHOEREN
	1843	# FEHLERMELDUNGEN WERDEN ABGEFANGEN, DA * AUCH VERZEICHNISSNAMEN
	1844	# LIEFERT
	1845	package_source_list=`grep "defined( __$package " * 2>/dev/null \| cut -f1 -d:`
	1846
	1847
	1848	# FUEGE DIESE DATEIEN DER LISTE DER ZU UEBERSETZENDEN DATEIEN
	1849	# HINZU, FALLS SIE NOCH NICHT DAZUGEHOEREN
	1850	for source_list_name in $package_source_list
	1851	do
	1852	if [[ $(echo $source_list \| grep -c $source_list_name) = 0 ]]
	1853	then
	1854
	1855	# NUR DATEIEN MIT GUELTIGEN ENDUNGEN VERWENDEN
	1856	ending=`echo $source_list_name \| cut -f2 -d.`
	1857	if [[ "$ending" = f90 \|\| "$ending" = F90 \|\| "$ending" = f \|\| "$ending" = F \|\| "$ending" = c ]]
	1858	then
	1859	cp $source_list_name $working_directory/SOURCES_FOR_RUN_$fname
	1860	source_list="$source_list $source_list_name"
	1861	fi
	1862	fi
	1863	done
	1864	done
	1865
	1866	cd - > /dev/null
	1867	fi
	1868
	1869
	1870	# MAKEFILE AUF VORHANDENSEIN PRUEFEN UND KOPIEREN
	1871	# BEI RESTART-LAEUFEN LIEGT ES SCHON IM VERZEICHNIS SOURCES_FOR_RUN...
[211]	1872	# if [[ "$restart_run" != true ]]
	1873	# then
	1874	# [[ "$makefile" = "" ]] && makefile=$source_path/Makefile
	1875	# if [[ ! -f $makefile ]]
	1876	# then
	1877	# printf "\n +++ file \"$makefile\" does not exist"
	1878	# locat=make; exit
	1879	# else
	1880	# cp $makefile SOURCES_FOR_RUN_$fname/Makefile
	1881	# fi
	1882	# fi
[1]	1883
	1884	fi # do_compile=true
	1885
	1886
	1887	# FALLS PROGRAMMTEILE UEBERSETZT WERDEN SOLLEN, FOLGEN JETZT EINIGE
	1888	# UEBERPRUEFUNGEN UND DAS SETZEN DER PRAEPROZESSOR-DIREKTIVEN
	1889	if [[ $do_compile = true ]]
	1890	then
	1891
	1892	# PRAEPROZESSOR-DIREKTIVEN ZUM SELEKTIVEN AUSWAEHLEN VON CODETEILEN
	1893	# ZUSAMMENSETZEN
[83]	1894	# DIREKTIVEN ZUM AKTIVIEREN VON RECHNERSPEZIFISCHEM CODE
[1]	1895	if [[ $(echo $localhost \| cut -c1-3) = ibm ]]
	1896	then
[82]	1897	cpp_options="${cpp_options},-D__ibm=__ibm"
[1]	1898	elif [[ $(echo $localhost \| cut -c1-3) = nec ]]
	1899	then
[82]	1900	cpp_options="$cpp_options -D__nec"
	1901	elif [[ $(echo $localhost \| cut -c1-2) = lc ]]
	1902	then
	1903	cpp_options="$cpp_options -D__lc"
[1]	1904	else
[82]	1905	cpp_options="$cpp_options -D__$localhost"
[1]	1906	fi
[83]	1907
	1908	# DIREKTIVEN DIE DURCH OPTION -K BESTIMMT WERDEN (Z.B. PARALLEL)
[1]	1909	if [[ $(echo $localhost \| cut -c1-3) = ibm ]]
	1910	then
[82]	1911	[[ -n $cond1 ]] && cpp_options="${cpp_options},-D__$cond1=__$cond1"
	1912	[[ -n $cond2 ]] && cpp_options="${cpp_options},-D__$cond2=__$cond2"
[1]	1913	else
[82]	1914	[[ -n $cond1 ]] && cpp_options="$cpp_options -D__$cond1"
	1915	[[ -n $cond2 ]] && cpp_options="$cpp_options -D__$cond2"
[1]	1916	fi
[83]	1917
	1918	# DIREKTIVEN DIE SOFTWAREPAKETE AKTIVIEREN (OPTION -p)
[1]	1919	if [[ -n $package_list ]]
	1920	then
	1921	for package in $package_list
	1922	do
	1923	if [[ $(echo $localhost \| cut -c1-3) = ibm ]]
	1924	then
	1925	if [[ $package != "dvrp_graphics+1PE" ]]
	1926	then
[82]	1927	cpp_options="${cpp_options},-D__$package=__$package"
[1]	1928	else
[82]	1929	cpp_options="${cpp_options},-D__dvrp_graphics=__dvrp_graphics"
[1]	1930	export use_seperate_pe_for_dvrp_output=true
	1931	fi
	1932	else
	1933	if [[ $package != "dvrp_graphics+1PE" ]]
	1934	then
[82]	1935	cpp_options="$cpp_options -D__$package"
[1]	1936	else
[82]	1937	cpp_options="$cpp_options -D__dvrp_graphics"
[1]	1938	export use_seperate_pe_for_dvrp_output=true
	1939	fi
	1940	fi
	1941	done
	1942	fi
[83]	1943
	1944	# DIREKTIVEN DIE DURCH OPTION -D FESTGELEGT SIND
[1]	1945	if [[ -n $cpp_opts ]]
	1946	then
	1947	for popts in $cpp_opts
	1948	do
[82]	1949	if [[ $(echo $localhost \| cut -c1-3) = ibm ]]
	1950	then
	1951	cpp_options="${cpp_options},-D__$popts=__$popts"
	1952	else
	1953	cpp_options="$cpp_options -D__$popts"
	1954	fi
[1]	1955	done
	1956	fi
	1957
	1958	else
	1959
	1960
	1961	# BEI LOKALEN RECHNUNGEN PRUEFEN, OB EXECUTABLE VORHANDEN
	1962	if [[ $do_remote = false ]]
	1963	then
	1964	if [[ ! -f $executable ]]
	1965	then
	1966	printf "\n +++ executable file: $executable"
	1967	printf "\n does not exist"
	1968	locat=executable; exit
	1969	fi
	1970	fi
	1971	fi
	1972
	1973
	1974	# JOBMODUS FESTSTELLEN
[635]	1975	if [[ "$ENVIRONMENT" = BATCH ]]
[1]	1976	then
	1977	jobmo=BATCH
	1978	else
	1979	jobmo=INTERACTIVE
	1980	fi
	1981
	1982
[635]	1983	# no interactive runs on lctit
	1984	if [[ $host = lctit && $jobmo = INTERACTIVE && $do_batch = false ]]
	1985	then
	1986	printf "\n +++ no interactive runs allowed on host \"$host\" "
	1987	printf "\n please submit batch job using mrun option \"-b\" \n"
	1988	locat=normal; exit
	1989	fi
	1990
	1991
[1]	1992	# HOSTSPEZIFISCHE DEFAULT-COMPILER SETZEN, FALLS NICHT BEREITS
	1993	# DURCH BENUTZER ANDERWEITIG VEREINBART
	1994	if [[ "$compiler_name" = "" ]]
	1995	then
[82]	1996
[220]	1997	printf "\n +++ no compiler specified for \"$host $cond1 $cond2\""
[82]	1998	locat=compiler_name; exit
	1999
[1]	2000	fi
	2001
	2002
[251]	2003	# COMPILER AUF RIAMS NEC UEBERSCHREIBEN
	2004	[[ $localhost = necriam ]] && compiler_name=mpif90
[1]	2005
	2006
	2007
	2008	# TEMPORAEREN KATALOGNAMEN BESTIMMEN
	2009	kennung=$RANDOM
	2010	if [[ "$tmp_user_catalog" = "" ]]
	2011	then
[312]	2012	if [[ $localhost = ibmh ]]
[1]	2013	then
[312]	2014	tmp_user_catalog=$SCRATCH
[1]	2015	elif [[ $localhost = nech ]]
	2016	then
	2017	tmp_user_catalog=$WRKSHR
	2018	else
	2019	tmp_user_catalog=/tmp
	2020	fi
	2021	fi
	2022	TEMPDIR=$tmp_user_catalog/${usern}.$kennung
	2023
	2024
	2025	# KATALOGNAMEN FUER ZWISCHENSPEICHERUNG VON FORTSETZUNGSLAUFDATEIEN
	2026	# BESTIMMEN
	2027	if [[ "$tmp_data_catalog" = "" ]]
	2028	then
[622]	2029	if [[ $localhost = nech ]]
[1]	2030	then
	2031	tmp_data_catalog=$WRKSHR/mrun_restart_data
	2032	else
	2033	tmp_data_catalog=/tmp/mrun_restart_data
	2034	fi
	2035	fi
	2036
	2037
	2038	# EVENTUELL BEI LOKALEN RECHNUNGEN $-ZEICHEN IN ENVIRONMENT-VARIABLEN
	2039	# ERSETZEN
	2040	if [[ $do_remote = false && $do_compile = true ]]
	2041	then
	2042	eval fopts=\"$fopts\"
	2043	eval lopts=\"$lopts\"
	2044	fi
	2045
	2046
	2047
	2048	# COMPILE- UND LINK-OPTIONEN BESTIMMEN
[260]	2049	fopts="$fopts $netcdf_inc $dvr_inc"
	2050	lopts="$lopts $netcdf_lib $dvr_lib"
[1]	2051	ROPTS="$ropts"
[717]	2052	if [[ ( $(echo $host \| cut -c1-3) = nec \|\| $(echo $host \| cut -c1-3) = ibm \|\| $host = lckyoto \|\| $host = lcsgih \|\| $host = lcsgib \|\| $host = lctit \|\| $host = lcfimm \|\| $host = lcxt4 \|\| $host = lcxt5m \|\| $host = lck \|\| $host = lckordi) && -n $numprocs ]]
[1]	2053	then
	2054	XOPT="-X $numprocs"
	2055	fi
	2056
	2057
	2058
	2059	# PRUEFEN DER CPU-ZEIT. (CPUMAX WIRD ALS ENV-VARIABLE VOM HAUTPRO-
	2060	# GRAMM BENOETIGT
	2061	done=false
	2062	while [[ $done = false ]]
	2063	do
	2064	cputime=$cpumax
	2065	if (( $cputime == 0 ))
	2066	then
[246]	2067	if [[ $do_batch = true ]]
	2068	then
	2069	printf "\n +++ cpu-time is undefined"
	2070	printf "\n >>> Please type CPU-time in seconds as INTEGER:"
	2071	printf "\n >>> "
	2072	read cputime 1>/dev/null 2>&1
	2073	else
	2074	cputime=10000000 # NO CPU LIMIT FOR INTERACTIVE RUNS
	2075	fi
[1]	2076	else
	2077	done=true
	2078	fi
	2079	cpumax=$cputime
	2080	done
	2081
	2082	(( minuten = cputime / 60 ))
	2083	(( sekunden = cputime - minuten * 60 ))
	2084
	2085
	2086	# PRUEFEN DER KERNSPEICHERANFORDERUNG
	2087	if [[ $do_batch = true ]]
	2088	then
	2089	done=false
	2090	while [[ $done = false ]]
	2091	do
	2092	if (( memory == 0 ))
	2093	then
	2094	printf "\n +++ memory demand is undefined"
	2095	printf "\n >>> Please type memory in MByte per process as INTEGER:"
	2096	printf "\n >>> "
	2097	read memory 1>/dev/null 2>&1
	2098	else
	2099	done=true
	2100	fi
	2101	done
	2102	fi
	2103
	2104
	2105	# PRUEFEN, OB FUER REMOTE-RECHNUNGEN EIN BENUTZERNAME ANGEGEBEN WURDE
	2106	if [[ $do_remote = true && -z $remote_username ]]
	2107	then
	2108	while [[ -z $remote_username ]]
	2109	do
	2110	printf "\n +++ username on remote host \"$host\" is undefined"
	2111	printf "\n >>> Please type username:"
	2112	printf "\n >>> "
	2113	read remote_username
	2114	done
	2115	mc="$mc -u$remote_username"
	2116	fi
	2117
[221]	2118
[493]	2119	# set module load command and export for subjob
	2120	if [[ "$modules" != "" ]]
	2121	then
[678]	2122	if [[ $host = lctit ]]
	2123	then
	2124	export module_calls=". $modules"
	2125	else
	2126	export module_calls="module load ${modules};"
	2127	fi
[493]	2128	fi
	2129
[683]	2130	# bugfix for wrong netcdf module and for netCDF4 usage in case of mpt
[562]	2131	if [[ $host = lcsgib \|\| $host = lcsgih ]]
[555]	2132	then
	2133	if [[ $(echo $module_calls \| grep -c netcdf/3.6.3-intel) != 0 ]]
	2134	then
[562]	2135	export module_calls="$module_calls export LD_LIBRARY_PATH=/sw/dataformats/netcdf/3.6.3-intel/lib:\$LD_LIBRARY_PATH;"
[555]	2136	fi
[721]	2137	# if [[ $(echo $module_calls \| grep -c mpt) != 0 ]]
	2138	# then
	2139	# export module_calls="$module_calls export LD_LIBRARY_PATH=/sw/sgi/mpt/2011-02-07/lib:\$LD_LIBRARY_PATH;"
	2140	# echo "*** module_calls = $module_calls"
	2141	# fi
[555]	2142	fi
[493]	2143
[555]	2144
[221]	2145	# SET DEFAULT VALUE FOR MPI MODULE TO BE USED ON SGI-ALTIX
	2146	if [[ $host = lcsgib \|\| $host = lcsgih ]]
	2147	then
[493]	2148	if [[ $(echo $modules \| grep -c mpt ) != 0 ]]
	2149	then
	2150	mpilib=mpt
	2151	elif [[ $(echo $modules \| grep -c mvapich ) != 0 ]]
	2152	then
	2153	mpilib=mvapich
[738]	2154	elif [[ $(echo $modules \| grep -c impi ) != 0 ]]
	2155	then
	2156	mpilib=impi
[493]	2157	fi
[221]	2158	fi
	2159
	2160
	2161
[1]	2162	###########################################################################
	2163	# HEADER-AUSGABE
	2164	###########################################################################
	2165
	2166
	2167	calltime=$(date)
	2168	printf "\n"
	2169	# [[ $silent = false ]] && clear
	2170	printf "#--------------------------------------------------------------# \n"
	2171	printf "\| $version$calltime \| \n"
	2172	printf "\| \| \n"
	2173	spalte1="called on:"; spalte2=$localhost_realname
	2174	printf "\| $spalte1$spalte2 \| \n"
	2175	if [[ $local_compile = false ]]
	2176	then
	2177	if [[ $do_remote = true ]]
	2178	then
	2179	spalte1="execution on:"; spalte2="$host (username: $remote_username)"
	2180	else
	2181	spalte1="execution on:"; spalte2="$host ($localhost_realname)"
	2182	fi
	2183	else
	2184	spalte1="compiling test only!"; spalte2=""
	2185	fi
	2186	printf "\| $spalte1$spalte2 \| \n"
	2187	if [[ -n $numprocs ]]
	2188	then
	2189	spalte1="number of PEs:"; spalte2=$numprocs
	2190	printf "\| $spalte1$spalte2 \| \n"
	2191	fi
	2192	if [[ -n $tasks_per_node ]]
	2193	then
	2194	spalte1="tasks per node:"; spalte2="$tasks_per_node (number of nodes: $nodes)"
	2195	printf "\| $spalte1$spalte2 \| \n"
	2196	fi
[622]	2197	if [[ $use_openmp = true ]]
[1]	2198	then
	2199	spalte1="threads per task:"; spalte2="$threads_per_task"
	2200	printf "\| $spalte1$spalte2 \| \n"
	2201	fi
	2202	printf "\| \| \n"
	2203	if [[ $do_compile = true ]]
	2204	then
[475]	2205	if [[ "$mopts" != "" ]]
	2206	then
[493]	2207	spalte1="make options:"; spalte2=$mopts
[475]	2208	printf "\| $spalte1$spalte2 \| \n"
	2209	zeile=$(echo "$mopts" \| cut -c41-)
	2210	while [[ "$zeile" != "" ]]
	2211	do
	2212	spalte1=""
	2213	spalte2=$zeile
	2214	printf "\| $spalte1$spalte2 \| \n"
	2215	zeile=$(echo "$zeile" \| cut -c41-)
	2216	done
	2217	fi
	2218
[493]	2219	spalte1="cpp directives:"; spalte2=$cpp_options
[1]	2220	printf "\| $spalte1$spalte2 \| \n"
[82]	2221	zeile=$(echo "$cpp_options" \| cut -c41-)
[1]	2222	while [[ "$zeile" != "" ]]
	2223	do
	2224	spalte1=""
	2225	spalte2=$zeile
	2226	printf "\| $spalte1$spalte2 \| \n"
	2227	zeile=$(echo "$zeile" \| cut -c41-)
	2228	done
	2229
[493]	2230	spalte1="compiler options:"; spalte2="$fopts"
[1]	2231	printf "\| $spalte1$spalte2 \| \n"
[82]	2232	zeile=$(echo "$fopts" \| cut -c41-)
[1]	2233	while [[ "$zeile" != "" ]]
	2234	do
	2235	spalte1=""
	2236	spalte2=$zeile
	2237	printf "\| $spalte1$spalte2 \| \n"
	2238	zeile=$(echo "$zeile" \| cut -c41-)
	2239	done
	2240
[493]	2241	spalte1="linker options:"; spalte2=$lopts
[1]	2242	printf "\| $spalte1$spalte2 \| \n"
[82]	2243	zeile=$(echo "$lopts" \| cut -c41-)
[1]	2244	while [[ "$zeile" != "" ]]
	2245	do
	2246	spalte1=""
	2247	spalte2=$zeile
	2248	printf "\| $spalte1$spalte2 \| \n"
	2249	zeile=$(echo "$zeile" \| cut -c41-)
	2250	done
	2251
[493]	2252	spalte1="modules to be load:"; spalte2=$modules
	2253	printf "\| $spalte1$spalte2 \| \n"
	2254	zeile=$(echo "$modules" \| cut -c41-)
	2255	while [[ "$zeile" != "" ]]
	2256	do
	2257	spalte1=""
	2258	spalte2=$zeile
	2259	printf "\| $spalte1$spalte2 \| \n"
	2260	zeile=$(echo "$zeile" \| cut -c41-)
	2261	done
	2262
[1]	2263	spalte1="main program":; spalte2=$mainprog
	2264	printf "\| $spalte1$spalte2 \| \n"
	2265	else
	2266	spalte1=executable:; spalte2=$executable
	2267	printf "\| $spalte1$spalte2 \| \n"
	2268	fi
	2269	printf "\| \| \n"
	2270	spalte1="base name of files":; spalte2=$fname
	2271	printf "\| $spalte1$spalte2 \| \n"
	2272	if [[ $fname != $afname ]]
	2273	then
	2274	spalte1="base name of input files":; spalte2=$afname
	2275	printf "\| $spalte1$spalte2 \| \n"
	2276	fi
	2277	spalte1="INPUT control list":; spalte2=$input_list
	2278	printf "\| $spalte1$spalte2 \| \n"
	2279	spalte1="OUTPUT control list":; spalte2=$output_list
	2280	printf "\| $spalte1$spalte2 \| \n"
	2281
	2282	if [[ $do_batch = true \|\| "$LOADLBATCH" = yes ]]
	2283	then
	2284	spalte1="memory demand / PE":; spalte2="$memory MB"
	2285	printf "\| $spalte1$spalte2 \| \n"
	2286	spalte1=CPU-time:; spalte2="$minuten:$sekunden"
	2287	printf "\| $spalte1$spalte2 \| \n"
	2288	fi
	2289
	2290	if [[ $do_compile = true ]]
	2291	then
	2292	printf "\| \| \n"
	2293	printf "\| Files to be compiled: \| \n"
	2294	zeile=$source_list
	2295	while [[ "$zeile" != "" ]]
	2296	do
	2297	spalte3=$zeile
	2298	printf "\| $spalte3 \| \n"
	2299	zeile=$(echo "$zeile" \| cut -c61-)
	2300	done
	2301	fi
	2302	printf "#--------------------------------------------------------------#"
	2303
	2304
	2305
	2306	# BEDINGTE AUSGABE DER DATEIVERBINDUNGEN
	2307	if [[ $do_trace = true ]]
	2308	then
	2309	(( i = 0 ))
	2310	while (( i < iin ))
	2311	do
	2312	(( i = i + 1 ))
	2313	if (( i == 1 ))
	2314	then
	2315	printf "\n\n >>> INPUT-file assignments:\n"
	2316	fi
	2317	printf "\n ${localin[$i]} : ${absnamein[$i]}"
	2318	done
	2319	(( i = 0 ))
	2320	while (( i < iout ))
	2321	do
	2322	(( i = i + 1 ))
	2323	if (( i == 1 ))
	2324	then
	2325	printf "\n\n >>> OUTPUT-file assignments:\n"
	2326	fi
	2327	printf "\n ${localout[$i]} : ${pathout[$i]}"
	2328	done
	2329	(( i = 0 ))
	2330	while (( i < iic ))
	2331	do
	2332	(( i = i + 1 ))
	2333	if (( i == 1 ))
	2334	then
	2335	printf "\n\n >>> INPUT-commands:\n"
	2336	fi
	2337	printf "\n ${in_command[$i]}"
	2338	done
	2339	(( i = 0 ))
	2340	while (( i < ioc ))
	2341	do
	2342	(( i = i + 1 ))
	2343	if (( i == 1 ))
	2344	then
	2345	printf "\n\n >>> OUTPUT-commands:\n"
	2346	fi
	2347	printf "\n ${out_command[$i]}"
	2348	done
	2349	fi
	2350
	2351
	2352	# ABFRAGEN BEI AUFRUF AUF LOKALER MASCHINE
	2353	if [[ $remotecall = false && $silent = false && $jobmo != BATCH ]]
	2354	then
	2355	antwort=dummy
	2356	printf "\n\n"
	2357	while [[ "$antwort" != y && "$antwort" != Y && "$antwort" != n && "$antwort" != N ]]
	2358	do
	2359	printf " >>> everything o.k. (y/n) ? "
	2360	read antwort
	2361	done
	2362	if [[ $antwort = n \|\| $antwort = N ]]
	2363	then
	2364	locat=user_abort; (( iec = 0 )); exit
	2365	fi
	2366	if [[ $do_batch = true ]]
	2367	then
	2368	printf " >>> batch-job will be created and submitted"
	2369	else
	2370	if [[ $local_compile = false ]]
	2371	then
	2372	printf " >>> MRUN will now continue to execute on this machine"
	2373	else
	2374	printf " >>> a test compilation will now be carried out on this machine"
	2375	fi
	2376	fi
	2377	fi
	2378
	2379
	2380
	2381
	2382	# FALLS AUF DIESER MASCHINE GERECHNET WERDEN SOLL, WERDEN JETZT ENTSPRE-
	2383	# CHENDE AKTIONEN DURCHGEFUEHRT
	2384	if [[ $do_batch = false ]]
	2385	then
	2386
	2387
	2388	# TEMPORAEREN KATALOG ERZEUGEN
	2389	mkdir -p $TEMPDIR
	2390	chmod go+rx $TEMPDIR
	2391	tmpcreate=true
	2392
[493]	2393	# set striping on lustre file system
	2394	# if [[ $localhost = lcsgih ]]
	2395	# then
	2396	# lfs setstripe -s 8192k -c 16 $TEMPDIR
	2397	# lfs getstripe $TEMPDIR
	2398	# fi
[1]	2399
[493]	2400
[1]	2401	# SAEMTLICHE QUELLTEXT-DATEIEN BZW. AUSFUEHRBARES PROGRAMM IN
	2402	# TEMPORAERES VERZEICHNIS KOPIEREN
	2403	if [[ $do_compile = true ]]
	2404	then
	2405
	2406	# ON NEC, COMPILATION IS DONE ON HOST CROSS VIA CROSS COMPILING
	2407	# CREATE A TEMPORARY DIRECTORY ON THAT MACHINE (HOME MOUNTED VIA NFS)
	2408	if [[ $localhost = nech ]]
	2409	then
	2410	TEMPDIR_COMPILE=$HOME/work/${usern}.$kennung
	2411	if mkdir -p $TEMPDIR_COMPILE
	2412	then
	2413	printf "\n *** \"$TEMPDIR_COMPILE\" "
	2414	printf "\n is generated as temporary directory for cross compiling\n"
	2415	else
	2416	printf "\n +++ creating directory \"$TEMPDIR_COMPILE\" "
	2417	printf "\n needed for cross compilation failed"
	2418	locat=compile
	2419	exit
	2420	fi
	2421	else
	2422	TEMPDIR_COMPILE=$TEMPDIR
	2423	fi
	2424
	2425
[82]	2426	# PFADNAMEN FUER DAS MAKE-DEPOSITORY ERMITTELN
	2427	line=""
	2428	grep "%depository_path" $config_file > tmp_mrun
	2429	while read line
	2430	do
	2431	if [[ "$line" != "" && $(echo $line \| cut -c1) != "#" ]]
	2432	then
	2433	if [[ "$(echo $line \| cut -d" " -s -f3)" = "" ]]
[1]	2434	then
[82]	2435	global_depository_path=`echo $line \| cut -d" " -s -f2`
[1]	2436	fi
[82]	2437	fi
	2438	done < tmp_mrun
[1]	2439
[82]	2440	line=""
	2441	grep " $localhost" $config_file \| grep "%depository_path" > tmp_mrun
	2442	while read line
	2443	do
	2444	if [[ "$line" != "" && $(echo $line \| cut -c1) != "#" ]]
[1]	2445	then
[211]	2446	if [[ "$(echo $line \| cut -d" " -s -f4)" = "$cond1" && "$(echo $line \| cut -d" " -s -f5)" = "$cond2" ]]
	2447	then
	2448	local_depository_path=`echo $line \| cut -d" " -s -f2`
	2449	fi
[1]	2450	fi
[82]	2451	done < tmp_mrun
[1]	2452
[82]	2453	if [[ "$local_depository_path" = "" ]]
	2454	then
	2455	if [[ "$global_depository_path" != "" ]]
[1]	2456	then
[82]	2457	local_depository_path=$global_depository_path
[1]	2458	else
[82]	2459	printf "\n\n +++ no depository path found in configuration file"
	2460	printf "\n for local host \"$localhost\" "
	2461	printf "\n please set \"\%depository_path\" in configuration file\n"
	2462	locat=config_file; exit
[1]	2463	fi
[82]	2464	fi
	2465	eval local_depository_path=$local_depository_path
[215]	2466	[[ "$cond1" != "" ]] && local_depository_path=${local_depository_path}_$cond1
	2467	[[ "$cond2" != "" ]] && local_depository_path=${local_depository_path}_$cond2
[1]	2468
[215]	2469
[82]	2470	basename=`print $mainprog \| cut -f1 -d"."`
	2471	eval make_depository=${local_depository_path}/${basename}_current_version.tar
	2472	if [[ ! -f $make_depository ]]
	2473	then
	2474	printf "\n"
	2475	printf "\n *** WARNING: make depository \"$make_depository\" not found"
	2476	printf "\n \"make\" will fail, if the Makefile or other source files are missing\n"
	2477	else
	2478	cp $make_depository $TEMPDIR_COMPILE
[1]	2479	cd $TEMPDIR_COMPILE
[82]	2480	tar -xf $make_depository > /dev/null 2>&1
[1]	2481	cd - > /dev/null
	2482	fi
[82]	2483
	2484	cp SOURCES_FOR_RUN_$fname/* $TEMPDIR_COMPILE
	2485
[1]	2486	else
[475]	2487
[1]	2488	cp $executable ${TEMPDIR}/a.out
[475]	2489
[1]	2490	fi
	2491
	2492
	2493	# WECHSEL IN TEMPORAEREN KATALOG
[475]	2494	cd $TEMPDIR
[1]	2495	printf "\n *** changed to temporary directory: $TEMPDIR"
	2496
	2497
	2498	# OUTPUT-DATEI-VERBINDUNGEN AUF TEMPORAERER DATEI ABLEGEN
	2499	# DIESE DATEI KANN VON SPAETER AUFZURUFENDEN BENUTZERPROZEDUREN GELESEN
	2500	# WERDEN, UM ZU LOKALEN DATEINAMEN GEHOERENDE PERMANENTE NAMEN ZU
	2501	# ERMITTELN
	2502	(( i = 0 ))
	2503	while (( i < iout ))
	2504	do
	2505	(( i = i + 1 ))
	2506	if [[ "${actionout[$i]}" = tr \|\| "${actionout[$i]}" = tra \|\| "${actionout[$i]}" = trpe ]]
	2507	then
	2508	printf "${localout[$i]} ${actionout[$i]}\n${pathout[$i]}\n${localhost}_${fname}${endout[$i]}\n" >> OUTPUT_FILE_CONNECTIONS
	2509	else
	2510	printf "${localout[$i]} ${actionout[$i]}\n${pathout[$i]}\n${frelout[$i]}\n" >> OUTPUT_FILE_CONNECTIONS
	2511	fi
	2512	done
	2513
	2514
	2515	# EVTL. UEBERSETZUNGSAKTIONEN STARTEN
	2516	if [[ $do_compile = true ]]
	2517	then
	2518
	2519
	2520	# COMPILING WITH MAKE (ON NEC COMPILER IS CALLED ON HOST CROSS)
[82]	2521	printf "\n\n\n *** compilation starts \n$striche\n"
	2522	printf " *** compilation with make using following options:\n"
[211]	2523	printf " make depository: $make_depository"
[475]	2524	if [[ "$mopts" != "" ]]
	2525	then
	2526	printf " make options: $mopts\n"
	2527	fi
[82]	2528	printf " compilername: $compiler_name\n"
[475]	2529	printf " compiler options: $fopts\n"
[493]	2530	printf " preprocessor directives: $cpp_options \n"
	2531	printf " linker options: $lopts \n"
	2532	if [[ "$modules" != "" ]]
	2533	then
	2534	printf " modules to be load: $modules \n"
	2535	fi
[82]	2536	printf " source code files: $source_list \n"
	2537
	2538	if [[ $localhost = nech ]]
[1]	2539	then
[493]	2540	ssh 136.172.44.192 -l $usern ". /SX/opt/etc/initsx.sh; $module_calls cd \$HOME/work/${usern}.$kennung; sxmake $mopts -f Makefile PROG=a.out F90=$compiler_name COPT=\"$cpp_options\" F90FLAGS=\"$fopts\" LDFLAGS=\"$lopts\" "
[82]	2541	cp $TEMPDIR_COMPILE/a.out .
	2542	[[ $? != 0 ]] && compile_error=true
	2543	rm -rf $TEMPDIR_COMPILE
[312]	2544	elif [[ $localhost = ibmh ]]
	2545	then
[325]	2546	printf " compiler is called via ssh on \"plogin1\" \n"
[493]	2547	ssh plogin1 -l $usern "export PATH=/sw/ibm/xlf/12.1.0.3/usr/bin:$PATH; $module_calls cd $TEMPDIR; make $mopts -f Makefile PROG=a.out F90=$compiler_name COPT=\"$cpp_options\" F90FLAGS=\"$fopts\" LDFLAGS=\"$lopts\" "
[312]	2548	[[ ! -f a.out ]] && compile_error=true
	2549	continue # ANDERENFALLS IST STATUS=1, FALLS A.OUT VORHANDEN
[197]	2550	elif [[ $localhost = lcsgib ]]
	2551	then
[540]	2552	printf " compiler is called via ssh on \"bicegate0\" \n"
	2553	ssh bicegate0 -l $usern ". /usr/share/modules/init/bash; $module_calls cd $TEMPDIR; make $mopts -f Makefile PROG=a.out F90=$compiler_name COPT=\"$cpp_options\" F90FLAGS=\"$fopts\" LDFLAGS=\"$lopts\" "
[197]	2554	[[ ! -f a.out ]] && compile_error=true
	2555	continue # ANDERENFALLS IST STATUS=1, FALLS A.OUT VORHANDEN
	2556	elif [[ $localhost = lcsgih ]]
	2557	then
[738]	2558	printf " compiler is called via ssh on \"hicegate0\" \n"
	2559	ssh hicegate0 -l $usern ". /usr/share/modules/init/bash; $module_calls cd $TEMPDIR; make $mopts -f Makefile PROG=a.out F90=$compiler_name COPT=\"$cpp_options\" F90FLAGS=\"$fopts\" LDFLAGS=\"$lopts\" 2>&1 "
[197]	2560	[[ ! -f a.out ]] && compile_error=true
	2561	continue # ANDERENFALLS IST STATUS=1, FALLS A.OUT VORHANDEN
[1]	2562	else
[544]	2563	[[ $localhost = lcxt4 ]] && . /opt/modules/default/init/ksh
[493]	2564	[[ "$module_calls" != "" ]] && eval $module_calls
[475]	2565	make $mopts -f Makefile PROG=a.out F90=$compiler_name COPT="$cpp_options" F90FLAGS="$fopts" LDFLAGS="$lopts"
[82]	2566	fi
[1]	2567
	2568	if [[ $? != 0 \|\| "$compile_error" = true \|\| "$module_compile_error" = true ]]
	2569	then
	2570	printf "\n +++ error occured while compiling or linking"
	2571	locat=compile
[181]	2572
[182]	2573	# WORKAROUND: REMOVE IF CONSTRUCT LATER, BUT KEEP THE EXIT!
[197]	2574	# if [[ $localhost != lcsgib && $localhost != lcsgih ]]
	2575	# then
[181]	2576	exit
[197]	2577	# else
	2578	# locat=normal
	2579	# fi
[1]	2580	else
	2581	printf "$striche\n *** compilation finished \n"
	2582	fi
	2583	fi
	2584
	2585
	2586	# FALLS NUR TESTWEISE KOMPILIERT WERDEN SOLLTE, IST MRUN JETZT FERTIG
	2587	if [[ $local_compile = true ]]
	2588	then
	2589	cd $HOME
	2590	rm -rf $TEMPDIR
	2591	locat=local_compile; exit
	2592	fi
	2593
	2594
	2595	# BEREITSTELLEN DER INPUT-DATEIEN
	2596	# SCHLEIFE UEBER ALLE VOM BENUTZER ANGEGEBENEN DATEIEN
	2597	(( i = 0 ))
	2598	while (( i < iin ))
	2599	do
	2600	(( i = i + 1 ))
	2601	if (( i == 1 ))
	2602	then
	2603	printf "\n\n *** providing INPUT-files:\n$striche"
	2604	fi
	2605
	2606
	2607	# OPTIONALE DATEIEN BEI NICHTVORHANDENSEIN UEBERGEHEN
	2608	if [[ "${transin[$i]}" = unavailable ]]
	2609	then
	2610	if [[ "${extin[$i]}" = "" \|\| "${extin[$i]}" = " " ]]
	2611	then
	2612	printf "\n +++ WARNING: input file \"${pathin[$i]}/${afname}${endin[$i]}\" "
	2613	printf "\n is not available!"
	2614	else
	2615	printf "\n +++ WARNING: input file \"${pathin[$i]}/${afname}${endin[$i]}.${extin[$i]}\" "
	2616	printf "\n is not available!"
	2617	fi
	2618	continue
	2619	fi
	2620
	2621	# PRUEFEN, OB EINZELDATEI ODER DATEI PRO PROZESSOR
	2622	files_for_pes=false; datentyp=file
	2623	if [[ "${actionin[$i]}" = pe && -n $numprocs ]]
	2624	then
	2625	files_for_pes=true; datentyp=directory
	2626	actionin[$i]=""
	2627	elif [[ "${actionin[$i]}" = pe && ! -n $numprocs ]]
	2628	then
	2629	actionin[$i]=""
	2630	elif [[ "${actionin[$i]}" = arpe && -n $numprocs ]]
	2631	then
	2632	files_for_pes=true; datentyp=directory
	2633	actionin[$i]="ar"
	2634	elif [[ "${actionin[$i]}" = arpe && ! -n $numprocs ]]
	2635	then
	2636	actionin[$i]="ar"
	2637	elif [[ "${actionin[$i]}" = flpe && -n $numprocs ]]
	2638	then
	2639	files_for_pes=true; datentyp=directory
	2640	actionin[$i]="fl"
	2641	elif [[ "${actionin[$i]}" = flpe && ! -n $numprocs ]]
	2642	then
	2643	actionin[$i]="fl"
	2644	fi
	2645
	2646	if [[ $files_for_pes = true ]]
	2647	then
	2648	printf "\n >>> INPUT: ${absnamein[$i]}/.... to ${localin[$i]}"
	2649	else
	2650	printf "\n >>> INPUT: ${absnamein[$i]} to ${localin[$i]}"
	2651	fi
	2652
	2653	# INPUT-DATEI FUER EINEN FORTSETZUNGSLAUF. ES WIRD GEPRUEFT,
	2654	# OB DIESE DATEI NOCH AUF DEM TEMPORAEREN DATENKATALOG VORHANDEN
	2655	# IST. FALLS NICHT, WIRD VERSUCHT, SIE ANSCHLIESSEND VOM ARCHIV-
	2656	# SERVER ZU HOLEN
	2657	if [[ "${actionin[$i]}" = fl ]]
	2658	then
	2659	printf "\n $datentyp will be fetched from temporary directory \"${tmp_data_catalog}\" !"
	2660	if [[ $files_for_pes = false ]]
	2661	then
	2662	if [[ -f "$tmp_data_catalog/${frelin[$i]}" ]]
	2663	then
	2664	ln $tmp_data_catalog/${frelin[$i]} ${localin[$i]}
	2665	got_tmp[$i]=true
	2666	elif [[ -f "$WORK/${frelin[$i]}" && $ignore_archive_error = true ]]
	2667	then
	2668	printf "\n +++ $datentyp not found in \"$tmp_data_catalog\" !"
	2669	printf "\n *** trying to use backup copy in \"$WORK\" "
	2670	cp $WORK/${frelin[$i]} ${localin[$i]}
	2671	else
	2672	printf "\n +++ $datentyp not found in \"$tmp_data_catalog\" "
	2673	printf "\n or \"$tmp_data_catalog\" does not exist!"
	2674	printf "\n *** trying to get copy from archive"
	2675	actionin[$i]=ar
	2676	fi
	2677	else
	2678	if [[ -d "$tmp_data_catalog/${frelin[$i]}" ]]
	2679	then
	2680	mkdir ${localin[$i]}
[149]	2681	cd $tmp_data_catalog/${frelin[$i]}
	2682	for file in $(ls *)
	2683	do
	2684	ln $file $TEMPDIR/${localin[$i]}
	2685	done
	2686	cd $TEMPDIR
[1]	2687	got_tmp[$i]=true
	2688	elif [[ -d "$WORK/${frelin[$i]}" && $ignore_archive_error = true ]]
	2689	then
	2690	printf "\n +++ $datentyp not found in \"$tmp_data_catalog\" !"
	2691	printf "\n *** trying to use backup copy in \"$WORK\" "
	2692	cp -r $WORK/${frelin[$i]} ${localin[$i]}
	2693	else
	2694	printf "\n +++ $datentyp not found in \"$tmp_data_catalog\" "
	2695	printf "\n or \"$tmp_data_catalog\" does not exist!"
	2696	printf "\n *** trying to get copy from archive"
	2697	actionin[$i]=ar
	2698	fi
	2699	fi
	2700	fi
	2701
	2702
	2703	# DATEI LIEGT AUF ARCHIV-SERVER
	2704	if [[ "${actionin[$i]}" = ar ]]
	2705	then
	2706
	2707	if [[ $files_for_pes = false ]]
	2708	then
	2709	printf "\n file will be restored from archive-system ($archive_system)!"
	2710	else
	2711	printf "\n directory will be restored from archive-system ($archive_system)!"
	2712	fi
	2713
	2714	file_restored=false
	2715
	2716	if [[ $archive_system = asterix ]]
	2717	then
	2718	do_stagein=true
	2719	(( stagein_anz = 0 ))
	2720	while [[ $do_stagein = true ]]
	2721	do
	2722	if [[ $files_for_pes = false ]]
	2723	then
	2724	stagein -O ${frelin[$i]} > STAGEIN_OUTPUT
	2725	else
	2726	stagein -t -O ${frelin[$i]} > STAGEIN_OUTPUT
	2727	fi
	2728	cat STAGEIN_OUTPUT
	2729	if [[ $(grep -c "st.msg:i24" STAGEIN_OUTPUT) != 0 ]]
	2730	then
	2731	file_restored=true
	2732	do_stagein=false
	2733	else
	2734	(( stagein_anz = stagein_anz + 1 ))
	2735	if (( stagein_anz == 10 ))
	2736	then
	2737	printf "\n +++ stagein stoped after 10 tries"
	2738	locat=stage
	2739	exit
	2740	fi
	2741	printf "\n +++ restoring from archive failed, trying again:"
	2742	sleep 900
	2743	fi
	2744	done
	2745	elif [[ $archive_system = DMF ]]
	2746	then
	2747	if [[ $files_for_pes = false ]]
	2748	then
	2749	printf "\n +++ restoring of single files impossible with $archive_system !\n"
	2750	locat=DMF
	2751	exit
	2752	else
	2753	find $ARCHIVE/${frelin[$i]} -type m -print \| dmget
	2754	cp -r $ARCHIVE/${frelin[$i]} $PWD
	2755	file_restored=true
	2756	fi
	2757	elif [[ $archive_system = tivoli ]]
	2758	then
	2759	if [[ $files_for_pes = false ]]
	2760	then
[399]	2761	if [[ $localhost = lcsgih ]]
[1]	2762	then
[399]	2763	ssh $usern@hicedata.hlrn.de "cp $PERM/${frelin[$i]} $PWD"
[1]	2764	else
[399]	2765	ssh $usern@bicedata.hlrn.de "cp $PERM/${frelin[$i]} $PWD"
[1]	2766	fi
	2767	else
	2768	(( inode = 0 ))
	2769	while (( inode < nodes ))
	2770	do
[399]	2771	if [[ $localhost = lcsgih ]]
[1]	2772	then
[399]	2773	ssh $usern@hicedata.hlrn.de "cd $PWD; tar xf $PERM/${frelin[$i]}/${frelin[$i]}.node_$inode.tar"
[1]	2774	else
[399]	2775	ssh $usern@bicedata.hlrn.de "cd $PWD; tar xf $PERM/${frelin[$i]}/${frelin[$i]}.node_$inode.tar"
[1]	2776	fi
	2777	(( inode = inode + 1 ))
	2778	done
	2779	fi
	2780	file_restored=true
	2781	elif [[ $archive_system = ut ]]
	2782	then
	2783	if [[ $files_for_pes = false ]]
	2784	then
	2785	cp $UT/${frelin[$i]} .
	2786	else
	2787	(( inode = 0 ))
	2788	while (( inode < nodes ))
	2789	do
	2790	tar xf $UT/${frelin[$i]}/${frelin[$i]}.node_$inode.tar
	2791	(( inode = inode + 1 ))
	2792	done
	2793	fi
	2794	file_restored=true
	2795	else
	2796	printf "\n +++ archive_system=\"$archive_system\" restore impossible!"
	2797	locat=rearchive
	2798	exit
	2799	fi
	2800
	2801	if [[ $file_restored = true ]]
	2802	then
	2803
	2804	# DATEI AUCH AUF TEMPORAERES DATENVERZEICHNIS LEGEN, DAMIT
	2805	# SIE BEI WEITEREN ZUGRIFFEN NOCH VORHANDEN IST
	2806	[[ ! -d $tmp_data_catalog ]] && mkdir -p $tmp_data_catalog; chmod g+rx $tmp_data_catalog
	2807	if [[ $files_for_pes = false ]]
	2808	then
	2809	ln -f ${frelin[$i]} $tmp_data_catalog/${frelin[$i]}
	2810	else
	2811	mkdir $tmp_data_catalog/${frelin[$i]}
	2812	ln -f ${frelin[$i]}/* $tmp_data_catalog/${frelin[$i]}
	2813	fi
	2814	got_tmp[$i]=true
	2815
	2816	# DATEI UNTER LOKALEM NAMEN ZUR VERFUEGUNG STELLEN
	2817	mv ${frelin[$i]} ${localin[$i]}
	2818
	2819	fi
	2820	fi
	2821
	2822
	2823	# DATEI LIEGT IM VOM BENUTZER ANGEGEBENEN VERZEICHNIS
	2824	if [[ "${actionin[$i]}" = "" \|\| "${actionin[$i]}" = "di" \|\| "${actionin[$i]}" = "npe" ]]
	2825	then
	2826
	2827	if [[ "${actionin[$i]}" = "npe" && -n $numprocs ]]
	2828	then
	2829
	2830	# DATEI WIRD FUER DIE PROZESSOREN EINES PARALLERECHNERS BEREITGESTELLT
	2831	printf "\n file will be provided for $numprocs processors"
	2832	mkdir ${localin[$i]}
	2833	ival=$numprocs
	2834	(( ii = 0 ))
	2835	while (( ii <= ival-1 ))
	2836	do
	2837	if (( ii < 10 ))
	2838	then
	2839	cp ${absnamein[$i]} ${localin[$i]}/_000$ii
	2840	elif (( ii < 100 ))
	2841	then
	2842	cp ${absnamein[$i]} ${localin[$i]}/_00$ii
	2843	elif (( ii < 1000 ))
	2844	then
	2845	cp ${absnamein[$i]} ${localin[$i]}/_0$ii
	2846	else
	2847	cp ${absnamein[$i]} ${localin[$i]}/_$ii
	2848	fi
	2849	(( ii = ii + 1 ))
	2850	done
	2851
	2852	else
	2853
	2854	if [[ $files_for_pes = true ]]
	2855	then
	2856
	2857	# DIE DEN PROZESSOREN EINES PARALLELRECHNERS ZUGEHOERIGEN
	2858	# DATEIEN WERDEN BEREITGESTELLT, INDEM ZUERST DER GESAMTE
	2859	# KATALOGINHALT KOPIERT UND DANN DIE EINZELNEN DATEIEN
	2860	# PER MOVE UMBENANNT WERDEN
	2861	printf "\n providing $numprocs files for the respective processors"
	2862	mkdir ${localin[$i]}
[183]	2863	if [[ $link_local_input = true ]]
	2864	then
	2865	printf " using ln -f\n"
	2866	cd ${absnamein[$i]}
	2867	for file in $(ls *)
	2868	do
	2869	ln -f $file ${localin[$i]}
	2870	done
	2871	cd $TEMPDIR
	2872	fi
	2873	# If "ln -f" fails of if "$link_local_input = false" do a normal "cp -r"
	2874	if [[ ! -f "${localin[$i]}/_0000" ]]
	2875	then
	2876	if [[ $link_local_input = true ]]
	2877	then
	2878	printf " ln failed for .../_0000, using cp...\n"
	2879	fi
	2880	cp -r ${absnamein[$i]}/* ${localin[$i]}
	2881	fi
[1]	2882
	2883	else
	2884	# BEREITSTELLUNG AUF EINPROZESSORRECHNERN
[183]	2885	if [[ $link_local_input = true ]]
	2886	then
	2887	printf " using ln -f\n"
	2888	ln -f ${absnamein[$i]} ${localin[$i]}
	2889	fi
	2890	# If "ln -f" fails of if "$link_local_input = false" do a normal "cp"
	2891	if [[ ! -f "${localin[$i]}" ]]
	2892	then
	2893	if [[ $link_local_input = true ]]
	2894	then
	2895	printf " ln failed, using cp...\n"
	2896	fi
	2897	cp ${absnamein[$i]} ${localin[$i]}
	2898	fi
[1]	2899	fi
	2900	fi
	2901	fi
	2902
	2903	done
	2904	if (( i != 0 ))
	2905	then
	2906	printf "\n$striche\n *** all INPUT-files provided \n"
	2907	fi
	2908
	2909
	2910	# EVENTUELLE INPUT-KOMMANDOS ABARBEITEN
	2911	(( i = 0 ))
	2912	while (( i < iic ))
	2913	do
	2914	(( i = i + 1 ))
	2915	if (( i == 1 ))
	2916	then
	2917	printf "\n\n *** execution of INPUT-commands:\n$striche"
	2918	fi
	2919	printf "\n >>> ${in_command[$i]}"
	2920	eval ${in_command[$i]}
	2921	if (( i == iic ))
	2922	then
	2923	printf "\n$striche\n"
	2924	fi
	2925	done
	2926
	2927
	2928	# VERBLEIBENDE CPU-ZEIT BERECHNEN
	2929	cpurest=${cpumax}.
	2930
	2931
[260]	2932	# START DVR STREAMING SERVER
	2933	if [[ $(echo $package_list \| grep -c dvrp_graphics) != 0 ]]
	2934	then
	2935	if [[ "$dvr_server" != "" ]]
	2936	then
	2937
	2938	printf "\n\n *** preparing the dvr streaming server configuration file"
	2939
	2940	# Check, if a dvr server is already running
	2941	running_dvrserver_id=`echo $(ps -edaf \| grep .dvrserver.config \| grep -v grep) \| cut -d" " -f2`
	2942	if [[ "$running_dvrserver_id" != "" ]]
	2943	then
	2944
[261]	2945	printf "\n\n +++ WARNING: A dvr server with id=$running_dvrserver_id is already running!"
[260]	2946	printf "\n This server is used instead starting a new one!"
[261]	2947	printf "\n If required, script \"process_dvr_output\" has to be run manually."
[260]	2948
	2949	else
	2950
	2951	# COPY CONFIGURATION FILE FOR STREAMING SERVER FROM REPOSITORY TO HERE
	2952	if [[ -f ${PALM_BIN}/.dvrserver.config ]]
	2953	then
	2954	cp ${PALM_BIN}/.dvrserver.config .
	2955
	2956	# Entering the BASEDIR, UID and GID into this file
	2957	user_id=`id -u`
	2958	group_id=`id -g`
	2959	# & is needed as seperator, because TEMPDIR contains /
	2960	sed "s&<replace by dvr data directory>&${TEMPDIR}&g" .dvrserver.config > .dvrserver.1
	2961	sed "s/<replace by user id>/$user_id/g" .dvrserver.1 > .dvrserver.2
	2962	sed "s/<replace by group id>/$group_id/g" .dvrserver.2 > .dvrserver.3
	2963	mv .dvrserver.3 .dvrserver.config
	2964	rm .dvrserver.1 .dvrserver.2
	2965
	2966	# Start dvr server in background, get his id and print on terminal
	2967	$dvr_server .dvrserver.config >> DVR_LOGFILE 2>&1 &
	2968	dvrserver_id=`echo $(ps -edaf \| grep .dvrserver.config) \| cut -d" " -f2`
	2969	printf "\n *** streaming server with id=$dvrserver_id is started in background"
	2970	local_dvrserver_running=.TRUE.
	2971	else
	2972	printf "\n +++ missing file \".dvrserver.config\" in directory:"
	2973	printf "\n \"$PALM_BIN\" "
	2974	locat=dvr
	2975	exit
	2976	fi
	2977
	2978	fi
	2979
	2980	else
	2981	printf "\n\n --- INFORMATIVE: no dvr streaming server will be started"
	2982	fi
	2983	fi
	2984
	2985
	2986	# NAMELIST-DATEI MIT WERTEN VON ENVIRONMENT-VARIABLEN ERZEUGEN (ZU
	2987	# LESEN VON PALM)
	2988	cat > ENVPAR << %%END%%
	2989	&envpar run_identifier = '$fname', host = '$localhost',
	2990	write_binary = '$write_binary', tasks_per_node = $tasks_per_node,
	2991	maximum_cpu_time_allowed = ${cpumax}.,
	2992	revision = '$global_revision',
	2993	local_dvrserver_running = $local_dvrserver_running /
	2994
	2995	%%END%%
	2996
	2997
[1]	2998	# PROGRAMMSTART
	2999	printf "\n\n *** execution starts in directory\n \"`pwd`\"\n$striche\n"
	3000	PATH=$PATH:$TEMPDIR
	3001
[359]	3002	# MPI debug option (argument checking, slows down execution due to increased latency)
	3003	if [[ "$mpi_debug" = true ]]
	3004	then
	3005	export MPI_CHECK_ARGS=1
	3006	printf "\n +++ MPI_CHECK_ARGS=$MPI_CHECK_ARGS"
	3007	fi
	3008
[693]	3009	if [[ "$totalview" = true ]]
	3010	then
	3011	printf "\n *** totalview debugger will be used"
	3012	tv_opt="-tv"
	3013	else
	3014	tv_opt=""
	3015	fi
	3016
[1]	3017	if [[ "$cond1" = debug \|\| "$cond2" = debug ]]
	3018	then
	3019	if [[ "$ENVIRONMENT" = BATCH ]]
	3020	then
[211]	3021	if [[ $(echo $localhost \| cut -c1-5) != lcsgi ]]
	3022	then
	3023	printf "\n +++ debug is allowed in interactive mode only"
	3024	locat=debug
	3025	exit
	3026	fi
[1]	3027	fi
[5]	3028	if [[ $localhost = decalpha ]]
[1]	3029	then
	3030	dxladebug a.out
[622]	3031	elif [[ $localhost = ibmh ]]
[1]	3032	then
	3033
	3034	# SETUP THE IBM MPI ENVIRONMENT
	3035	export MP_SHARED_MEMORY=yes
	3036	export AIXTHREADS_SCOPE=S
	3037	export OMP_NUM_THREADS=$threads_per_task
	3038	export AUTHSTATE=files
	3039	export XLFRTEOPTS="nlwidth=132:err_recovery=no" # RECORD-LENGTH OF NAMELIST-OUTPUT
	3040
	3041	# FOLLOWING OPTIONS ARE MANDATORY FOR TOTALVIEW
	3042	export MP_ADAPTER_USE=shared
	3043	export MP_CPU_USE=multiple
	3044	export MP_TIMEOUT=1200
	3045
	3046	unset MP_TASK_AFFINITY
	3047
	3048	# SO FAR, TOTALVIEW NEEDS HOSTFILE MECHANISM FOR EXECUTION
	3049	#(( ii = 1 ))
	3050	#while (( ii <= $numprocs ))
	3051	#do
	3052	# echo $localhost_realname >> hostfile
	3053	# (( ii = ii + 1 ))
	3054	#done
	3055	#export MP_HOSTFILE=hostfile
	3056
	3057	if [[ "$LOADLBATCH" = yes ]]
	3058	then
	3059	totalview poe a.out $ROPTS
	3060	else
	3061	echo totalview poe -a a.out -procs $numprocs -rmpool 0 -nodes 1 $ROPTS
	3062	export TVDSVRLAUNCHCMD=ssh
	3063	totalview poe -a a.out -procs $numprocs -rmpool 0 -nodes 1 $ROPTS
	3064	fi
[211]	3065	elif [[ $(echo $localhost \| cut -c1-5) = lcsgi ]]
	3066	then
	3067	# CURRENTLY NO DEBUGGER ON LCSGI
	3068	if [[ $run_coupled_model = true ]]
	3069	then
	3070	printf "\n +++ no debug in coupled mode available on \"$localhost\" "
	3071	locat=debug
	3072	exit
[223]	3073	else
	3074	echo "no_coupling" > runfile_atmos
[211]	3075	fi
	3076	(( ii = $numprocs / $threads_per_task ))
	3077	export OMP_NUM_THREADS=$threads_per_task
	3078	echo "OMP_NUM_THREADS=$OMP_NUM_THREADS"
	3079	if [[ $threads_per_task != 1 ]]
	3080	then
	3081	printf "\n threads per task: $threads_per_task"
	3082	fi
	3083	printf "\n\n"
[223]	3084	if [[ $( echo $mpilib \| cut -c1-3 ) = mpt ]]
	3085	then
	3086	# export MPI_LAUNCH_TIMEOUT=360
	3087	if [[ "$totalview" = true ]]
	3088	then
	3089	printf "\n running totalview debugger"
[693]	3090	mpiexec_mpt $tv_opt -n $ii ./a.out $ROPTS < runfile_atmos
[223]	3091	else
	3092	mpiexec_mpt -np $ii ./a.out $ROPTS < runfile_atmos
	3093	fi
	3094	elif [[ $( echo $mpilib \| cut -c1-3 ) = mva ]]
	3095	then
	3096	# ulimit -s 300000 # A too large stack size causes problems
	3097	# export MV2_NUM_PORTS=2
	3098	# export MV2_CPU_MAPPING=0:1:2:3
[362]	3099	if [[ "$totalview" = true ]]
	3100	then
	3101	printf "\n running totalview debugger"
[693]	3102	mpiexec $tv_opt ./a.out $ROPTS < runfile_atmos
[362]	3103	else
	3104	mpiexec ./a.out $ROPTS < runfile_atmos
	3105	fi
[223]	3106	fi
[1]	3107	else
	3108	printf "\n +++ no debug available on \"$localhost\" "
	3109	printf "\n or not implemented in mrun so far"
	3110	locat=debug
	3111	exit
	3112	fi
[622]	3113
	3114	# end debug mode
[1]	3115	else
[622]	3116
	3117	# normal execution
[1]	3118	if [[ -n $numprocs ]]
	3119	then
	3120	# RUNNING THE PROGRAM ON PARALLEL MACHINES
	3121	if [[ $(echo $host \| cut -c1-3) = ibm ]]
	3122	then
	3123	# SETUP THE IBM MPI ENVIRONMENT
[693]	3124	if [[ $host != ibmh && $host != ibmkisti ]]
[312]	3125	then
	3126	export MP_SHARED_MEMORY=yes
	3127	export AIXTHREAD_SCOPE=S
	3128	export OMP_NUM_THREADS=$threads_per_task
	3129	export XLSMPOPTS="spins=0:yields=0:stack=20000000"
	3130	export AUTHSTATE=files
	3131	export XLFRTEOPTS="nlwidth=132:err_recovery=no" # RECORD-LENGTH OF NAMELIST-OUTPUT
	3132	# export MP_PRINTENV=yes
[1]	3133
[312]	3134	# TUNING-VARIABLEN ZUR VERBESSERUNG DER KOMMUNIKATION
	3135	# ZEIGEN ABER DERZEIT (SEP 04, FEDERATION) KAUM WIRKUNG
	3136	export MP_WAIT_MODE=poll
	3137	[[ $node_usage = not_shared ]] && export MP_SINGLE_THREAD=yes
	3138	fi
[693]	3139
	3140	if [[ $host = ibmkisti ]]
	3141	then
	3142	export LANG=en_US
	3143	export MP_SHARED_MEMORY=yes
[696]	3144	if [[ $threads_per_task = 1 ]]
	3145	then
	3146	export MP_SINGLE_THREAD=yes
	3147	export MEMORY_AFFINITY=MCM
	3148	else
	3149	export OMP_NUM_THREADS=$threads_per_task
	3150	fi
[693]	3151	fi
	3152
[1]	3153	if [[ "$LOADLBATCH" = yes ]]
	3154	then
	3155	printf "\n--- Control: OMP_NUM_THREADS = \"$OMP_NUM_THREADS\" \n"
	3156	if [[ "$cond1" = hpmcount \|\| "$cond2" = hpmcount ]]
	3157	then
[102]	3158	/opt/optibm/HPM_2_4_1/bin/hpmcount a.out $ROPTS
[1]	3159	else
[343]	3160	if [[ $run_coupled_model = false ]]
	3161	then
	3162	if [[ "$ocean_file_appendix" = true ]]
	3163	then
	3164	echo "precursor_ocean" > runfile_atmos
	3165	else
	3166	echo "precursor_atmos" > runfile_atmos
	3167	fi
	3168	else
	3169	(( iia = $numprocs_atmos / $threads_per_task ))
	3170	(( iio = $numprocs_ocean / $threads_per_task ))
	3171	printf "\n coupled run ($iia atmosphere, $iio ocean)"
	3172	printf "\n using $coupled_mode coupling"
	3173	printf "\n\n"
	3174	echo "coupled_run $iia $iio" > runfile_atmos
	3175	fi
	3176	poe ./a.out $ROPTS < runfile_atmos
[1]	3177	fi
	3178	else
[622]	3179	if [[ $localhost = ibmh \|\| $localhost = ibms ]]
[1]	3180	then
[102]	3181	poe a.out -procs $numprocs -nodes 1 -rmpool 0 $ROPTS
[693]	3182	elif [[ $localhost = ibmkisti \|\| $localhost = ibmku \|\| $localhost = ibmy ]]
[1]	3183	then
	3184	if [[ -f $hostfile ]]
	3185	then
	3186	cp $hostfile hostfile
	3187	else
	3188	(( ii = 1 ))
	3189	while (( ii <= $numprocs ))
	3190	do
	3191	echo $localhost_realname >> hostfile
	3192	(( ii = ii + 1 ))
	3193	done
	3194	fi
	3195	export MP_HOSTFILE=hostfile
[331]	3196	if [[ $run_coupled_model = false ]]
	3197	then
	3198	if [[ "$ocean_file_appendix" = true ]]
	3199	then
	3200	echo "precursor_ocean" > runfile_atmos
	3201	else
	3202	echo "precursor_atmos" > runfile_atmos
	3203	fi
	3204	else
	3205	(( iia = $numprocs_atmos / $threads_per_task ))
	3206	(( iio = $numprocs_ocean / $threads_per_task ))
	3207	printf "\n coupled run ($iia atmosphere, $iio ocean)"
	3208	printf "\n using $coupled_mode coupling"
	3209	printf "\n\n"
	3210	echo "coupled_run $iia $iio" > runfile_atmos
	3211	fi
[622]	3212	if [[ $localhost = ibmy ]]
	3213	then
	3214	./a.out -procs $tasks_per_node $ROPTS < runfile_atmos
	3215	else
	3216	poe ./a.out -procs $numprocs $ROPTS < runfile_atmos
	3217	fi
[331]	3218
[1]	3219	else
	3220	if [[ "$host_file" = "" ]]
	3221	then
	3222	printf "\n +++ no hostfile given in configuration file"
	3223	locat=config_file
	3224	exit
	3225	else
	3226	eval host_file=$host_file
	3227	fi
	3228	export MP_HOSTFILE=$host_file
[102]	3229	poe a.out -procs $numprocs -tasks_per_node $numprocs $ROPTS
[1]	3230	fi
	3231	fi
[251]	3232	elif [[ $host = nech \|\| $host = necriam ]]
[1]	3233	then
	3234	(( ii = nodes ))
	3235	if [[ $ii = 1 ]]
	3236	then
	3237	export F_ERRCNT=0 # acceptable number of errors before program is stopped
	3238	export MPIPROGINF=YES
[61]	3239	# export F_TRACE=YES\|FMT1\|FMT2 # output of ftrace informations to job protocol
[1]	3240	echo "*** execution on single node with mpirun"
[102]	3241	mpirun -np $numprocs ./a.out $ROPTS
[1]	3242	else
	3243	(( i = 0 ))
	3244	while (( i < ii ))
	3245	do
	3246	echo "-h $i -p $tasks_per_node -e ./mpi_exec_shell" >> multinode_config
	3247	(( i = i + 1 ))
	3248	done
	3249
	3250	echo "#!/bin/sh" > mpi_exec_shell
	3251	echo " " >> mpi_exec_shell
	3252	echo "set -u" >> mpi_exec_shell
	3253	echo "F_ERRCNT=0" >> mpi_exec_shell
	3254	echo "MPIPROGINV=YES" >> mpi_exec_shell
	3255	echo "OMP_NUM_THREADS=$threads_per_task" >> mpi_exec_shell
	3256	echo "cpurest=$cpurest" >> mpi_exec_shell
	3257	echo "fname=$fname" >> mpi_exec_shell
	3258	echo "localhost=$localhost" >> mpi_exec_shell
	3259	echo "return_addres=$return_addres" >> mpi_exec_shell
	3260	echo "return_username=$return_username" >> mpi_exec_shell
	3261	echo "tasks_per_node=$tasks_per_node" >> mpi_exec_shell
	3262	echo "write_binary=$write_binary" >> mpi_exec_shell
	3263	echo "use_seperate_pe_for_dvrp_output=$use_seperate_pe_for_dvrp_output" >> mpi_exec_shell
	3264	echo " " >> mpi_exec_shell
	3265	echo "export F_ERRCNT" >> mpi_exec_shell
	3266	echo "export MPIPROGINV" >> mpi_exec_shell
	3267	echo "export OMP_NUM_THREADS" >> mpi_exec_shell
	3268	echo "export cpurest" >> mpi_exec_shell
	3269	echo "export fname" >> mpi_exec_shell
	3270	echo "export localhost" >> mpi_exec_shell
	3271	echo "export return_addres" >> mpi_exec_shell
	3272	echo "export return_username" >> mpi_exec_shell
	3273	echo "export tasks_per_node" >> mpi_exec_shell
	3274	echo "export write_binary" >> mpi_exec_shell
	3275	echo "export use_seperate_pe_for_dvrp_output" >> mpi_exec_shell
	3276	echo " " >> mpi_exec_shell
[102]	3277	echo "exec ./a.out" >> mpi_exec_shell
[1]	3278
	3279	chmod u+x mpi_exec_shell
	3280	export MPIPROGINF=YES
	3281	mpirun -f multinode_config &
	3282	wait
	3283
	3284	fi
[440]	3285	elif [[ $(echo $host \| cut -c1-2) = lc && $host != lckyoto && $host != lctit ]]
[1]	3286	then
	3287
	3288	# COPY HOSTFILE FROM SOURCE DIRECTORY OR CREATE IT, IF IT
	3289	# DOES NOT EXIST
[179]	3290	if [[ $host != lcsgih && $host != lcsgib ]]
[1]	3291	then
[179]	3292	if [[ -f $hostfile ]]
	3293	then
	3294	cp $hostfile hostfile
[746]	3295	(( ii = $numprocs / $threads_per_task ))
[179]	3296	else
	3297	(( ii = 1 ))
	3298	while (( ii <= $numprocs / $threads_per_task ))
	3299	do
	3300	echo $localhost_realname >> hostfile
	3301	(( ii = ii + 1 ))
	3302	done
	3303	fi
	3304	eval zeile=\"`head -n $ii hostfile`\"
	3305	printf "\n *** running on: $zeile"
[1]	3306	fi
[179]	3307
[1]	3308	(( ii = $numprocs / $threads_per_task ))
	3309	export OMP_NUM_THREADS=$threads_per_task
[696]	3310	# echo "*** OMP_NUM_THREADS=$OMP_NUM_THREADS"
[1]	3311	if [[ $threads_per_task != 1 ]]
	3312	then
[405]	3313	# increase stack size to unlimited, because large runs
	3314	# may abort otherwise
	3315	ulimit -Ss unlimited
	3316	printf "\n threads per task: $threads_per_task stacksize: unlimited"
[1]	3317	fi
[102]	3318	if [[ $run_coupled_model = false ]]
[1]	3319	then
[291]	3320	if [[ "$ocean_file_appendix" = true ]]
	3321	then
	3322	echo "precursor_ocean" > runfile_atmos
	3323	else
	3324	echo "precursor_atmos" > runfile_atmos
	3325	fi
[102]	3326	printf "\n\n"
[179]	3327	if [[ $host = lcsgih \|\| $host = lcsgib ]]
[164]	3328	then
[221]	3329	if [[ $( echo $mpilib \| cut -c1-3 ) = mpt ]]
	3330	then
[366]	3331	# MPI_DSM_DISTRIBUTE not necessary when MPI_DSM_CPULIST is set
	3332	# export MPI_DSM_DISTRIBUTE=1
	3333	# MPI_DSM_CPULIST: pin MPI processes to cores
[377]	3334	if [[ $use_openmp = false ]]
	3335	then
[412]	3336	if [[ "$sgi_feature" = ice2 ]]
	3337	then
	3338	export MPI_DSM_CPULIST="0,4,1,5,2,6,3,7:allhosts"
	3339	else
	3340	export MPI_DSM_CPULIST="0,1,4,5,2,3,6,7:allhosts"
	3341	fi
[622]	3342	else
	3343	unset MPI_DSM_CPULIST
[377]	3344	fi
[366]	3345	# MPI_IB_RAILS: use both IB rails on ICE2
	3346	export MPI_BUFS_PER_HOST=512
	3347	export MPI_IB_RAILS=2
[369]	3348	# NECESSARY, IF MORE THAN 4096 PEs ARE USED
	3349	export MPI_CONNECTIONS_THRESHOLD=8192
[738]	3350	# echo "*** MPI_DSM_CPULIST=$MPI_DSM_CPULIST"
[562]	3351	export MPI_TYPE_DEPTH=20
[738]	3352	# echo "*** MPI_TYPE_DEPTH=$MPI_TYPE_DEPTH"
[683]	3353	export MPI_GROUP_MAX=64
[738]	3354	# echo "*** MPI_GROUP_MAX=$MPI_GROUP_MAX"
[693]	3355	if [[ $use_openmp = true ]]
	3356	then
	3357	echo " mpiexec -npernode $tasks_per_node ./a.out $ROPTS < runfile_atmos"
	3358	mpiexec -npernode $tasks_per_node $tv_opt ./a.out $ROPTS < runfile_atmos
	3359	else
	3360	mpiexec_mpt -np $ii $tv_opt ./a.out $ROPTS < runfile_atmos
	3361	fi
[388]	3362
	3363	# next is test for openmp usage
[622]	3364	# echo "mpiexec -npernode $tasks_per_node ./a.out $ROPTS < runfile_atmos"
	3365	# mpiexec -npernode $tasks_per_node ./a.out $ROPTS < runfile_atmos
[221]	3366	elif [[ $( echo $mpilib \| cut -c1-3 ) = mva ]]
	3367	then
[374]	3368	export MV2_NUM_PORTS=2
[376]	3369	# The default setting of MV2_CPU_MAPPING gives best results
[405]	3370	# export MV2_ENABLE_AFFINITY=1
[376]	3371	# export MV2_CPU_MAPPING=0,1,4,5,2,3,6,7
[412]	3372	# if [[ "$sgi_feature" = ice2 ]]
	3373	# then
	3374	# export MV2_CPU_MAPPING=0,4,1,5,2,6,3,7
	3375	# else
	3376	# export MV2_CPU_MAPPING=0,1,4,5,2,3,6,7
	3377	# fi
[678]	3378	if [[ $use_openmp = true ]]
	3379	then
	3380	unset MV2_CPU_MAPPING
	3381	export MV2_ENABLE_AFFINITY=0
	3382	fi
[412]	3383	echo "*** MV2_CPU_MAPPING=$MV2_CPU_MAPPING"
[678]	3384	echo "*** MV2_ENABLE_AFFINITY=$MV2_ENABLE_AFFINITY"
[622]	3385	if [[ $use_openmp = true ]]
[405]	3386	then
[678]	3387	echo " mpiexec -npernode $tasks_per_node ./a.out $ROPTS < runfile_atmos"
[693]	3388	mpiexec -npernode $tasks_per_node $tv_opt ./a.out $ROPTS < runfile_atmos
[405]	3389	else
[693]	3390	mpiexec -np $ii $tv_opt ./a.out $ROPTS < runfile_atmos
[405]	3391	fi
[738]	3392	elif [[ "$mpilib" = impi ]]
	3393	then
	3394	echo "mpirun -np $tasks_per_node inspxe-cl -r result -collect mi3 -- ./a.out < runfile_atmos"
	3395	mpirun -np $tasks_per_node inspxe-cl -r result -collect mi3 -- ./a.out < runfile_atmos
[221]	3396	fi
[437]	3397	elif [[ $host = lcxt4 \|\| $host = lcxt5m ]]
[179]	3398	then
[544]	3399	aprun -n $ii -N $tasks_per_node a.out $ROPTS < runfile_atmos
[164]	3400	else
[223]	3401	mpiexec -machinefile hostfile -n $ii a.out < runfile_atmos $ROPTS
[164]	3402	fi
[1]	3403	else
[164]	3404
[206]	3405	# currently there is no full MPI-2 support on ICE and XT4
	3406	(( iia = $numprocs_atmos / $threads_per_task ))
	3407	(( iio = $numprocs_ocean / $threads_per_task ))
	3408	printf "\n coupled run ($iia atmosphere, $iio ocean)"
	3409	printf "\n using $coupled_mode coupling"
[102]	3410	printf "\n\n"
[164]	3411
[206]	3412	if [[ $coupled_mode = "mpi2" ]]
[164]	3413	then
[206]	3414	echo "atmosphere_to_ocean $iia $iio" > runfile_atmos
	3415	echo "ocean_to_atmosphere $iia $iio" > runfile_ocean
	3416	if [[ $host = lcsgih \|\| $host = lcsgib ]]
	3417	then
[197]	3418
[221]	3419	if [[ $( echo $mpilib \| cut -c1-3 ) = mpt ]]
	3420	then
	3421	# export MPI_LAUNCH_TIMEOUT=360
	3422	mpiexec_mpt -np $iia ./a.out $ROPTS < runfile_atmos &
	3423	mpiexec_mpt -np $iio ./a.out $ROPTS < runfile_ocean &
	3424	elif [[ $( echo $mpilib \| cut -c1-3 ) = mva ]]
	3425	then
	3426	# ulimit -s 300000 # A too large stack size causes problems
	3427	# export MV2_NUM_PORTS=2
	3428	# export MV2_CPU_MAPPING=0:1:2:3
	3429	mpiexec -n $iia ./a.out $ROPTS < runfile_atmos &
	3430	mpiexec -n $iio ./a.out $ROPTS < runfile_ocean &
	3431	fi
[197]	3432
[437]	3433	elif [[ $host = lcxt4 \|\| $host = lcxt5m ]]
[206]	3434	then
[197]	3435
[206]	3436	aprun -n $iia -N $tasks_per_node a.out < runfile_atmos $ROPTS &
	3437	aprun -n $iio -N $tasks_per_node a.out < runfile_ocean $ROPTS &
	3438
	3439	else
	3440	# WORKAROUND BECAUSE mpiexec WITH -env option IS NOT AVAILABLE ON SOME SYSTEMS
	3441	mpiexec -machinefile hostfile -n $iia a.out $ROPTS < runfile_atmos &
	3442	mpiexec -machinefile hostfile -n $iio a.out $ROPTS < runfile_ocean &
	3443	# mpiexec -machinefile hostfile -n $iia -env coupling_mode atmosphere_to_ocean a.out $ROPTS &
	3444	# mpiexec -machinefile hostfile -n $iio -env coupling_mode ocean_to_atmosphere a.out $ROPTS &
	3445	fi
	3446	wait
	3447
[164]	3448	else
	3449
[206]	3450	echo "coupled_run $iia $iio" > runfile_atmos
	3451	if [[ $host = lcsgih \|\| $host = lcsgib ]]
	3452	then
	3453
[221]	3454	if [[ $( echo $mpilib \| cut -c1-3 ) = mpt ]]
	3455	then
	3456	# export MPI_LAUNCH_TIMEOUT=360
	3457	mpiexec_mpt -np $ii ./a.out $ROPTS < runfile_atmos
	3458	elif [[ $( echo $mpilib \| cut -c1-3 ) = mva ]]
	3459	then
	3460	# ulimit -s 300000 # A too large stack size causes problems
	3461	# export MV2_NUM_PORTS=2
	3462	# export MV2_CPU_MAPPING=0:1:2:3
	3463	mpiexec ./a.out $ROPTS < runfile_atmos
	3464	fi
[206]	3465
[437]	3466	elif [[ $host = lcxt4 \|\| $host = lcxt5m ]]
[206]	3467	then
	3468
	3469	aprun -n $ii -N $tasks_per_node a.out < runfile_atmos $ROPTS
	3470
[718]	3471	elif [[ $host = lck \|\| $host = lckordi ]]
[367]	3472	then
	3473	mpiexec -n $ii ./a.out $ROPTS < runfile_atmos &
[206]	3474	fi
	3475	wait
[164]	3476	fi
[102]	3477
[206]	3478	fi
[1]	3479	elif [[ $host = decalpha ]]
	3480	then
[102]	3481	dmpirun -np $numprocs a.out $ROPTS
[440]	3482	elif [[ $host = lckyoto ]]
	3483	then
	3484	set -xv
	3485	export P4_RSHCOMMAND=plesh
	3486	echo " P4_RSHCOMMAND = $P4_RSHCOMMAND"
	3487	if [[ "$ENVIRONMENT" = BATCH ]]
	3488	then
[505]	3489	if [[ "$cond2" = fujitsu ]]
	3490	then
	3491	mpiexec -n $numprocs ./a.out $ROPTS # for fujitsu-compiler
	3492	elif [[ "cond2" = pgi ]]
	3493	then
	3494	mpirun -np $numprocs -machinefile ${QSUB_NODEINF} ./a.out $ROPTS
	3495	else
	3496	mpirun_rsh -np $numprocs -hostfile ${QSUB_NODEINF} MV2_USE_SRQ=0 ./a.out ${ROPTS} \|\| /bin/true
	3497	fi
[440]	3498	else
[505]	3499	if [[ "$cond2" = "" ]]
	3500	then
	3501	mpiruni_rsh -np $numprocs ./a.out $ROPTS # for intel
	3502	else
	3503	mpirun -np $numprocs ./a.out $ROPTS
	3504	fi
[440]	3505	fi
	3506	set +xv
[1]	3507	elif [[ $host = lctit ]]
	3508	then
[678]	3509	export OMP_NUM_THREADS=$threads_per_task
	3510	echo "OMP_NUM_THREADS=$OMP_NUM_THREADS"
	3511	if [[ "$threads_per_task" != 1 ]]
	3512	then
	3513	export MV2_ENABLE_AFFINITY=0
	3514	fi
	3515	echo "----- PBS_NODEFILE content:"
	3516	cat $PBS_NODEFILE
	3517	echo "-----"
	3518	(( ii = $numprocs / $threads_per_task ))
	3519	echo "mpirun -np $ii -hostfile $PBS_NODEFILE ./a.out $ROPTS"
	3520	mpirun -np $ii -hostfile $PBS_NODEFILE ./a.out $ROPTS
[1]	3521	else
[102]	3522	mpprun -n $numprocs a.out $ROPTS
[1]	3523	fi
	3524	[[ $? != 0 ]] && execution_error=true
	3525
	3526
	3527	# PERFORMANCE-AUSWERTUNG MIT APPRENTICE
	3528	if [[ "$cond1" = apprentice \|\| "$cond2" = apprentice ]]
	3529	then
	3530	apprentice
	3531	fi
	3532	else
[102]	3533	a.out $ROPTS
[1]	3534	fi
	3535	fi
	3536	if [[ $? != 0 \|\| $execution_error = true ]]
	3537	then
	3538
	3539	# ABBRUCH BEI LAUFZEITFEHLER
[102]	3540	# [[ ! ( "$cond1" = debug \|\| "$cond2" = debug ) ]] && cat aout_output*
[1]	3541	printf "\n +++ runtime error occured"
	3542	locat=execution
	3543	exit
	3544	else
[102]	3545	# [[ ! ( "$cond1" = debug \|\| "$cond2" = debug ) ]] && cat aout_output*
[1]	3546	printf "\n$striche\n *** execution finished \n"
[260]	3547
[261]	3548	# Stop the dvr streaming server and process the dvr output in order
	3549	# to create dvrs- and html-files containing all streams
[260]	3550	if [[ "$dvrserver_id" != "" ]]
	3551	then
	3552	kill $dvrserver_id
	3553	printf "\n *** dvr server with id=$dvrserver_id has been stopped"
[261]	3554
[262]	3555	# If there is a directory, data have been output by the
	3556	# streaming server. Otherwise, user has chosen dvrp_output=local
	3557	if [[ -d DATA_DVR ]]
	3558	then
[261]	3559
[262]	3560	# Add the current dvr configuration file to the dvr output
	3561	# directory
	3562	cp .dvrserver.config DATA_DVR
	3563
	3564	# Process the dvr output (option -s for also generating
	3565	# sequence mode data)
	3566	process_dvr_output -d DATA_DVR -f $fname -s
	3567
	3568	else
	3569
	3570	# Process the local output
	3571	process_dvr_output -l -d DATA_DVR -f $fname
	3572
	3573	fi
	3574
	3575	elif [[ $(echo $package_list \| grep -c dvrp_graphics) != 0 ]]
	3576	then
	3577
	3578	# Process dvr output generated in local mode (dvrp_output=local)
	3579	process_dvr_output -l -d DATA_DVR -f $fname
	3580
[260]	3581	fi
[1]	3582	fi
	3583
	3584
	3585
[503]	3586	# Call of combine_plot_fields in order to merge single files written
	3587	# by each PE into one file.
	3588	if [[ ! -f ${PALM_BIN}/combine_plot_fields${block}.x ]]
	3589	then
	3590	printf "\n\n\n +++ WARNING: no combine_plot_fields found for given block \"$cond1 $cond2\""
	3591	printf "\n 2d- and/or 3d-data may be incomplete!"
	3592	printf "\n Run \"mbuild -u -h $localhost\" to generate utilities for this block.\n"
	3593	else
	3594	printf "\n\n\n *** post-processing: now executing \"combine_plot_fields${block}.x\" ..."
	3595	combine_plot_fields${block}.x
	3596	fi
	3597
	3598
	3599
[1]	3600	# EVENTUELLE OUTPUT-KOMMANDOS ABARBEITEN
	3601	(( i = 0 ))
	3602	while (( i < ioc ))
	3603	do
	3604	(( i = i + 1 ))
	3605	if (( i == 1 ))
	3606	then
	3607	printf "\n\n *** execution of OUTPUT-commands:\n$striche"
	3608	fi
	3609	printf "\n >>> ${out_command[$i]}"
	3610	eval ${out_command[$i]}
	3611	if (( i == ioc ))
	3612	then
	3613	printf "\n$striche\n"
	3614	fi
	3615	done
	3616
	3617
	3618	# EVTL. INHALT DES AKTUELLEN VERZEICHNISSES AUSGEBEN
	3619	if [[ $do_trace = true ]]
	3620	then
	3621	printf "\n\n"
	3622	ls -al
	3623	fi
	3624
	3625
	3626
	3627	# OUTPUT-DATEIEN AN IHRE ZIELORTE KOPIEREN
	3628	(( i = 0 ))
	3629	while (( i < iout ))
	3630	do
	3631	(( i = i + 1 ))
	3632	if (( i == 1 ))
	3633	then
	3634	printf "\n\n *** saving OUTPUT-files:\n$striche"
	3635	fi
	3636
	3637	# PRUEFEN, OB EINZELDATEI ODER DATEI PRO PROZESSOR
	3638	files_for_pes=false; filetyp=file
	3639	if [[ "${actionout[$i]}" = pe && -n $numprocs ]]
	3640	then
	3641	files_for_pes=true; filetyp=directory
	3642	actionout[$i]=""
	3643	elif [[ "${actionout[$i]}" = pe && ! -n $numprocs ]]
	3644	then
	3645	actionout[$i]=""
	3646	elif [[ "${actionout[$i]}" = arpe && -n $numprocs ]]
	3647	then
	3648	files_for_pes=true; filetyp=directory
	3649	actionout[$i]="ar"
	3650	elif [[ "${actionout[$i]}" = arpe && ! -n $numprocs ]]
	3651	then
	3652	actionout[$i]="ar"
	3653	elif [[ "${actionout[$i]}" = flpe && -n $numprocs ]]
	3654	then
	3655	files_for_pes=true; filetyp=directory
	3656	actionout[$i]="fl"
	3657	elif [[ "${actionout[$i]}" = flpe && ! -n $numprocs ]]
	3658	then
	3659	actionout[$i]="fl"
	3660	elif [[ "${actionout[$i]}" = trpe && -n $numprocs ]]
	3661	then
	3662	files_for_pes=true; filetyp=directory
	3663	actionout[$i]="tr"
	3664	elif [[ "${actionout[$i]}" = trpe && ! -n $numprocs ]]
	3665	then
	3666	actionout[$i]="tr"
	3667	fi
	3668
	3669	if [[ ! -f ${localout[$i]} && $files_for_pes = false ]]
	3670	then
	3671	printf "\n +++ temporary OUTPUT-file ${localout[$i]} does not exist\n"
[260]	3672	elif [[ ! -d ${localout[$i]} && $files_for_pes = true ]]
[1]	3673	then
	3674	printf "\n +++ temporary OUTPUT-file ${localout[$i]}/.... does not exist\n"
	3675	else
	3676
	3677
	3678	# KOPIEREN PER FTP/SCP (IMMER IM BINAERMODUS, -M: FALLS ZIELKATALOG
	3679	# NICHT VORHANDEN, WIRD VERSUCHT, IHN ANZULEGEN), ABER NUR BEI
	3680	# REMOTE-RECHNUNGEN
	3681	if [[ "${actionout[$i]}" = tr ]]
	3682	then
	3683	if [[ $localhost != $fromhost ]]
	3684	then
	3685	if [[ $files_for_pes = false ]]
	3686	then
	3687	cps=""
	3688	cst=""
	3689	else
	3690	cps=-c
	3691	cst="/"
	3692	fi
[622]	3693	if [[ $localhost = nech ]]
[1]	3694	then
	3695
	3696	# TRANSFER IN EIGENSTAENDIGEM JOB
	3697	# ZUERST KOPIE DER DATEI INS TEMPORAERE DATENVERZEICHNIS
	3698	[[ ! -d $tmp_data_catalog/TRANSFER ]] && mkdir -p $tmp_data_catalog/TRANSFER
	3699	file_to_transfer=${fname}_${localout[$i]}_to_transfer_$kennung
	3700	if [[ $files_for_pes = false ]]
	3701	then
	3702	ln -f ${localout[$i]} $tmp_data_catalog/TRANSFER/$file_to_transfer
	3703	else
	3704	mkdir $tmp_data_catalog/TRANSFER/$file_to_transfer
	3705	ln ${localout[$i]}/* $tmp_data_catalog/TRANSFER/$file_to_transfer
	3706	fi
	3707
	3708	echo "set -x" > transfer_${localout[$i]}
	3709	echo "cd $tmp_data_catalog/TRANSFER" >> transfer_${localout[$i]}
[82]	3710
	3711	printf "\n >>> OUTPUT: ${localout[$i]}$cst by SCP in seperate job to"
	3712	printf "\n ${pathout[$i]}/${localhost}_${fname}${endout[$i]}$cst"
	3713	printf "\n or higher cycle\n"
	3714	echo "batch_scp $cps -b -m -u $return_username $return_addres $file_to_transfer \"${pathout[$i]}\" ${localhost}_${fname}${endout[$i]} ${extout[$i]}" >> transfer_${localout[$i]}
	3715
[1]	3716	echo "[[ \$? = 0 ]] && rm $file_to_transfer" >> transfer_${localout[$i]}
	3717
	3718	if [[ $localhost = nech ]]
	3719	then
[82]	3720	subjob -d -c /pf/b/$usern/job_queue -v -q pp -X 0 -m 1000 -t 900 transfer_${localout[$i]}
[1]	3721	else
	3722	if [[ "$LOGNAME" = b323013 ]]
	3723	then
[164]	3724	subjob -v -q c1 -X 0 -m 1000 -t 900 -c $job_catalog transfer_${localout[$i]}
[1]	3725	else
[164]	3726	subjob -d -v -q c1 -X 0 -m 1000 -t 900 -c $job_catalog transfer_${localout[$i]}
[1]	3727	fi
	3728	fi
	3729
	3730	else
	3731
	3732	# TRANSFER INNERHALB DIESES JOBS
	3733	transfer_failed=false
[82]	3734	printf "\n >>> OUTPUT: ${localout[$i]}$cst by SCP to"
	3735	printf "\n ${pathout[$i]}/${localhost}_${fname}${endout[$i]}$cst\n"
	3736	batch_scp $cps -b -m -u $return_username $return_addres ${localout[$i]} "${pathout[$i]}" ${localhost}_${fname}${endout[$i]} ${extout[$i]}
	3737	[[ $? != 0 ]] && transfer_failed=true
[1]	3738
	3739	# BEI FEHLGESCHLAGENEM TRANSFER SICHERUNGSKOPIE AUF
	3740	# LOKALER MASCHINE ANLEGEN
	3741	if [[ $transfer_failed = true ]]
	3742	then
	3743	printf " +++ transfer failed. Trying to save a copy on the local host under:\n"
	3744	printf " ${pathout[$i]}/${localhost}_${fname}${endout[$i]}_$kennung\n"
	3745
	3746	# ERSTMAL PRUEFEN, OB VERZEICHNIS EXISTIERT. GEGEBENENFALLS
	3747	# ANLEGEN.
	3748	eval local_catalog=${pathout[$i]}
	3749	if [[ ! -d $local_catalog ]]
	3750	then
	3751	printf " *** local directory does not exist. Trying to create:\n"
	3752	printf " $local_catalog \n"
	3753	mkdir -p $local_catalog
	3754	fi
	3755	eval cp ${localout[$i]} ${pathout[$i]}/${localhost}_${fname}${endout[$i]}_$kennung
	3756	transfer_problems=true
	3757	fi
	3758	fi
	3759	else
	3760
	3761	# WERTZUWEISUNG, SO DASS WEITER UNTEN NUR KOPIERT WIRD
	3762	actionout[$i]=""
	3763	fi
	3764	fi
	3765
	3766
	3767	# APPEND PER FTP/SCP (IMMER IM BINAERMODUS, -M: FALLS ZIELKATALOG
	3768	# NICHT VORHANDEN, WIRD VERSUCHT, IHN ANZULEGEN), ABER NUR BEI
	3769	# REMOTE-RECHNUNGEN
	3770	if [[ "${actionout[$i]}" = tra ]]
	3771	then
	3772	if [[ $localhost != $fromhost ]]
	3773	then
[622]	3774	if [[ $localhost = ibmh \|\| $localhost = nech ]]
[1]	3775	then
	3776
	3777	# TRANSFER IN EIGENSTAENDIGEM JOB
	3778	# ZUERST KOPIE DER DATEI INS TEMPORAERE DATENVERZEICHNIS
	3779	[[ ! -d $tmp_data_catalog/TRANSFER ]] && mkdir -p $tmp_data_catalog/TRANSFER
	3780	file_to_transfer=${fname}_${localout[$i]}_to_transfer_$kennung
	3781	ln -f ${localout[$i]} $tmp_data_catalog/TRANSFER/$file_to_transfer
	3782
	3783	echo "set -x" > transfer_${localout[$i]}
	3784	echo "cd $tmp_data_catalog/TRANSFER" >> transfer_${localout[$i]}
[82]	3785
	3786	printf "\n >>> OUTPUT: ${localout[$i]} append by SCP in seperate job to"
	3787	printf "\n ${pathout[$i]}/${localhost}_${fname}${endout[$i]}"
	3788	printf "\n or higher cycle\n"
	3789	echo "batch_scp -A -b -m -u $return_username $return_addres $file_to_transfer \"${pathout[$i]}\" ${localhost}_${fname}${endout[$i]} ${extout[$i]}" >> transfer_${localout[$i]}
	3790
[1]	3791	echo "[[ \$? = 0 ]] && rm $file_to_transfer" >> transfer_${localout[$i]}
	3792
	3793	if [[ $localhost = nech ]]
	3794	then
[82]	3795	subjob -d -c /pf/b/$usern/job_queue -v -q pp -X 0 -m 1000 -t 900 transfer_${localout[$i]}
[1]	3796	else
	3797	if [[ $LOGNAME = b323013 ]]
	3798	then
[164]	3799	subjob -v -q c1 -X 0 -m 1000 -t 900 -c $job_catalog transfer_${localout[$i]}
[1]	3800	else
[164]	3801	subjob -d -v -q c1 -X 0 -m 1000 -t 900 -c $job_catalog transfer_${localout[$i]}
[1]	3802	fi
	3803	fi
	3804
	3805	else
	3806
	3807	# TRANSFER INNERHALB DIESES JOBS
	3808	transfer_failed=false
[82]	3809	printf "\n >>> OUTPUT: ${localout[$i]} append by SCP to"
	3810	printf "\n ${pathout[$i]}/${localhost}_${fname}${endout[$i]}\n"
	3811	batch_scp -A -b -m -u $return_username $return_addres ${localout[$i]} "${pathout[$i]}" ${localhost}_${fname}${endout[$i]} ${extout[$i]}
	3812	[[ $? != 0 ]] && transfer_failed=true
[1]	3813
	3814	# BEI FEHLGESCHLAGENEM TRANSFER SICHERUNGSKOPIE AUF
	3815	# LOKALER MASCHINE ANLEGEN
	3816	if [[ $transfer_failed = true ]]
	3817	then
	3818	printf " +++ transfer failed. Trying to save a copy on the local host under:\n"
	3819	printf " ${pathout[$i]}/${localhost}_${fname}${endout[$i]}_$kennung\n"
	3820
	3821	# ERSTMAL PRUEFEN, OB VERZEICHNIS EXISTIERT. GEGEBENENFALLS
	3822	# ANLEGEN
	3823	eval local_catalog=${pathout[$i]}
	3824	if [[ ! -d $local_catalog ]]
	3825	then
	3826	printf " *** local directory does not exist. Trying to create:\n"
	3827	printf " $local_catalog \n"
	3828	mkdir -p $local_catalog
	3829	fi
	3830
	3831	eval cp ${localout[$i]} ${pathout[$i]}/${localhost}_${fname}${endout[$i]}_$kennung
	3832	transfer_problems=true
	3833	fi
	3834	fi
	3835	else
	3836
	3837	# WERTZUWEISUNG, SO DASS WEITER UNTEN NUR APPEND AUF
	3838	# LOKALER MASCHINE DURCHGEFUEHRT WIRD
	3839	actionout[$i]=a
	3840	fi
	3841	fi
	3842
	3843
	3844	# OUTPUT-DATEI FUER EINEN FORTSETZUNGSLAUF. DATEI WIRD PER
	3845	# LINK AUF DEN TEMPORAEREN DATENKATALOG GELEGT. OPTION -f WIRD
	3846	# VERWENDET, FALLS DORT NOCH EINE DATEI GLEICHEN NAMENS VORHANDEN
	3847	# SEIN SOLLTE. ANSCHLIESSEND WIRD SEINE ARCHIVIERUNG ANGESTOSSEN
	3848	if [[ "${actionout[$i]}" = fl ]]
	3849	then
	3850	[[ ! -d $tmp_data_catalog ]] && mkdir -p $tmp_data_catalog
	3851	chmod g+rx $tmp_data_catalog
	3852	if [[ $files_for_pes = false ]]
	3853	then
	3854	printf "\n >>> OUTPUT: ${localout[$i]} to"
	3855	printf "\n $tmp_data_catalog/${frelout[$i]} (temporary data catalog)\n"
	3856	ln -f ${localout[$i]} $tmp_data_catalog/${frelout[$i]}
	3857	else
	3858	printf "\n >>> OUTPUT: ${localout[$i]}/.... to"
	3859	printf "\n $tmp_data_catalog/${frelout[$i]} (temporary data catalog)\n"
	3860	mkdir $tmp_data_catalog/${frelout[$i]}
[149]	3861	cd ${localout[$i]}
	3862	for file in $(ls *)
	3863	do
	3864	ln -f $file $tmp_data_catalog/${frelout[$i]}
	3865	done
	3866	cd $TEMPDIR
[1]	3867	fi
	3868
	3869
	3870	# ARCHIVIERUNGSJOB WIRD ERZEUGT UND ABGESCHICKT
	3871	if [[ $store_on_archive_system = true ]]
	3872	then
	3873
	3874	if [[ $archive_system = asterix ]]
	3875	then
	3876	echo "cd $tmp_data_catalog" >> archive_${frelout[$i]}
	3877	if [[ $files_for_pes = false ]]
	3878	then
	3879	echo "stageout ${frelout[$i]} > STAGE_OUTPUT${i}_$kennung" >> archive_${frelout[$i]}
	3880	else
	3881	echo "stageout -t ${frelout[$i]} > STAGE_OUTPUT${i}_$kennung" >> archive_${frelout[$i]}
	3882	fi
	3883	echo "cat STAGE_OUTPUT${i}_$kennung" >> archive_${frelout[$i]}
	3884	echo "if [[ \$(grep -c \"st.msg:150\" STAGE_OUTPUT${i}_$kennung) != 0 ]]" >> archive_${frelout[$i]}
	3885	echo "then" >> archive_${frelout[$i]}
	3886	echo " do_stageout=false" >> archive_${frelout[$i]}
	3887	echo "else" >> archive_${frelout[$i]}
	3888	echo " echo \" +++ $filetyp ${frelout[$i]} could not be stored on archive-system\" " >> archive_${frelout[$i]}
	3889	echo " cat /dev/null > ~/job_queue/ARCHIVE_ERROR_$fname" >> archive_${frelout[$i]}
	3890	echo " cat STAGE_OUTPUT${i}_$kennung > ~/job_queue/archive_${frelout[$i]}_error" >> archive_${frelout[$i]}
	3891	echo " echo \" *** $filetyp ${frelout[$i]} will be copied to \$WORK as backup\" " >> archive_${frelout[$i]}
	3892	if [[ $files_for_pes = false ]]
	3893	then
	3894	echo " cp ${frelout[$i]} \$WORK" >> archive_${frelout[$i]}
	3895	else
	3896	echo " cp -r ${frelout[$i]} \$WORK/${frelout[$i]}" >> archive_${frelout[$i]}
	3897	fi
	3898	echo " echo \" *** $filetyp ${frelout[$i]} saved\" " >> archive_${frelout[$i]}
	3899	echo "fi" >> archive_${frelout[$i]}
	3900	echo "rm STAGE_OUTPUT${i}_$kennung" >> archive_${frelout[$i]}
	3901	elif [[ $archive_system = DMF ]]
	3902	then
	3903	echo "cd $tmp_data_catalog" >> archive_${frelout[$i]}
	3904	if [[ $files_for_pes = false ]]
	3905	then
	3906	printf "\n +++ archiving of single files impossible with $archive_system !\n"
	3907	locat=DMF
	3908	exit
	3909	else
	3910	# FUER RECHNER IN JUELICH. DORT KOENNTEN AUCH
	3911	# EINZELNE DATEIEN GESICHERT WERDEN (SPAETER KORR.)
	3912	echo "rm -rf \$ARCHIVE/${frelout[$i]}" >> archive_${frelout[$i]}
	3913	echo "cp -r ${frelout[$i]} \$ARCHIVE" >> archive_${frelout[$i]}
	3914	fi
	3915	elif [[ $archive_system = tivoli ]]
	3916	then
	3917	echo "cd $tmp_data_catalog" >> archive_${frelout[$i]}
	3918	if [[ $files_for_pes = false ]]
	3919	then
	3920	# EVENTUELL NOCH VORHANDENE DATEI IM ARCHIV LOSCHEN
	3921	echo "set -x" >> archive_${frelout[$i]}
	3922	echo "rm -rf \$PERM/${frelout[$i]}" >> archive_${frelout[$i]}
	3923	echo "cp ${frelout[$i]} \$PERM/${frelout[$i]}" >> archive_${frelout[$i]}
	3924	else
	3925
	3926	echo "set -x" >> archive_${frelout[$i]}
	3927	echo "rm -rf \$PERM/${frelout[$i]}/*" >> archive_${frelout[$i]}
	3928	echo "[[ ! -d \$PERM/${frelout[$i]} ]] && mkdir $PERM/${frelout[$i]}" >> archive_${frelout[$i]}
	3929	cd $tmp_data_catalog
	3930	all_files=`ls -1 ${frelout[$i]}/*`
	3931	cd - > /dev/null
	3932	(( inode = 0 ))
	3933	(( tp1 = tasks_per_node + 1 ))
	3934	while (( inode < nodes ))
	3935	do
	3936	# echo "*** all_files = \"$all_files\" "
	3937	files=`echo $all_files \| cut -d" " -f1-$tasks_per_node`
	3938	all_files=`echo $all_files \| cut -d" " -f$tp1-`
	3939	# echo "*** tasks_per_node = \"$tasks_per_node\" "
	3940	# echo "*** files = \"$files\" "
	3941	# echo "*** all_files = \"$all_files\" "
[416]	3942	echo "tar cvf \$PERM/${frelout[$i]}/${frelout[$i]}.node_$inode.tar $files" >> archive_${frelout[$i]}
	3943	### echo "tar cvf ${frelout[$i]}.node_$inode.tar $files" >> archive_${frelout[$i]}
	3944	### echo "cp ${frelout[$i]}.node_$inode.tar \$PERM/${frelout[$i]}/" >> archive_${frelout[$i]}
	3945	###echo "rm ${frelout[$i]}.node_$inode.tar" >> archive_${frelout[$i]}
[397]	3946	# echo "dsmc incremental \$PERM/${frelout[$i]}/${frelout[$i]}.node_$inode.tar" >> archive_${frelout[$i]}
	3947	# echo "dsmmigrate \$PERM/${frelout[$i]}/${frelout[$i]}.node_$inode.tar" >> archive_${frelout[$i]}
[1]	3948	(( inode = inode + 1 ))
	3949	done
	3950	# echo "rm -rf \$PERM/${frelout[$i]}.tar" >> archive_${frelout[$i]}
	3951	# echo "tar cvf \$PERM/${frelout[$i]}.tar ${frelout[$i]}" >> archive_${frelout[$i]}
	3952	fi
	3953	elif [[ $archive_system = ut ]]
	3954	then
	3955	echo "cd $tmp_data_catalog" >> archive_${frelout[$i]}
	3956	if [[ $files_for_pes = false ]]
	3957	then
	3958	# EVENTUELL NOCH VORHANDENE DATEI IM ARCHIV LOSCHEN
	3959	echo "set -x" >> archive_${frelout[$i]}
	3960	echo "rm -rf \$UT/${frelout[$i]}" >> archive_${frelout[$i]}
	3961	echo "cp ${frelout[$i]} \$UT/${frelout[$i]}" >> archive_${frelout[$i]}
	3962	else
	3963
	3964	echo "set -x" >> archive_${frelout[$i]}
	3965	echo "rm -rf \$UT/${frelout[$i]}/*" >> archive_${frelout[$i]}
	3966	echo "[[ ! -d \$UT/${frelout[$i]} ]] && mkdir $UT/${frelout[$i]}" >> archive_${frelout[$i]}
	3967	cd $tmp_data_catalog
	3968	all_files=`ls -1 ${frelout[$i]}/*`
	3969	cd - > /dev/null
	3970	(( inode = 0 ))
	3971	(( tp1 = tasks_per_node + 1 ))
	3972	while (( inode < nodes ))
	3973	do
	3974	files=`echo $all_files \| cut -d" " -f1-$tasks_per_node`
	3975	all_files=`echo $all_files \| cut -d" " -f$tp1-`
	3976	echo "tar cvf \$UT/${frelout[$i]}/${frelout[$i]}.node_$inode.tar $files" >> archive_${frelout[$i]}
	3977	(( inode = inode + 1 ))
	3978	done
	3979	fi
	3980	elif [[ $archive_system = none ]]
	3981	then
	3982	printf " +++ archiving on $localhost not available!\n"
	3983	fi
	3984
	3985	if [[ $archive_system != none ]]
	3986	then
[399]	3987	if [[ $localhost = lcsgih \|\| $localhost = lcsgib ]]
[1]	3988	then
[164]	3989	# subjob -d -v -q cdata -X 0 -m 1000 -t 43200 -c $job_catalog archive_${frelout[$i]}
[416]	3990	subjob -v -q permq -X 1 -T 1 -m 1000 -t 172800 -c $job_catalog archive_${frelout[$i]}
[1]	3991	elif [[ $localhost = nech ]]
	3992	then
[82]	3993	subjob -d -c /pf/b/$usern/job_queue -v -q pp -X 0 -m 1000 -t 7200 archive_${frelout[$i]}
[1]	3994	fi
	3995	printf " Archiving of $tmp_data_catalog/${frelout[$i]} initiated (batch job submitted)\n"
	3996	fi
	3997	else
[399]	3998	printf " +++ caution: option -A is switched off. No archiving on $archive_system!\n"
[1]	3999	fi
	4000
	4001
	4002	# LEERDATEI IM BENUTZERVERZEICHNIS ANLEGEN, DAMIT BEKANNT IST,
	4003	# WIE DIE HOECHSTE ZYKLUSNUMMER AUF DEM ARCHIV-SYSTEM LAUTET
	4004	if [[ $files_for_pes = false ]]
	4005	then
	4006	cat /dev/null > ${pathout[$i]}
	4007	else
	4008	mkdir -p ${pathout[$i]}
	4009	fi
	4010
	4011	fi
	4012
	4013
	4014	# KOPIEREN AUF LOKALER MASCHINE ZUM ARCHIV-SYSTEM
	4015	# AUSSERDEM MUSS LEERDATEI ANGELEGT WERDEN, DAMIT BEKANNT IST,
	4016	# WIE DIE HOECHSTE ZYKLUSNUMMER AUF DEM ARCHIV-SYSTEM LAUTET
	4017	# FALLS IRGENDEINE (VON MEHREREN) ARCHIVIERUNGEN SCHIEF GEHT,
	4018	# WIRD FLAG GESETZT UND BLEIBT AUCH BEI WEITEREN ORDNUNGS-
	4019	# GEMAESSEN ARCHIVIERUNGEN GESETZT
	4020	if [[ "${actionout[$i]}" = ar ]]
	4021	then
	4022	if [[ $files_for_pes = false ]]
	4023	then
	4024	printf "\n >>> OUTPUT: ${localout[$i]} to"
	4025	printf "\n ${pathout[$i]}"
	4026	printf "\n File will be copied to archive-system ($archive_system) !\n"
	4027	else
	4028	printf "\n >>> OUTPUT: ${localout[$i]}/_.... to"
	4029	printf "\n ${pathout[$i]}"
	4030	printf "\n Directory will be copied to archive-system ($archive_system) !\n"
	4031	fi
	4032	mv ${localout[$i]} ${frelout[$i]}
	4033
	4034	file_saved=false
	4035
	4036	if [[ $archive_system = asterix ]]
	4037	then
	4038	do_stageout=true
	4039	(( stageout_anz = 0 ))
	4040	while [[ $do_stageout = true ]]
	4041	do
	4042	if [[ $files_for_pes = false ]]
	4043	then
	4044	stageout ${frelout[$i]} > STAGE_OUTPUT
	4045	else
	4046	stageout -t ${frelout[$i]} > STAGE_OUTPUT
	4047	fi
	4048	cat STAGE_OUTPUT
	4049	if [[ $(grep -c "st.msg:150" STAGE_OUTPUT) != 0 ]]
	4050	then
	4051	file_saved=true
	4052	do_stageout=false
	4053	else
	4054	if [[ $files_for_pes = false ]]
	4055	then
	4056	printf "\n +++ file ${frelout[$i]} could not be saved on archive-system"
	4057	else
	4058	printf "\n +++ directory ${frelout[$i]} could not be saved on archive-system"
	4059	fi
	4060	(( stageout_anz = stageout_anz + 1 ))
	4061	if (( stageout_anz == 10 ))
	4062	then
	4063	printf "\n +++ stoped after 10 unsuccessful tries!"
	4064	archive_save=false
	4065	do_stageout=false
	4066	else
	4067	printf "\n *** new try to store on archive after 15 min:"
	4068	sleep 900
	4069	fi
	4070	fi
	4071	done
	4072	elif [[ $archive_system = DMF ]]
	4073	then
	4074	if [[ $files_for_pes = false ]]
	4075	then
	4076	printf "\n +++ archiving of single files impossible on $archive_system!\n"
	4077	locat=DMF
	4078	exit
	4079	else
	4080	rm -rf $ARCHIVE/${frelout[$i]}
	4081	cp -r ${frelout[$i]} $ARCHIVE
	4082	fi
	4083	file_saved=true
	4084	elif [[ $archive_system = tivoli ]]
	4085	then
	4086	# ARCHIVIERUNG NUR PER BATCH-JOB MOEGLICH
	4087	# DATEI MUSS ZWISCHENZEITLICH INS TEMPORAERE DATENVERZEICHNIS
	4088	# GELEGT WERDEN
	4089	[[ ! -d $tmp_data_catalog ]] && mkdir -p $tmp_data_catalog
	4090	chmod g+rx $tmp_data_catalog
	4091	if [[ $files_for_pes = false ]]
	4092	then
	4093	ln -f ${frelout[$i]} $tmp_data_catalog/${frelout[$i]}
	4094	else
	4095	mkdir $tmp_data_catalog/${frelout[$i]}
	4096	ln -f ${frelout[$i]}/* $tmp_data_catalog/${frelout[$i]}
	4097	fi
	4098
	4099	# BATCH JOB GENERIEREN UND ABSCHICKEN; DATEI MUSS WIEDER
	4100	# AUS TEMPORAEREM DATENVERZEICHNIS ENTFERNT WERDEN
	4101	echo "cd $tmp_data_catalog" > archive_${frelout[$i]}
	4102	if [[ $files_for_pes = false ]]
	4103	then
	4104	# EVENTUELL NOCH VORHANDENE DATEI IM ARCHIV LOSCHEN
	4105	echo "rm -rf \$PERM/${frelout[$i]}" >> archive_${frelout[$i]}
	4106	echo "cp ${frelout[$i]} \$PERM/${frelout[$i]}" >> archive_${frelout[$i]}
	4107	echo "rm -rf ${frelout[$i]}" >> archive_${frelout[$i]}
	4108	else
	4109	echo "rm -rf \$PERM/${frelout[$i]}.tar" >> archive_${frelout[$i]}
	4110	echo "tar cvf \$PERM/${frelout[$i]}.tar ${frelout[$i]}" >> archive_${frelout[$i]}
	4111	echo "rm -rf ${frelout[$i]}" >> archive_${frelout[$i]}
	4112	fi
	4113
[164]	4114	subjob -v -d -q cdata -X 0 -m 1000 -t 43200 -c $job_catalog archive_${frelout[$i]}
[1]	4115	printf " Archiving of $tmp_data_catalog/${frelout[$i]} initiated (batch job submitted)\n"
	4116	file_saved=true
	4117
	4118	elif [[ $archive_system = ut ]]
	4119	then
	4120	# ARCHIVIERUNG NUR PER BATCH-JOB MOEGLICH
	4121	# DATEI MUSS ZWISCHENZEITLICH INS TEMPORAERE DATENVERZEICHNIS
	4122	# GELEGT WERDEN
	4123	[[ ! -d $tmp_data_catalog ]] && mkdir -p $tmp_data_catalog
	4124	chmod g+rx $tmp_data_catalog
	4125	if [[ $files_for_pes = false ]]
	4126	then
	4127	ln -f ${frelout[$i]} $tmp_data_catalog/${frelout[$i]}
	4128	else
	4129	mkdir $tmp_data_catalog/${frelout[$i]}
	4130	ln -f ${frelout[$i]}/* $tmp_data_catalog/${frelout[$i]}
	4131	fi
	4132
	4133	# BATCH JOB GENERIEREN UND ABSCHICKEN; DATEI MUSS WIEDER
	4134	# AUS TEMPORAEREM DATENVERZEICHNIS ENTFERNT WERDEN
	4135	echo "cd $tmp_data_catalog" > archive_${frelout[$i]}
	4136	if [[ $files_for_pes = false ]]
	4137	then
	4138	# EVENTUELL NOCH VORHANDENE DATEI IM ARCHIV LOSCHEN
	4139	echo "rm -rf \$UT/${frelout[$i]}" >> archive_${frelout[$i]}
	4140	echo "cp ${frelout[$i]} \$UT/${frelout[$i]}" >> archive_${frelout[$i]}
	4141	echo "rm -rf ${frelout[$i]}" >> archive_${frelout[$i]}
	4142	else
	4143	echo "rm -rf \$UT/${frelout[$i]}.tar" >> archive_${frelout[$i]}
	4144	echo "tar cvf \$UT/${frelout[$i]}.tar ${frelout[$i]}" >> archive_${frelout[$i]}
	4145	echo "rm -rf ${frelout[$i]}" >> archive_${frelout[$i]}
	4146	fi
	4147
[82]	4148	subjob -v -c /pf/b/$usern/job_queue -d -q pp -X 0 -m 1000 -t 7200 archive_${frelout[$i]}
[1]	4149	printf " Archiving of $tmp_data_catalog/${frelout[$i]} initiated (batch job submitted)\n"
	4150	file_saved=true
	4151
	4152	else
	4153	printf "\n +++ archive_system=\"$archive_system\" archiving impossible!"
	4154	archive_save=false
	4155	fi
	4156	if [[ $file_saved = true ]]
	4157	then
	4158	if [[ $files_for_pes = false ]]
	4159	then
	4160	cat /dev/null > ${pathout[$i]}
	4161	else
	4162	mkdir -p ${pathout[$i]}
	4163	fi
	4164	fi
	4165	fi
	4166
	4167	# APPEND AUF LOKALER MASCHINE
	4168	if [[ "${actionout[$i]}" = "a" ]]
	4169	then
[741]	4170	if [[ "${extout[$i]}" != " " && "${extout[$i]}" != "" ]]
	4171	then
	4172	printf "\n >>> OUTPUT: ${localout[$i]} append to ${pathout[$i]}.${extout[$i]}\n"
	4173	cat ${localout[$i]} >> ${pathout[$i]}.${extout[$i]}
	4174	else
	4175	printf "\n >>> OUTPUT: ${localout[$i]} append to ${pathout[$i]}\n"
	4176	cat ${localout[$i]} >> ${pathout[$i]}
	4177	fi
[1]	4178	fi
	4179
	4180	# KOPIEREN AUF LOKALER MASCHINE
	4181	# ES MUSS KOPIERT WERDEN, DA MOVE NICHT UEBER FILESYSTEM HINAUS MOEGLICH
	4182	if [[ "${actionout[$i]}" = "" && $files_for_pes = false ]]
	4183	then
	4184
	4185	# KOPIEREN AUF EINPROZESSORMASCHINE
[5]	4186	if [[ "${extout[$i]}" != " " && "${extout[$i]}" != "" ]]
[1]	4187	then
	4188	printf "\n >>> OUTPUT: ${localout[$i]} to ${pathout[$i]}.${extout[$i]}\n"
[149]	4189	if [[ $link_local_output = true ]]
	4190	then
	4191	printf " using ln -f\n"
	4192	ln -f ${localout[$i]} ${pathout[$i]}.${extout[$i]}
	4193	fi
	4194	# If "ln -f" fails of if "$link_local_output = false" do a normal "cp"
	4195	if [[ ! -f "${pathout[$i]}.${extout[$i]}" ]]
	4196	then
	4197	if [[ $link_local_output = true ]]
	4198	then
	4199	printf " ln failed, using cp...\n"
	4200	fi
	4201	cp ${localout[$i]} ${pathout[$i]}.${extout[$i]}
	4202	fi
[1]	4203	else
	4204	printf "\n >>> OUTPUT: ${localout[$i]} to ${pathout[$i]}\n"
[149]	4205	if [[ $link_local_output = true ]]
	4206	then
	4207	printf " using ln -f\n"
	4208	ln -f ${localout[$i]} ${pathout[$i]}
	4209	fi
	4210	# If "ln -f" fails of if "$link_local_output = false" do a normal "cp"
	4211	if [[ ! -f "${pathout[$i]}" ]]
	4212	then
	4213	if [[ $link_local_output = true ]]
	4214	then
	4215	printf " ln failed, using cp...\n"
	4216	fi
	4217	cp ${localout[$i]} ${pathout[$i]}
	4218	fi
[1]	4219	fi
	4220
	4221	elif [[ "${actionout[$i]}" = "" && $files_for_pes = true ]]
	4222	then
	4223
	4224	# DIE DEN PROZESSOREN EINES PARALLELRECHNERS ZUGEHOERIGEN
	4225	# DATEIEN WERDEN ERST IN EINEM KATALOG GESAMMELT UND DIESER
	4226	# WIRD DANN KOPIERT
	4227	# PER MOVE UMBENANNT WERDEN
	4228	printf "\n >>> OUTPUT: ${localout[$i]}/_.... to ${pathout[$i]}\n"
[149]	4229	if [[ $link_local_output = true ]]
	4230	then
	4231	printf " using ln -f\n"
	4232	mkdir ${pathout[$i]}
	4233	cd ${localout[$i]}
	4234	for file in $(ls *)
	4235	do
	4236	ln -f $file ${pathout[$i]}
	4237	done
	4238	cd $TEMPDIR
	4239	fi
	4240	# If "ln -f" fails of if "$link_local_output = false" do a normal "cp -r"
	4241	if [[ ! -f "${pathout[$i]}/_0000" ]]
	4242	then
	4243	if [[ $link_local_output = true ]]
	4244	then
	4245	printf " ln failed for .../_0000, using cp...\n"
	4246	fi
	4247	cp -r ${localout[$i]} ${pathout[$i]}
	4248	fi
[1]	4249
	4250	fi
	4251	fi
	4252	done
	4253	if (( i != 0 ))
	4254	then
	4255	if [[ $transfer_problems = true ]]
	4256	then
	4257	printf "\n$striche\n *** OUTPUT-files saved"
	4258	printf "\n +++ WARNING: some data transfers failed! \n"
	4259	else
	4260	printf "\n$striche\n *** all OUTPUT-files saved \n"
	4261	fi
	4262	fi
	4263
	4264
	4265	# EVENTUELL FOLGEJOB STARTEN
	4266	# DATEI CONTINUE_RUN MUSS VOM BENUTZERPROGRAMM AUS ERZEUGT WERDEN
	4267	if [[ -f CONTINUE_RUN ]]
	4268	then
	4269
	4270	if [[ $archive_save = true ]]
	4271	then
	4272
	4273	# ZUERST IN MRUN-AUFRUF OPTIONEN FUER FORTSETZUNGSLAUF, FUER
	4274	# STILLES ABARBEITEN (OHNE INTERAKTIVE RUECKFAGEN) UND FUER
	4275	# BATCH-BETRIEB (NUR WICHTIG, FALLS BATCH AUF LOKALER MASCHINE
	4276	# DURCHGEFUEHRT WERDEN SOLL) EINFUEGEN, FALLS NICHT BEREITS VOR-
	4277	# HANDEN
	4278	[[ $(echo $mc \| grep -c "\-C") = 0 ]] && mc="$mc -C"
	4279	[[ $(echo $mc \| grep -c "\-v") = 0 ]] && mc="$mc -v"
	4280	[[ $(echo $mc \| grep -c "\-b") = 0 ]] && mc="$mc -b"
	4281	if [[ $(echo $mc \| grep -c "#") != 0 ]]
	4282	then
	4283	mc=`echo $mc \| sed 's/#/f/g'`
	4284	fi
	4285
	4286
	4287	# JOB STARTEN
	4288	printf "\n\n *** initiating restart-run on \"$return_addres\" using command:\n"
	4289	echo " $mc"
	4290	printf "\n$striche\n"
	4291	if [[ $localhost != $fromhost ]]
	4292	then
	4293
[693]	4294	if [[ $localhost = lcsgih \|\| $localhost = lcsgib \|\| $localhost = nech \|\| $localhost = ibmh \|\| $localhost = ibmkisti \|\| $localhost = ibmku \|\| $localhost = ibms ]]
[1]	4295	then
	4296	echo "*** ssh will be used to initiate restart-runs!"
	4297	echo " return_addres=\"$return_addres\" "
	4298	echo " return_username=\"$return_username\" "
[635]	4299	if [[ $(echo $return_addres \| grep -c "130.75.105") = 1 ]]
[1]	4300	then
[374]	4301	ssh $return_addres -l $return_username "PATH=\$PATH:$LOCAL_MRUN_PATH;export LD_LIBRARY_PATH=\$LD_LIBRARY_PATH:/muksoft/packages/intel/Compiler/11.0/083/lib/intel64/;export PALM_BIN=$LOCAL_MRUN_PATH;cd $LOCAL_PWD; $mc "
[1]	4302	else
[693]	4303	if [[ $localhost = ibmkisti ]]
	4304	then
	4305	ssh $usern@gaiad "ssh $return_addres -l $return_username \"PATH=\\\$PATH:$LOCAL_MRUN_PATH;export PALM_BIN=$LOCAL_MRUN_PATH;cd $LOCAL_PWD; $mc\" "
	4306	else
	4307	ssh $return_addres -l $return_username "PATH=\$PATH:$LOCAL_MRUN_PATH;export PALM_BIN=$LOCAL_MRUN_PATH;cd $LOCAL_PWD; $mc "
	4308	fi
[1]	4309	fi
	4310	else
	4311	printf "\n +++ no restart mechanism available for host \"$localhost\" "
	4312	locat=restart; exit
	4313	fi
	4314
	4315	# WARTEN, DAMIT SICH RESTART JOB IN QUEUE EINREIHEN KANN, BEVOR
	4316	# DER AKTUELLE JOB ENDET
[399]	4317	if [[ $queue = special1q ]]
	4318	then
	4319	sleep 120
	4320	else
	4321	sleep 30
	4322	fi
[1]	4323
	4324	else
	4325
	4326	# BEI RECHNUNGEN AUF LOKALER MASCHINE KANN MRUN DIREKT AUFGE-
[108]	4327	# RUFEN WERDEN, AUSSER AUF lcfimm
[1]	4328	cd $LOCAL_PWD
[108]	4329	if [[ $localhost = lcfimm ]]
	4330	then
	4331	ssh $return_addres -l $return_username "PATH=\$PATH:$LOCAL_MRUN_PATH;export PALM_BIN=$LOCAL_MRUN_PATH;cd $LOCAL_PWD; $mc "
	4332	else
	4333	eval $mc # ' MUESSEN AUSGEWERTET WERDEN
	4334	fi
[1]	4335	cd - > /dev/null
	4336	fi
	4337	printf "\n$striche\n *** restart-run initiated \n"
	4338
	4339
	4340	# EVENTUELL INPUT-DATEIEN, DIE VON TEMPORAEREM DATENVERZEICHNIS
	4341	# GEHOLT WORDEN SIND, LOESCHEN
	4342	(( i = 0 ))
	4343	while (( i < iin ))
	4344	do
	4345	(( i = i + 1 ))
	4346	if [[ "${got_tmp[$i]}" = true && $keep_data_from_previous_run = false ]]
	4347	then
	4348	rm -r $tmp_data_catalog/${frelin[$i]}
	4349	fi
	4350	done
	4351
	4352	else
	4353
	4354	printf "\n +++ no restart-run possible, since errors occured"
	4355	printf "\n during the archive process"
	4356	fi
	4357
	4358	fi
	4359
	4360
	4361
	4362	# EVTL. EMAIL-BENACHRICHTIGUNG UEBER ABGESCHLOSSENEN LAUF
[352]	4363	if [[ "$email_notification" != "none" ]]
[1]	4364	then
	4365
	4366	if [[ $localhost != $fromhost ]]
	4367	then
	4368	if [[ -f CONTINUE_RUN ]]
	4369	then
	4370	echo "PALM restart run necessary" > email_text
	4371	echo "description header of actual run:" >> email_text
	4372	cat CONTINUE_RUN >> email_text
	4373	echo "mrun-command to restart:" >> email_text
	4374	echo "$mc" >> email_text
	4375	else
	4376	echo "PALM run with base filename \"$fname\" on host \"$localhost\" finished" > email_text
	4377	fi
	4378	mail $email_notification < email_text
	4379	printf "\n *** email notification sent to \"$email_notification\" "
	4380	fi
	4381	fi
	4382
	4383
	4384
	4385	# ALLE ARBEITEN BEENDET. TEMPORAERER KATALOG KANN GELOESCHT WERDEN
	4386	cd $HOME
	4387	[[ $delete_temporary_catalog = true ]] && rm -rf $TEMPDIR
	4388
	4389	else
	4390
	4391
	4392	# FALLS AUF REMOTE-MASCHINE GERECHNET WERDEN SOLL, WERDEN JETZT ENTSPRE-
	4393	# CHENDE AKTIONEN DURCHGEFUEHRT
	4394
	4395	# MRUN-BEFEHL FUER REMOTE-MASCHINE ZUSAMMENSTELLEN
[693]	4396	mrun_com="$mrun_script_name -a $afname -c $config_file -d $fname -h $host -H $fromhost -m $memory -t $cpumax -q \$queue -R $return_addres -U $return_username -u $remote_username"
[1]	4397	[[ "$cpp_opts" != "" ]] && mrun_com=${mrun_com}" -D \"$cpp_opts\""
[79]	4398	[[ "$global_revision" != "" ]] && mrun_com=${mrun_com}" -G \"$global_revision\""
[123]	4399	[[ $group_number != none ]] && mrun_com=${mrun_com}" -g $group_number"
[1]	4400	[[ $do_compile = true ]] && mrun_com=${mrun_com}" -s \"$source_list\""
	4401	[[ "$input_list" != "" ]] && mrun_com=${mrun_com}" -i \"$input_list\""
	4402	[[ $ignore_archive_error = true ]] && mrun_com=${mrun_com}" -I"
	4403	[[ $keep_data_from_previous_run = true ]] && mrun_com=${mrun_com}" -k"
	4404	[[ "$additional_conditions" != "" ]] && mrun_com=${mrun_com}" -K \"$additional_conditions\""
	4405	# [[ "$makefile" != "$source_path/Makefile" ]] && mrun_com=${mrun_com}" -M \"$makefile\""
	4406	[[ "$output_list" != "" ]] && mrun_com=${mrun_com}" -o \"$output_list\""
	4407	[[ "$read_from_config" = false ]] && mrun_com=${mrun_com}" -S"
	4408	[[ $do_trace = true ]] && mrun_com=${mrun_com}" -x"
	4409	[[ "$numprocs" != "" ]] && mrun_com=${mrun_com}" -X $numprocs"
	4410	if [[ $use_openmp = true ]]
	4411	then
[622]	4412	mrun_com=${mrun_com}" -O $threads_per_task"
[1]	4413	fi
[622]	4414	[[ "$tasks_per_node" != "" ]] && mrun_com=${mrun_com}" -T $tasks_per_node"
[397]	4415	[[ $store_on_archive_system = true ]] && mrun_com=${mrun_com}" -A"
[1]	4416	[[ $package_list != "" ]] && mrun_com=${mrun_com}" -p \"$package_list\""
	4417	[[ $return_password != "" ]] && mrun_com=${mrun_com}" -P $return_password"
	4418	[[ $delete_temporary_catalog = false ]] && mrun_com=${mrun_com}" -B"
[127]	4419	[[ $node_usage != default && "$(echo $node_usage \| cut -c1-3)" != "sla" && $node_usage != novice ]] && mrun_com=${mrun_com}" -n $node_usage"
[291]	4420	[[ "$ocean_file_appendix" = true ]] && mrun_com=${mrun_com}" -y"
[206]	4421	[[ $run_coupled_model = true ]] && mrun_com=${mrun_com}" -Y \"$coupled_dist\""
[1]	4422	if [[ $do_remote = true ]]
	4423	then
	4424	printf "\n>>>> MRUN-command on execution host:\n>>>> $mrun_com \n"
	4425	fi
	4426
	4427
	4428	# ZUSAMMENSTELLUNG DES JOBSCRIPTS AUF DATEI jobfile
	4429	jobfile=jobfile.$RANDOM
	4430
	4431
	4432	# TEMPORAERES VERZEICHNIS GENERIEREN UND NACH DORT WECHSELN
	4433	echo "mkdir $TEMPDIR" >> $jobfile
	4434	echo "cd $TEMPDIR" >> $jobfile
	4435
	4436
	4437	# EVENTUELL FEHLERVERFOLGUNG AKTIVIEREN
	4438	if [[ $do_trace = true ]]
	4439	then
	4440	echo "set -x" >> $jobfile
	4441	else
	4442	echo "set +vx" >> $jobfile
	4443	fi
	4444
	4445
[191]	4446	# BEREITSTELLUNG VON QUELLTEXTEN, MRUN-SCRIPTS UND KONFIGURATIONS-
	4447	# DATEI FUER DEN JOB
[693]	4448	if [[ $( echo $host \| cut -c1-5 ) = lcsgi \|\| $host = ibmkisti ]]
[1]	4449	then
	4450
[191]	4451	# KONFIGURATIONSDATEI UND MRUN_SCRIPT IN DAS SOURCE-VERZEICHNIS
	4452	# KOPIEREN
	4453	if [[ $restart_run != true ]]
	4454	then
	4455	cp $config_file $working_directory/SOURCES_FOR_RUN_$fname
	4456	cp ${PALM_BIN}/$mrun_script_name $working_directory/SOURCES_FOR_RUN_$fname
	4457	fi
[1]	4458
[191]	4459	# SOURCE-VERZEICHNIS VOM LOKALEN RECHNER PER SCP TRANSFERIEREN
[693]	4460	# (BEI JOBS AUF LOKALEM RECHNER REICHT CP)
[193]	4461	echo "set -x" >> $jobfile
[693]	4462	if [[ $host = $localhost ]]
	4463	then
	4464	echo "cp -r $working_directory/SOURCES_FOR_RUN_$fname ." >> $jobfile
	4465	else
	4466	if [[ $host != ibmkisti ]]
	4467	then
	4468	echo "scp -r $return_username@$return_addres:$working_directory/SOURCES_FOR_RUN_$fname ." >> $jobfile
	4469	else
	4470	# on KISTI's IBM firewall is only opened on interactive node
	4471	echo "localdir=\`pwd\`" >> $jobfile
	4472	echo "ssh $remote_username@gaiad \"cd \$localdir; scp -r $return_username@$return_addres:$working_directory/SOURCES_FOR_RUN_$fname .\" " >> $jobfile
	4473	fi
	4474	fi
[193]	4475	echo "export SOURCES_COMPLETE=true" >> $jobfile
[1]	4476
[193]	4477	# QUELLTEXTE, MRUN-SCRIPT UND KONFIGURATIONSDATEI IN DAS AKTUELLE
	4478	# ARBEITSVERZEICHNIS VERSCHIEBEN
	4479	echo "mv SOURCES_FOR_RUN_$fname/$config_file . " >> $jobfile
	4480	echo "mv SOURCES_FOR_RUN_$fname/$mrun_script_name . " >> $jobfile
	4481	echo "execute_mrun=true" >> $jobfile
	4482	echo " " >> $jobfile
	4483
[191]	4484	else
	4485
	4486	# ABSPEICHERN DER QUELLTEXTE (NUR FALLS UEBERSETZT WERDEN SOLL)
	4487	# SOWIE GEGEBENENFALLS DES MAKEFILES
	4488	if [[ $do_compile = true ]]
	4489	then
	4490
	4491	source_catalog=SOURCES_FOR_RUN_$fname
	4492
	4493	# UNTERVERZEICHNIS FUER QUELLTEXTE UND MAKEFILE ANLEGEN
	4494	# MRUN WIRD DIESES VRZEICHNIS UEBER ENVIRONMENT VARIABLE
	4495	# MITGETEILT (UEBERSTEUERT ANGABE IN KONFIGURATIONSDATEI)
	4496	echo "mkdir SOURCES_FOR_RUN_$fname" >> $jobfile
	4497	echo "export SOURCES_COMPLETE=true" >> $jobfile
	4498	echo "cd SOURCES_FOR_RUN_$fname" >> $jobfile
	4499
	4500	for filename in $source_list
	4501	do
	4502	# ABDECKZEICHEN VERHINDERN, DASS ERSETZUNGEN ERFOLGEN
	4503	echo "cat > $filename << \"%END%\"" >> $jobfile
	4504	cat $source_catalog/$filename >> $jobfile
	4505	echo " " >> $jobfile
	4506	echo "%END%" >> $jobfile
	4507	echo " " >> $jobfile
	4508	done
	4509
[1]	4510	# ABDECKZEICHEN VERHINDERN, DASS ERSETZUNGEN ERFOLGEN
[191]	4511	echo "cat > Makefile << \"%END%\"" >> $jobfile
	4512	cat $source_catalog/Makefile >> $jobfile
[1]	4513	echo " " >> $jobfile
	4514	echo "%END%" >> $jobfile
	4515	echo " " >> $jobfile
	4516
[191]	4517	echo "cd - > /dev/null" >> $jobfile
[1]	4518
[191]	4519	fi
[1]	4520
	4521
[191]	4522	# ABSPEICHERN DER KONFIGURATIONSDATEI
	4523	# ABDECKZEICHEN VERHINDERN, DASS ERSETZUNGEN ERFOLGEN
	4524	echo "cat > $config_file << \"%END%\"" >> $jobfile
	4525	cat $config_file >> $jobfile
	4526	echo "%END%" >> $jobfile
	4527	echo " " >> $jobfile
[1]	4528
	4529
[191]	4530	# ABSPEICHERN DER AKTUELLEN MRUN-VERSION
	4531	# ABDECKZEICHEN VERHINDERN, DASS ERSETZUNGEN ERFOLGEN
[693]	4532	echo "cat > $mrun_script_name <<\"%END%\"" >> $jobfile
	4533	cat ${PALM_BIN}/$mrun_script_name >> $jobfile
	4534	echo "%END%" >> $jobfile
[191]	4535	echo "chmod u+x $mrun_script_name" >> $jobfile
	4536	echo "execute_mrun=true" >> $jobfile
	4537	echo " " >> $jobfile
[1]	4538
	4539	fi
	4540
[191]	4541
[1]	4542	# EVTL. BENOETIGTE INPUT-DATEIEN PER FTP HOLEN ODER DEM JOB DIREKT
	4543	# MITGEBEN UND AUF DEM REMOTE-RECHNER IM BENUTZERVERZEICHNIS ABLEGEN
	4544	# FALLS DIESES NICHT EXISTIERT, WIRD VERSUCHT, DAS JEWEILS LETZTE
	4545	# UNTERVERZEICHNIS DES PFADNAMENS ANZULEGEN
	4546	if [[ $do_remote = true ]]
	4547	then
	4548	(( i = 0 ))
	4549	while (( i < iin ))
	4550	do
	4551	(( i = i + 1 ))
	4552	echo "[[ ! -d ${pathin[$i]} ]] && mkdir -p ${pathin[$i]}" >> $jobfile
	4553	if [[ "${transin[$i]}" = job ]]
	4554	then
	4555	echo "cat > ${remotepathin[$i]} <<\"%END%\"" >> $jobfile
	4556	eval cat ${pathin[$i]}/${frelin[$i]} >> $jobfile
	4557	echo " " >> $jobfile
	4558	echo "%END%" >> $jobfile
	4559	else
[82]	4560	echo "batch_scp -b -o -g -s -u $return_username $return_addres ${remotepathin[$i]} \"${pathin[$i]}\" ${frelin[$i]}" >> $jobfile
[1]	4561	fi
	4562
	4563	# UEBERPRUEFEN, OB DATEI ANGELEGT WERDEN KONNTE
	4564	echo "if [[ \$? = 1 ]]" >> $jobfile
	4565	echo "then" >> $jobfile
	4566	echo " echo \" \" " >> $jobfile
	4567	echo " echo \"+++ file ${remotepathin[$i]} could not be created\" " >> $jobfile
	4568	echo " echo \" please check, if directory exists on $host!\" " >> $jobfile
	4569	echo " echo \"+++ MRUN will not be continued\" " >> $jobfile
	4570	echo " execute_mrun=false" >> $jobfile
	4571	echo "fi" >> $jobfile
	4572	done
	4573	fi
	4574
	4575	# ARBEITSKATALOG AUF DER LOKALEN MASCHINE FUER EVENTUELLE
	4576	# FORTSETZUNGSLAUEFE PER ENV-VARIABLE UEBERGEBEN
	4577	echo "LOCAL_PWD=$working_directory" >> $jobfile
	4578	echo "export LOCAL_PWD" >> $jobfile
	4579
	4580	# EBENSO LOKALEN MRUN-KATALOG UEBERGEBEN
[22]	4581	echo "LOCAL_MRUN_PATH=$PALM_BIN" >> $jobfile
[1]	4582	echo "export LOCAL_MRUN_PATH" >> $jobfile
	4583
[66]	4584	# WORKAROUND FUER RIAM-NEC-JOBS WEGEN PROFILE-SCHWIERIGKEITEN
[69]	4585	if [[ $localhost_realname = "gate" \|\| $localhost = lctit ]]
[66]	4586	then
	4587	echo "export PALM_BIN=$PALM_BIN" >> $jobfile
	4588	fi
[1]	4589
[693]	4590	# MRUN AUF ZIELRECHNER AUFRUFEN (queue is workaround for ibmkisti)
[1]	4591	echo "set -x" >> $jobfile
[693]	4592	echo "queue=$queue" >> $jobfile
[1]	4593	echo "[[ \$execute_mrun = true ]] && ./$mrun_com" >> $jobfile
	4594
	4595	echo 'ls -al; echo `pwd`' >> $jobfile
	4596	echo "cd \$HOME" >> $jobfile
	4597	echo "rm -rf $TEMPDIR" >> $jobfile
	4598
	4599
	4600
	4601
	4602	# JOB PER SUBJOB STARTEN
	4603	if [[ $silent = false ]]
	4604	then
	4605	printf "\n "
	4606	else
	4607	printf "\n\n"
	4608	fi
	4609
[352]	4610	subjob $job_on_file -h $host -u $remote_username -g $group_number -q $queue -m $memory -N $node_usage -t $cpumax $XOPT $TOPT $OOPT -n $fname -v -c $job_catalog -e $email_notification $jobfile
[1]	4611	rm -rf $jobfile
	4612
	4613
	4614	fi # ENDE REMOTE-TEIL
	4615
	4616
	4617
	4618	# ENDE DER PROZEDUR

Note: See TracBrowser for help on using the repository browser.

Download in other formats:

| Impressum | ©Leibniz Universität Hannover |