PALM   a parallelized LES model


technical/numerical documentation 1998-2008
 
This document has been frozen in February 2010
Links may not work properly!

Table of contents
 
 


1.0  History of changes

The following table documents all changes to the model since July, 7th 1998. Entries in the fourth column specify the type of the change: change  of existing code (C), new code (N) or bugfix (B). More detailed information about the changes may be found in the header comment lines of the respective routines.
 
 
date author model
version
type description affected routines
06/07/98 SR 1.0 C √„nderung von Default-Werten (initializing_actions, end_time, fcl_factor, use_prior_plot1d_parameters), zus√§tzliche √œberpr√ľfung von Parametern auf Zul√§ssigkeit (prandtl_layer, psolver, nx, ny, nz, fcl_factor), H√∂he der Schnittebenen der Horizontalschnitte wird auf Plots exakt ausgegeben (uv- bzw. w-Gitter). modules, check_parameters, plot_2d
" SR 1.0 N Neues Modul test_variables wird in (fast) allen Programmteilen benutzt (USE). Wichtig: dieses Modul steht in der Datei module_test.f90. Diese Datei mu√Ÿ in der mrun- und add_library - Konfigurationsdatei bei der Variablen module_files aufgef√ľhrt werden. (fast) alle 
Neu: module_test
16/07/98 SR 1.0 N Anpassung an die T3E-Rechner in Juelich (cpp - Direktiven) advec_s_bc, check_open, check_parameters, cpu_zeitmessung, init_pegrid, local_getenv, local_system, local_tremain, local_tremain_ini, poisfft
24/07/98 MS 1.0 C Anpassung des Druckloesealgorithmusses an Vorgehensweise von Schmidt, Schumann und Volkert (DFVLR-Mitteilung 84-15, S. 20-26)  poisfft
27/07/98 MS 1.0 C Berechnung von l in maketri geschieht nun ausserhalb der k-Schleife, da unabhaengig von k, dadurch Rechenzeitersparnis  poisfft
05/08/98 SR 1.0 C Entfernung der Sonderbehandlung der Advektionsterme (Piascek-Williams) bei k=1. Terme werden nun √ľberall gleich berechnet. Evtl. leichte Erh√∂hung der Varianzen in Oberfl√§chenn√§he. advec_u_pw, advec_v_pw, advec_w_pw
" SR 1.0 C Mischungsweganpassung (s. adjust_mixing_length) erfolgt nur noch bei k=1. Unrealistische sekund√§re Maxima in den Km-Profilen oberhalb der Prandtl-Schicht werden damit vermieden. Leichte √„nderungen in den oberfl√§chennahen Turbulenzgr√∂√Ÿen sind die Folge. diffusivities
" SR 1.0 B Falsche Berechnung der ben√∂tigten CPU-Zeit pro Sekunde simulierter Zeit bei Fortsetzungsl√§ufen korrigiert. 

Berechnung von cross_uxmin(1) und cross_uxmax(1) aus den Werten von ug und vg entfernt. Lieferte falsche Wertebereiche f√ľr Plots von Vertikalprofilen, wenn im ersten Koordinatenkreuz nicht die Windgeschwindigkeiten gezeichnet wurden.

header, check_parameters
" SR 1.0 N Das erlaubte Maximum und Minimum der Rif-Zahl ist nun mittels der Initialparameter rif_max und rif_min einstellbar (Werte waren bisher fest auf -2.0, +1.0 eingestellt). check_parameters, header, init_1d_model, modules, parin, prandtl_fluxes, read_var_list, write_var_list
23/09/98 SR 1.0 B Initialisierung von Feldern im Fall von initializing_action = "set_constant_profiles" und "set_1d-model_profiles" korrigiert. 

2D-Plotausgabe von u* und theta* korrigiert. Hier wurden bisher falsche Felder herausgeschrieben. 

Bei SOR-Aufrufen war bisher dp nicht initialisiert. Entsprechend geändert.

init_3d_model, plot_2d, pres
" SR 1.0 C Ausgabeformate f√ľr Statistik-Ausgaben auf Datei RUN_CONTROL um einige Kommastellen erweitert. 

Lieferung von ausf√ľhrlicheren Informationen bei Programmabbruch durch Unterschreitung des minimal erlaubten Zeitschritts.

run_control, timestep
" SR 1.0 N Rechnungen mit geneigter Ebene zur Simulation von Kaltluftabfl√ľssen sind erlaubt. Gestartet und gesteuert werden sie mit dem Intialisierungsparameter alpha_surface. Realisiert wird die geneigte Ebene durch entsprechende Schr√§gstellung des Schwerkraftvektors. Dadurch erh√§lt auch die u-Komponente der Windgeschwindigkeit einen Anteil der Auftriebskraft. 

Diese Implementierung ist noch nicht abgeschlossen. Die Simulation von Kaltluftabfl√ľssen erfordert bisher noch Eingriffe "von Hand".

buoyancy, check_parameters, header, leap_frog, modules, parin, prandtl_fluxes, read_var_list, write_var_list
15/12/98 SR 1.0 N Vereinbarung einer Cache-Layer zur schnelleren bin√§ren I/O auf T3E-Rechnern. 

Ermittelung wahrer I/O-Zeiten f√ľr bin√§re Ein-/Ausgabe auf T3E-Rechnern mit TIMEF.

check_open, 
cpu_zeitmessung, init_3d_model, write_3d_model
" SR 1.0 B Bekanntgabe von run_description_header an alle PE's. 

Diese Programm√§nderung erfordert auf T3E-Rechnern zwingend den Einsatz der MPI-Version mpt.1.2.0.1. In dieser neuen Version tritt auch der alte Fehler in MPI_ALLREDUCE nicht mehr auf, so da√Ÿ ab jetzt mpt.1.1.0.1 nicht mehr verwendet werden mu√Ÿ.

parles
05/02/99 SR 1.1 C/N Implementierung des Upstream-Spline- Verfahrens zur Advektion von Impuls und skalaren Gr√∂√Ÿen (neue UP's, s. rechts). Berechnung von Variablen f√ľr spezielle Advektionsverfahren in eigenst√§ndiges UP ausgelagert (init_advec + eigenes Modul advection). Reines Euler-/Upstream-Verfahren kann gerechnet werden. 

Schnelle binäre I/O (s. 15/12/98) wieder abgeschaltet, da Dateien nicht wieder einlesbar sind. Druckstörung (p) wird zu Beginn auf 0 gesetzt, damit sie keine Tendenzen im ersten Zeitschritt hervorruft.

advec_s_bc, check_open, check_parameters, flow_statistics, header, init_3d_model, init_grid, leap_frog, modules, parin, plot_2d, plot_ts, read_var_list, timestep, write_3d_binary, write_var_list 

Neu:
advec_s_ups, advec_u_ups, advec_v_ups, advec_w_ups, init_advec, long_filter, spline_x, spline_y, spline_z

24/02/99 SR 1.1a C/N Kleinere Erg√§nzungen / √„nderungen f√ľr Upstream - Spline, 
Diverse Verbesserungen bei Berechnung der Diffusionskoeffizienten (exaktere Formulierung von Scherungen, Dissipation gemaess Original - Deardorff - Ansatz, Mischungswegreduktion nur, wenn Schichtung sowohl oberhalb als auch unterhalb des entsprechenden Gitterpunktes stabil ist), Wertebegrenzung bei Spline - Berechnung zur Verhinderung von "√œberschwingern"
check_parameters, diffusion_e, diffusion_pt, diffusivities, header, init_advec, long_filter, modules, production_e, spline_x, spline_y, spline_z
" SR 1.1a N/B Erweiterung des des Bott - Chlond - Schemas auf Advektion der TKE, Korrektur der Dichteberechnung (war falsch bei Galilei-Transformation) advec_s_bc, leap_frog
01/03/99 MS 1.1b N Kleinere Erg√§nzungen f√ľr Upstream - Spline: 
  • Gradientenkontrolle f√ľr den nicht-parallelen Teil
  • statistische Auswertung √ľber den prozentualen Anteil des Upstream-Verfahrens an der Gesamtadvektion fuer nicht-parallelen Teil
modules, spline_x, spline_y, spline_z
" SR 1.1b N Datenkompression f√ľr 3D-Plotausgaben check_open, check_parameters, close_files, header, modules, parin, plot_3d 

Neu:
write_compressed

03/03/99 SR 1.1c B/C Schnelle bin√§re I/O auf T3E-Rechnern funktioniert jetzt. Beim bin√§ren Einlesen d√ľrfen Records nicht mit dem Trick "READ (..) idum" √ľberlesen werden. Zeitmessungen auf T3E erfolgen jetzt grunds√§tzlich mit TIMEF. check_open, cpu_zeitmessung, init_3d_model, modules, write_3d_binary
25/03/99 SR 1.1d N Einbau einer Rayleigh-D√§mpfungsschicht sowie flexiblere Wahl der numerischen Begrenzer f√ľr das Upstream - Spline - Verfahren. check_parameters, header, init_3d_model, leap_frog, modules, parin, read_var_list, spline_x, spline_y, spline_z, write_var_list
" SR 1.1d B Korrektur des Zeitschrittkriteriums bei z=zp im 1D-Modell init_1d_model
25/11/99 SR 1.1e N Partikelplots mit vtk-Graphiksoftware sind auf t3eh m√∂glich. Ausgabe erfolgt im dvr-Format. Bisher noch nicht nutzbar, da noch Einstellm√∂glichkeiten f√ľr die Partikelquelle(n) fehlen. Wichtig: Die Benutzung der vtk-Software erfordert die zus√§tzliche mrun-Option -g. Das Hauptprogramm parles wird in diesem Fall zum Unterprogramm und von einem c++-Programm aus aufgerufen. header, init_3d_model, init_pegrid, leap_frog, modules, parin, parles, read_var_list, write_var_list 

Neu:
advec_particles, init_particles, plot_particles

" SR 1.1e B/C TIMEF-Zeitmessungen von Subroutine- auf Function-Aufruf umgestellt. Laufindexkorrektur in spline_y. Fehler trat bei nx /= ny auf. cpu_zeitmessung, spline_y
28/12/99 SR 1.1f N Plot von Isooberfl√§chen mit vtk-Graphiksoftware sind auf t3eh m√∂glich. Ausgabe erfolgt im dvr-Format. Bisher noch nicht allgemein nutzbar, da noch diverse manuelle Eingriffe n√∂tig sind. header, leap_frog, modules, parin, plot_particles, read_var_list, write_var_list 

Neu:
plot_isosurface

10/01/00 SR 1.2 C Randbedingungen f√ľr w (=0) werden explizit gesetzt, und zwar nicht nur bei  k = nzb, nzt+1 sondern auch bei  k = nzt, wo bislang der vom vorigen Zeitschritt durch den Druckl√∂ser ermittelte Wert stand. Auch jetzt wird der Wert bei k = nzt anschlie√Ÿend durch den Druckl√∂ser modifiziert. Soweit bis jetzt abzusehen, hat diese √„nderung nur minimale Auswirkungen auf den Prognoseverlauf. Evtl. gibt es st√§rkere Auswirkungen bei exzessivem Auftreten von Schwerewellen am oberen Rand (die dort aber eigentlich sowieso nicht hingeh√∂ren). 
Variablen√ľbergabe an boundary_conds erfolgt 
√ľber Modul und nicht mehr per Parameterliste. 

Bei Galilei-Transformation wird jetzt defaultm√§√Ÿig der geostrophische Wind als Translationsgeschwindigkeit verwendet (neuer Initialisierungsparameter use_ug_for_galilei_tr). Dadurch wird das verst√§rkte Auftreten von Schwerewellen bei h√∂heren geostrophischen Windgeschwindigkeiten unterdr√ľckt. 

3D-Feldvariablen (u, v, w, pt, e, km, kh) sowie einige 2D-Variablen (usws, vsws, rif, shf) sind nun als Pointer deklariert, denen nach jedem Zeitschritt neue Targets zugeordnet werden (f√ľr die ben√∂tigten  3 Zeitebenen existieren die Targets ..._1 (z.B. u_1), ..._2, ..._3). Durch die jeweilige Neuzuordnung der Targets entf√§llt das Umspeichern der Zeitebenen komplett (Zeitersparung: ca. 5% der Gesamtrechenzeit). 
Die Umordnung der Zeitebenen wurde ins neue UP swap_timelevel ausgelagert. 
Wichtig: Die √œbersetzungszeiten haben durch die Einf√ľhrung der Pointer t.w. sehr deutlich zugenommen (leap_frog ben√∂tigt jetzt z.B. ca. 60 Sekunden). Au√Ÿerdem ben√∂tigt der Compiler jetzt wesentlich mehr Speicher. Der "memory"-Parameter in den Konfigurationsdateien (.mrun.config, .al.config) muss nun mindestens den Wert 60 (MB) (t3eh, t3ej2) bzw. 45 (t3eb) und 96 (vpp) haben. 
Achtung: bislang wurde diese √„nderung nur f√ľr das reine Leapfrog-Verfahren getestet! 

Neue Laderoption "-D preset=nan" empfohlen (s. Beispiel-Konfigurationsdatei). Ebenso empfehlenswert f√ľr Debug-L√§ufe: Compileroption "-ei". Diese bewirkt einen Programmabbruch, falls mit nicht initialisierten Variablen gearbeitet wird (ist aber zeitaufwendig in der Ausf√ľhrung!).

boundary_conds, check_parameters, header, init_3d_model, leap_frog, modules, parin, read_var_list, timestep, write_var_list 

Neu:
swap_timelevel

10/01/00 SR 1.2 B Der Asselin-Filter wird vor Umordnung der Zeitebenen aufgerufen. Dies geschah bisher f√§lschlicherweise erst danach, sodass der Zeitfilter asymmetrisch arbeitete. 
Achtung: diese √„nderung hat Auswirkungen auf den Prognoseverlauf. Die Testergebnis-Datei wurde entsprechend ge√§ndert.
leap_frog
14/02/00 MS 1.2 B/C 1D-Modell:
Der Asselin-Filter im 1D-Modell wird nun auch vor der Umordnung der Zeitebenen aufgerufen (s.o.). 

3D-Modell:
TKE wurde bisher bei scalar_advec=bc-scheme einer zeitlichen Filterung unterzogen. Dies wird nun unterbunden. 

Die Divergenz der Impulsfl√ľsse an der Prandtl - Schicht - Obergrenze wird nun √ľber eine ganze Gitterweite gebildet. Die bisherige Methode lieferte im Vergleich mit Ergebnissen anderer LES-Modelle  zu gro√Ÿe Impulsfl√ľsse. 

Bei der Berechnung der Diffusion der Temperatur werden die Diffusionskoeffizienten nun wieder durch Mittelung f√ľr die entsprechenden  Gitterpunkte bereitgestllt. Das Maximumkriterium, das im Zuge der Implementierung des Upstream-Spline Advektionsverfahrens eingebaut wurde, bewirkte ein zu rasches Anwachsen der Grenzschicht mit der Zeit. 

Achtung: diese √„nderungen haben Auswirkungen auf den Prognoseverlauf. Die Testergebnis-Datei wurde entsprechend ge√§ndert.

init_1d_model, asselin_filter, diffusion_u, diffusion_v, 
diffusion_pt
17/02/00 MS 2.0 N Allgemeine Beschreibung der √„nderungen:
vollst√§ndige Implementierung des wolkenphysikalischen Moduls 

Zur Implementierung der Wolkenphysik waren umfangreiche √„nderungen in zahlreichen Programmteilen notwendig (Beschreibung siehe unten ). Im einzelnen wurden folgende Unprogramme ge√§ndert: 
modules
1) Bereitstellung neuer 3D-, 2D- und 1D-Felder f√ľr die Prognose des Gesamtwassergehaltes 
2) "cloud_parameters" als neues Modul mit wolkenphysikalischen Variablen 
parin: Initialisierungsparameterliste wurde erweitert (s.u.) 
read_var_list, write_var_list, write_3d_binary: In/Output der neuen Initialisierungsparamter und Felder f√ľr Fortsetzungsl√§ufe 
check_parameters: Initialisierung des Feuchteprofils und Ausgabe von neuen Vertikalprofilen: 
vpt: virtuelle poteniellen Temperatur,
lpt: potentielle Fl√ľssigwassertemperatur,
q: Gesamtwassergehalt,
qv: spezifische Feuchte,
ql: Fl√ľssigwassergehalt,
w"vpt", w*vpt*, wvpt: Auftriebsfluss (subskalig, aufgelöst, gesamt),
w"q", w*q*, wq: Wasserfluss (subs., aufg., ges.),
w"qv", w*qv*, wqv: Feuchtefluss (subs, aufg., ges.)
init_3d_model: Initialisierung aller Feuchte/Wasserfelder 
init_1d_model: Anpassung an Rechnungen mit Feuchte 
header: Headerausgaben f√ľr Feuchtebehandlung und Wolkenphysik eingef√ľgt, au√Ÿerdem Formatnummern erweitert um Platz f√ľr weitere Ausgaben zu schaffen 
leap_frog:
1) alle prognostischen Gleichungen wurden in prognostic_equations ausgelagert 
2) Aufrufe der neuen Unterprogramme (UP) compute_vpt, calc_liquid_water_content 
3) veraenderter Aufruf von diffusivities 
boundary_conds, asselin_filter, swap_timelevel, flow_statistics: Anpassung an Feuchtebehandlung 
plot2d, plot3d: Ausgabemöglichkeiten von q und ql
buoyancy: Anpassung des Autriebstermes an Feuchtebehandlung durch neue √œbergabeparameter: Temperatur + Nummer des entsprechenden mittleren Temperaturprofils 
diffusion_e: Anpassung der Mischungswegberechnung an Feuchtebehandlung durch Variation der √œbergabeparameter 
production_e: TKE-Produktion durch Auftriebskr√§fte an Feuchtebehandlung angepasst 
diffusivities: siehe diffusion_e 
prandtl-fluxes: Prandtl-Schicht fuer den Gesamtwassergehalt eingef√ľgt 

Zus√§tzlich wurden folgende Unterprogramme NEU eingef√ľhrt: 
inti_cloud_physics: Initialisierung wolkenphysikalischer Parameter 
prognostic_equations: Auslagerung der prognostischen Gleichungen aus leap_frog, zus√§tzliche porgnostische Gleichung f√ľr den Gesamtwassergehalt 
diffusion_s: Berechnung der Diffusion skalarer Gr√∂√Ÿen (Temperatur und Gesamtwassergehalt), ersetzt diffusion_pt 
calc_liquid_water_content: Berechnung des Fl√ľssigwassergehalts 
calc_radiation: Parametrisierung langwelliger Strahlungsprozesse √ľber des Schema der effektiven Emissivit√§t 
calc_precipitation:Parametrisierung von Niederschlagsprozessen mit Hilfe eines vereinfachten Kesslerschemas 
impact_of_latent_heat: Ber√ľcksichtigung der durch Niederschlagsprozesse zur√ľckbleibenden latenten Waerme innerhalb eines Gittervolumens 
compute_vpt: Berechnung der virtuellen potentiellen Temperatur 

Liste der neuen Initialisierungsparameter:
 
Parameter Typ Default Beschreibung
moisture L FALSE Ab-/Zuschalten des Gesamtwassergerhaltes als neue progn. Variable
cloud_physics L FALSE Ab-/Zuschalten des Kondensationsschemas
radiation L FALSE Ab-/Zuschalten des Strahlungsschemas
precipitation L FALSE Ab-/Zuschalten der Niderschlagsparametrisierung
bc_q_b C 'dirichlet' untere Randbedingung f√ľr q (siehe auch bc_pt_b)
bc_q_t C 'neumann' obere RB f√ľr q (siehe auch bc_pt_t)
q_surface R 0.0 Feuchtewert an der Erdoberfläche
q_surface_initial_change R 0.0 vgl. pt_surface_initial_change
q_vertical_gradient R(10) 0.0 vgl. pt_vertical_gradient
q_vertical_gradient_level R(10) 10000.0 vgl. pt_vertical_gradient_level
surface_pressure R 1013.25 Luftdruck an der Erdoberfläche
surface_waterflux R 0.0 oberflächennaher Wasser/Feuchtefluss
Fortsetzungsl√§ufe mit Daten, die mit Modellversionen kleiner 2.0 erzeugt wurden, sind aufgrund der erweiterten Initialisierungsparameterliste INKOMPATIBEL zu den Einleseroutinen der Version 2.0. Ein Update auf Version 2.0 sollte deshalb nach Abschluss eines kompletten Modelllaufs erfolgen. 

Achtung: diese √„nderungen haben Auswirkungen auf den Prognoseverlauf. Die Testergebnis-Datei wurde entsprechend ge√§ndert.

modules, parin, read_var_list, check_parameters, init_3d_model, init_1d_model, header, leap_frog. buoyancy, diffusion_e, production_e, diffusivities, prandtl_fluxes, boundary_conds, asselin_filter, swap_timelevels, flow_statistics, plot_2d, plot_3d, write_var_list, write_3d_binary 

Neu:
init_cloud_physics, prognostic_equations, diffusion_s, calc_liquid_water_content, calc_radiation, calc_precipitation, impact_of_latent_heat, comute_vpt

18/02/00 MS 2.0 B/N Im Zuge der Umstellung auf Modellversion 2.0  wurden noch kleine √„nderungen an den Unterprogrammen diffusion_e, production_e und diffusivities vorgenommen. In allen genannten 
Unterprogrammen wurde bisher ein strengeres Kriterium zur Bestimmung der Schichtungsverh√§ltnisse verwendet. Ein Modellvergleich zeigte jedoch, dass dadurch die Diffusion im Bereich der Inversion zu gross wird. 

Eine weitere Erg√§nzung betrifft die Advektionsverfahren: 
Das Bott-Chlond Advektionsverfahren ist nun auch f√ľr Rechnungen mit Feuchte/Wolkenphysik verf√ľgbar. 

Achtung: diese √„nderungen haben Auswirkungen auf den Prognoseverlauf. Die Testergebnis-Datei wurde entsprechend ge√§ndert.

diffusion_e, production_e, diffusivities, check_parameters
26/04/00 SR 2.0a C Vollst√§ndige Umstellung von vtk-Grafik-Software auf dvrp-Software. Grafik-Ausgaben im dvr-Format sind mittlerweile begrenzt nutzbar (Isooberfl√§chen sind darstellbar, siehe neue Laufparameter  dt_dvrp, threshold), es wird aber in der Anwendung in naher Zukunft noch diverse √„nderungen geben. Partikelausgabe vorerst nicht mehr m√∂glich (Verlagerung schon). header, init_particles, init_3d_model, leap_frog, modules, parin, parles, read_var_list, write_var_list 
Gestrichen:
plot_isosurface, plot_particles 

Neu:
init_dvrp, plot_dvrp

26/04/00 SR 2.0a C Durch zus√§tzlichen zyklischen Rand im Bott-Chlond-Schema sehr viele SENDRECV-Aufrufe eingespart. 
√„nderung des Namens einer eingelesenen Environment-Variablen in check_open von remote_addres nach return_addres
Prozessor-Topologie kann durch Benutzer vorgegeben werden. Dazu neue Initialisierungsparameter npex und npey. Durch √„nderungen am Header-Format wurde Testergebnis-Datei ge√§ndert.
advec_s_bc, check_open, header, init_pegrid, modules, parin, read_var_list, write_var_list
26/04/00 SR 2.0a N Rechnungen mit geneigter Oberfl√§che erlauben jetzt die Vorgabe eines stabil geschichteten Temperaturprofils mit konstantem Gradienten. Hinzuschalten von Feuchte funktioniert f√ľr Hangrechnungen noch nicht. advec_s_bc, buoyancy, init_3d_model, modules, prognostic_equations 

Neu:
init_slope

13/06/00 MS 2.0a B Es werden nun auch die seitlichen R√§nder f√ľr ql gesetzt. Ein Nichtsetzen der Randbedingungen f√ľhrte zu Fehlern bei der Ausgabe von pt calc_liquid_water_content
03/07/00 SR 2.0b C F√ľr eine Reihe von  Unterprogrammen, die innerhalb von prognostic_equation aufgerufen werden und an die Pointer-Variablen √ľbergeben werden, sind jetzt explizite Interfaces vereinbart (neues Modul pointer_interfaces). Auch im Vereinbarungsteil der entsprechenden Unterprogramme sind diese √œbergabeparameter jetzt aus Konsistenzgr√ľnden (und weil es sonst Laufzeitfehler gibt) als Pointer-Variablen deklariert. Mit dieser √„nderung ist der gro√Ÿe Speicher- und CPU-Zeit-Bedarf beim √œbersetzen von prognostic_equations wieder auf ein ertr√§gliches Ma√Ÿ reduziert. 
Im Rahmen dieser √„nderungen wurde auch an diffusion_e ein Hilfsfeld weniger √ľbergeben. 
Aus nicht vollst√§ndig gek√§rten Gr√ľnden haben sich die Zahlen in der Testergebnis-Datei minimal ge√§ndert (und zwar die Spalte mit der Gesamtdivergenz nach Aufruf des Druckl√∂sers).
advec_s_bc, buoyancy, diffusion_e, diffusion_s, diffusion_u, diffusion_v, diffusion_w, disturb_field, modules, production_e, prognostic_equations
04/07/00 SR 2.0b B Diriclet-Randbedingungen f√ľr Temperatur und Fl√ľssigwassergehalt werden gesetzt. Dies w√§re bereits nach Einf√ľhrung der Pointer zwingend notwendig gewesen. Rechnungen mit vorgegebener Oberfl√§chentemperatur schlugen deshalb bislang fehl. boundary_conds
07/09/00 MS 2.0b B Die virtuelle potenielle Temperatur wird nun auch f√ľr k=nzb und k=nzt+1 berechnet, um die Randwerte auch f√ľr Ausgabezwecke bereichtzustellen.  compute_vpt
28/12/00 SR 2.1 C/N PALM erlaubt jetzt den Einsatz optionaler Software-Pakete, die zwar zum Modell geh√∂ren, standardm√§√Ÿig aber nicht mit √ľbersetzt werden, um so unter anderem Kompilationszeit einzusparen. Siehe neues Kapitel 3.7 in der Modelldokumentation. 

Einsatz der dvrp-Software wurde vollst√§ndig √ľberarbeitet. Sie ist nun als Software-Paket optional im Modell einsetzbar. Siehe √ľberarbeitetes Kapitel 4.5.6 in der Modelldokumentation. 

Die Prognose der Partikelverlagerung ist ebenfalls in ein optionales Software-Paket ausgelagert.

advec_particles, header, init_3d_model, init_dvrp, init_particles, init_pegrid, modules, parin, parles, plot_dvrp, prognostic_equations, read_var_list, write_var_list 

Neu:
package_parin

28/12/00 SR 2.1 B MPI_FINALIZE ans Ende des Hauptprogramms geschoben (stand vorher vor cpu_auswertung, wo noch ein Barrier-Aufruf steht). comm2d war zu Beginn undefiniert und wird nun erst einmal gleich MPI_COMM_WORLD gesetzt. parles
02/01/01 SR 2.1a C Offene Dateien werden sobald m√∂glich geschlossen. Wiederer√∂ffnung bestimmter Dateien mit POSITION='APPEND' m√∂glich. In diesem Zusammenhang wurde das Unterprogramm close_files in close_file umbenannt und hat nun 1 Argument. 
Anpassung der Diffusionsparametrisierung im 1D-Modell an das 3D-Modell.
advec_particles, check_cpu_time, check_open, cpu_auswertung, init_1d_model, init_3d_model, init_particles, modules.f90, parin, parles, plot_2d, write_3d_binary 

Neu:
close_file 

Eliminiert:
close_files

02/01/01 SR 2.1a B Beseitigung kleiner Fehler. check_cpu_time, diffusion_e, diffusivities, long_filter, production_e
05/01/01 SR 2.1b N Neues Software-Paket zur Berechnung von Spektren im Ortsraum check_open, header, init_3d_model, leap_frog, modules, package_parin, parles, read_var_list, write_3d_binary, write_var_list 

Neu:
calc_spectra, plot_spectra

25/01/01 SR 2.1c C/N Druckl√∂ser kann jetzt die Singleton-FFT benutzen. Modul singleton wurde deshalb von calc_spectra.f90 nach modules.f90 geschoben (neuer Initialisierungsparameter fft_method). 
Fl√ľsse k√∂nnen jetzt bei k=1 unabh√§ngig von der Verwendung einer Prandtl-Schicht vorgegeben werden (neuer Initialisierungsparameter use_surface_fluxes). 
Modul test_variables wurde aus allen Programmteilen entfernt. Hauptprogramm wurde von parles nach palm umbenannt. 
Modell kann auf DEC-Workstations eingesetzt werden (-D dec). 
Zusätzliche Zeitmessungen mit dvrp-Software. Schreiben von Partikel-Informationen ist optional (neuer Parameter write_particle_informations).
advec_particles, calc_spectra, check_parameters, cpu_zeitmessung, diffusion_s, header, init_3d_model, init_dvrp, init_particles, leap_frog, modules, package_parin, parin, poisfft, read_var_list, swap_timelevel, write_var_list 

Neu:
palm 

Eliminiert:
parles, module_test

25/01/01 SR 2.1c B Fehler beim √∂ffnen von Unit 23 beseitigt. Modul interface fehlte in disturb_field. 
Wertebereich von theta* in Prandtl-Fluxes eingeschränkt, weil sonst auf einigen Rechnern overflow droht, wenn u*=0.
check_open, disturb_field, prandtl_fluxes
30/01/01 SR 2.1d N/C Prognose eines passiven Skalars ist möglich (anstatt Feuchte), dazu neuer Initialisierungsparameter passive_scalar

Falls kein Pfad f√ľr ftpcopy existiert, werden in check_open die Dateinamen f√ľr AVS-Koordinaten- und Datendatei auf "unknown" gesetzt.

asselin_filter, boundary_conds, check_open, check_parameters, flow_statistics, header, init_1d_model, init_3d_model, modules, parin, plot_2d, plot_dvrp, prandtl_fluxes, prognostic_equations, read_var_list, swap_timelevel, write_3d_binary, write_var_list
30/01/01 SR 2.1d B String-Ausgabe f√ľr use_surface_fluxes korrigiert. Version vom 25/01/01 brach deswegen bei Fortsetzungsl√§ufen ab. 
xy-Plotausgabe von q und ql korrigiert.
plot_2d, write_var_list
30/03/01 SR 2.2 N/C S√§mtliche 3D-Felder der Zeitebene t+dt wurden entfernt. Ebenso die Arbeitsfelder work und work1. Hilfsfelder wurden soweit m√∂glich eingespart und werden grunds√§tzlich lokal allokiert. Dadurch konnten eine Reihe von √œbergabeparametern gestrichen werden (z.B. beim Aufruf von pres, advec_s_bc, poisfft, production_e, sor sowie den Spline-Unterprogrammen). Der Long-Filter wurde in das Upstream-Spline-Verfahren integriert. Da der Zeitfilter jetzt direkt in der prognostischen Gleichung angewendet wird (bisher geschah dies erst nach Aufruf des Druckl√∂sers), haben sich die Zahlen in der Testergebnis-Datei ge√§ndert. Die mittleren Profile sind aber unbeeinflusst. 

Eine weitere kleinere √„nderung der Testergebnis-Daten wird dadurch verursacht, dass jetzt der Parameter adjust_mixing_length defaultm√§√Ÿig .FALSE. ist und der Mischungsweg grunds√§tzlich zus√§tzlich auf  0.7 * zu   begrenzt wird. Falls adjust_mixing_length=T, wird jetzt der Mischungsweg in diffusivities analog zu diffusion_e an allen Gitterpunkten modifiziert. Achtung: Diese √„nderungen k√∂nnen bei anderen Simulationen t.w. erhebliche Auswirkungen auf die oberfl√§chennahen Ergebnisse haben. 

Kleinere √„nderungen: In init_cloud_physics wird der Wert der Variablen surface_pressure in hPa belassen (bisher Umwandlung in Pa). 
Die Namelist-Namen der Software-Pakete sind  t.w. ge√§ndert. 
Unit 14 (bin√§re Ausgabe f√ľr Fortsetzungsl√§ufe) wird jetzt im Hauptprogramm geschlossen, damit auf diese Datei noch benutzergesteuerte Ausgaben erfolgen k√∂nnen. 
Laufparameter werden in keinem Fall mehr auf Unit 14 ausgegeben bzw. von Unit 13 gelesen, d.h. sie gelten jetzt tats√§chlich nur f√ľr den jeweils aktuellen Lauf. 
Partikeladvektion funktioniert jetzt auch zusammen mit der Galilei-Transformation. 

Restliche deutsche Variablen-, Unterprogramm- und Modulnamen wurden ins Englische √ľbersetzt. Davon sind fast alle Programmteile betroffen (nicht in rechter Spalte aufgef√ľhrt).

advec_particles, advec_s_bc, advec_s_ups, advec_u_ups, advec_v_ups, advec_w_ups, boundary_conds, calc_spectra, check_parameters, diffusion_e, init_3d_model, init_cloud_physics, init_pegrid, init_rankine, leap_frog, modules, package_parin, palm, parin, poisfft, pres, production_e, prognostic_equations, read_var_list, sor, swap_timelevel, transpose_*, write_3d_binary, write_var_list 

Neu:
cpu_log, cpu_statistics 

Eliminiert:
asselin_filter, cpu_auswertung, cpu_zeitmessung, long_filter

12/07/01 SR 2.2a N/C Defaultwert der unteren Randbedingung f√ľr die TKE ist ab sofort bc_e_b = 'neumann'
Partikelquelle ist √ľber Paketparameter steuerbar. Partikeleigenschaften sind mittels benutzereigener Software steuerbar. Der dvrp-Ausgabeweg ist jetzt ebenfalls √ľber Paketparameter steuerbar. 
Allen Modulen (bis auf singleton) wurde eine SAVE-Anweisung hinzugef√ľgt, damit die durch sie vereinbarten Variablen w√§hrend der Rechnung in keinem Fall undefiniert werden k√∂nnen (diese Gefahr besteht z.B. auf SGI-Origin-Maschinen).
advec_particles, header, init_dvrp, init_particles, modules.f90, package_parin, plot_dvrp, user_interface
12/07/01 SR 2.2a B Verschiebung der OPEN-Anweisung f√ľr Unit 33 (AVS-FLD-Datei). check_open
20/07/01 SR 2.3 N Einbau des Mehrgitterverfahrens zur L√∂sung der Poisson-Gleichung. check_parameters, exchange_horiz, header, init_grid, init_pegrid, modules, parin, pres, run_control 

Neu:
poismg

21/08/01 SR 2.3a N/C Erg√§nzung der dvrp-Software: Partikel k√∂nnen mit Schw√§nzen versehen werden. Die Ausgabe der dvrp-Plotdaten durch einen separaten PE ist m√∂glich (mrun-Option "-p dvrp_graphics+1PE"). Der Kommunikator MPI_COMM_WORLD  wurde deshalb durch  comm_palm  ersetzt. 

Reduzierung des Mischungsweges im Wandbereich kann wahlweise abgeschaltet werden. F√ľr km und kh wird jetzt eine Neumann-Randbedingung am unteren Rand verwendet. Die TKE-Energieproduktionsberechnung durch Scherung des Grundstroms am unteren Rand wurde verbessert. 

Generelle Verwendung der bodennahen vertikalen Impulsfl√ľsse in den Diffusionstermen von u und v, immer wenn diese als Randbedingung vorgegeben sind (gem√§√Ÿ use_surface_fluxes, - bisher wurde dies nur bei eingeschalteter Prandtl-Schicht gemacht). Schubspannungen k√∂nnen jetzt als Randbedingungen vorgegeben werden (allerdings unter zuhilfenahme benutzereigener Software). 

Zus√§tzliche Schnittstellen f√ľr benutzereigene Software in allen Tendenztermen, in flow_statistics, und f√ľr Partikel- bzw. dvrp-Programmteile (Bestimmung von Partikeleigenschaften, Festlegung von Farbtabellen). 

Achtung: Diese √„nderungen k√∂nnen bei Simulationen mit mittlerem Wind durch die √„nderungen im bereich des unteren Randes t.w. erhebliche Auswirkungen auf die oberfl√§chennahen Ergebnisse haben.

advec_particles, diffusion_e, diffusion_u, diffusion_v, diffusivities, flow_statistics, header, init_dvrp, init_particles, init_pegrid, leap_frog, modules, package_parin, palm, parin, plot_dvrp, production_e, prognostic_equations, read_var_list, swap_timelevel, user_interface, write_var_list
21/08/01 SR 2.3a B Falsche Positionierung von MPI_ALLREDUCE in check_cpu_time korrigiert. check_cpu_time
04/09/01 SR 2.3b N Zus√§tzliche Profilausgaben f√ľr Energieproduktionsterme m√∂glich. St√∂rdruck wird bei der FFT-Methode nicht mehr durch Aufsummieren der Werte zu den einzelnen Zeitschritten ermittelt. check_parameters, flow_statistics, modules, pres
04/09/01 SR 2.3b B Austausch der Geisterr√§nder f√ľr die zeitgefilterten Felder direkt nach Durchf√ľhrung der Filterung (war fehlerhaft seit Entfernen der Zeitebene t+dt (Version 2.2) und f√ľhrte zu sehr kleinen St√∂rungen an den seitlichen R√§ndern der Teilgebiete). 

Achtung: Der Inhalt der Testergebnis-Datei ändert sich durch diese Fehlerkorrektur.

prognostic_equations
09/11/01 SR 2.3c N Farbe entlang der Partikelschwänze kann sich ändern. Einschalten der Partikeladvektion bei Fortsetzungsläufen möglich. advec_particles, init_particles, modules, package_parin, plot_dvrp
16/04/02 SR 2.3d N Vorgabe von Randbedingungen f√ľr Partikeladvektion m√∂glich. Partikeldaten k√∂nnen f√ľr sp√§tere Analysen auf Datei geschrieben werden (Unit 85). PARTICLE-Datentyp enth√§lt Informationen √ľber Partikelgeschwindigkeitskomponenten und Startposition. 

Skalartransport sowie entsprechende Datenausgaben k√∂nnen durch eigene Parameter gesteuert werden (nicht mehr √ľber die Feuchteparameter). 

Im Fall von pdims(1)=1 (eindimensionales virtuelles Prozessornetz in y-Richtung) werden die zyklischen Randbedingungen in x-Richtung durch direktes Umspeichern anstatt durch  sendrecv realisiert. Ebenso werden die Transponierungen xz, yz, zx  und zy eingespart (es wird aber weiter umsortiert). 

Im Fall von 3D-Plotausgaben Aufruf von ftpcopy-Script durch batch_scp-Script ersetzt (dient der Ermittelung von Dateizyklusnummern als Information f√ľr die AVS-fld-Datei). 

Bei fehlerhafter Er√∂ffnung einer CPU_MEASURES-Datei werden erneute Er√∂ffnungsversuche durchgef√ľhrt.

advec_particles, check_open, check_parameters, exchange_horiz, flow_statistics, header, init_particles, modules, package_parin, parin, plot_2d, plot_3d, prognostic_equations, read_var_list, transpose_xz, transpose_yz, transpose_zx, transpose_zy, write_var_list
02/05/02 SR 2.3e B Wiedereinf√ľhrung der 3D-Felder f√ľr die Zeitebene t+dt, die notwendig ist, damit in den Diffusionstermen bei Leapfrog-Zeitschritten mit der korrekten Zeitebene t-dt gearbeitet wird. Seit Version 2.2 wurde f√§lschlicherweise die Zeitebene t+dt verwendet. Asselin-Filter ist jetzt wieder eigenst√§ndiges Unterprogramm.
Die Wiedereinf√ľhrung dieser dritten Zeitebene ist auch Voraussetzung f√ľr die in der n√§chsten Version geplante skalare Optimierung. 

Fehler bei Berechnung von Ausgabezeitpunkten eliminiert (trat auf, wenn bei Fortsetzungsl√§ufen von Ausgabeabst√§nden 0.0 auf von Null verschiedene Werte gewechselt werden sollte). 

Achtung: Der Inhalt der Testergebnis-Datei ändert sich durch diese Fehlerkorrektur.

init_3d_model, leap_frog, modules, prognostic_equations, swap_timelevel 

Neu:
asselin_filter

02/05/02 SR 2.3e N/C Kleinere Anpassungen an IBM-Regatta-Systeme in check_parameters, cpu_log. 

Modul singleton in eigenst√§ndige Datei √ľberf√ľhrt. 

Global_min_max arbeitet mit REAL*4, um Komplikationen auf 32-bit-Rechnern zu vermeiden (Datentyp MPI_2REAL m√ľsste dort sonst auf MPI_2DOUBLE_PRECISION ge√§ndert werden).
Horizontale Geschwindigkeitskomponenten f√ľr Partikeladvektion werden exakt zwischen den jeweils benachbarten horizontalen Gitterfl√§chen interpoliert.

advec_particles, check_parameters, cpu_log, global_min_max, modules 

Neu:
singleton

11/06/02 SR 2.4 N/C Optimierung und Anpassungen f√ľr einzelne Knoten der IBM-Regatta-Systeme.

Skalare (Cache) Optimierung: Tendenzterme werden innerhalb einer gro√Ÿen (i,j)-Schleife berechnet, die nun jeweils die gesamte prognostische Gleichung umfasst. Schleifenindices i und j werden an die Tendenzunterprogramme als Argumente √ľbergeben. Die Tendenzunterprogramme sind als Module geschrieben, die mittels Technik der √ľberladenen Funktionen auch ohne diese Indices i und j aufgerufen werden k√∂nnen, und dann wie in fr√ľheren Versionen funktionieren (d.h. in ihnen laufen die Schleifen √ľber alle 3 Dimensionen). Solche Teile der prognostischen Gleichungen, die globale Kommunikation beinhalten, sind vor die (i,j)-Schleife gezogen. Falls ausschlie√Ÿlich mit Piascek-Williams-Advektion gearbeitet wird, werden alle prognostischen Gleichungen in einer einzigen (i,j)-Schleife gerechnet.

Kommunikationsoptimierung: Zusätzliche Implementierung einer 1D-Gebietszerlegung in x-Richtung. Mit dem neuen direkten Drucklöser (poisfft_hybrid) werden dadurch 4 von 6 notwendigen Transpositionen eingespart. Die 1D-Zerlegung wird automatisch bei Verwendung dieses neuen Lösers aktiviert. Austausch von Geisterrändern in y-Richtung wird bei dieser 1D-Zerlegung durch einfache zyklische Randbedingungen ersetzt.

Kleinere Anpassungen: String-Vergleiche erfordern beim IBM-Compiler in der Regel den Einsatz der TRIM-Funktion, um √ľbersch√ľssige Blanks am Ende des Strings abzuschneiden (sonst liefert der Vergleich .F.).
Lokal allokierte Felder t.w. in sogenannte automatische Felder umgewandelt (g√ľnstiger, um Gefahr von eventuellen Speicherlecks zu vermeiden). Einige Feldoperationen wurden wegen schlechter Performance auf der IBM entfernt (s. pres).

Achtung: Die √œbersetzung des Modells erfordert aufgrund der Vielzahl nunmehr eingesetzter Module die Verwendung des make-Mechanismus, um die korrekte Reihenfolge bei der √œbersetzung der Programmteile zu gew√§hrleisten. Die Beschreibung dieser Abh√§ngigkeiten erfolgt in der Datei Makefile, die zusammen mit den Programmdateien abgelegt ist und ebenfalls unter RCS-Verwaltung steht.

Im User-Interface ist das Unterprogramm user_actions nun ebenfalls als Modul geschrieben und bedient sich der Methode der √ľberladenen Funktionen. Existierende benutzereigene Software muss entsprechend angepasst werden.

advec_s_pw, advec_s_up, advec_u_pw, advec_u_up, advec_v_pw, advec_v_up, advec_w_pw, advec_w_up, buoyancy, calc_precipitation, calc_radiation, calc_spectra, check_parameters, coriolis, diffusion_e, diffusion_s, diffusion_u, diffusion_v, diffusion_w, diffusivities, exchange_horiz, exchange_horiz_2d, global_min_max, header, impact_of_latent_heat, init_pegrid, leap_frog, modules, plot_2d, plot_3d, plot_spectra, poisfft, pres, production_e, prognostic_equations, user_interface,

Neu:
fft_for_1d_decomp, Makefile, poisfft_hybrid, 

11/06/02 SR 2.4 B Fehlende Variablenvereinbarung in advec_particles ergänzt. advec_particles
12/09/02 SR 2.4a N Partikel können mit Trägheit versehen werden und einen Dichteunterschied zum umgebenden Fluid haben. Zur Steuerung dient der neue Paketparameter density_ratio. advec_particles, header, init_particles, modules, package_parin
12/09/02 SR 2.4a B Fehler in Berechnung der Anfangsprofile von pt und q entfernt (obere Feldgrenze von 10 konnte in pt_vertical_gradient_level_ind √ľberschritten werden). Fehler f√ľhrte unter Umst√§nden zu "segmentation fault" bei Verwendung sehr gro√Ÿer Gitterpunktzahlen in z-Richtung.

Fehler in Berechnung der v-Komponente am unteren Rand entfernt, die f√ľr den Scherungsproduktionsterm der TKE verwendet wird (Variable v_0). F√§lschlicherweise wurde zur Berechnung u(k=1) statt v(k=1) verwendet.

check_parameters, modules, production_e
19/12/02 SR 2.5 N Restart times can be set by user with new runtime (d3par) parameters restart_time and dt_restart. Run description header is written on file CONTINUE_RUN. Output of cpu statistics changed (PE results are now collected on PE0 in order to calculate mean cpu statistics). Output of warnings in the job protocol in case of negative measured cpu-times. Remaining cpu-time is also evaluated on IBM-Regatta. cpu_log, cpu_statistics, check_parameters,header, leap_frog, local_tremain, local_tremain_ini, modules,  palm, 

new:
check_for_restart

deleted:
check_cpu_time

 
 
 
C Unit 14 (BINOUT) must be opened using a special process id string, because on IBM, the PE rank differs between communicators MPI_COMM_WORLD and comm_2d. This leads to a mismatch between filename-PE-number and PE-number written on file. As the consequence, these files could not be read by restart-jobs. Reading of array hom moved from init_3d_model to read_var_list (binary version number incremented to 2.2).

Further optimization for IBM-Regatta-systems:
Additional optimization of the hybrid-solver for multinode usage (overlapping of communication and computation).

Further cache optimization by using strides and joining loops (hybrid_solver, pres and timestep). Joining of MPI_ALLREDUCE calls in timestep.

In case of 1d-decomposition along x only a part of the integral divisor conditions is checked, on IBM hosts a 1d-decomposition along x is the default, a switch from Poisson-FFT-solver to the hybrid-solver is made in case of 1d-decomposition along x, the hybrid-solver does not force a 1d-decomposition any more.

Array notation changed to do-loop constructs due to better performance.

Cyclic boundary conditions along y used instead of sendrecv in case of a 1d-decomposition along x. SENDRECV replaced by nonblocking routines ISEND and IRECV in exchange_horiz.f90.

Speed optimization by removing MINVAL/MAXVAL calls and by handling
the "abs" case in a different way than the min/max cases (routine global_min_max).

asselin_filter, check_open, exchange_horiz, global_min_max, init_3d_model, init_pegrid, parin, poisfft_hybrid, read_var_list, timestep, write_3d_binary, write_var_list



B Correction of mixing length term (l(k)/ll(k)). The condition kh=3*km in the unstable case is now also exactly met in the wall adjustment region. Factor 0.7 in wall adjustment part replaced by variable wall_adjustment_factor, which is set to 1.8 in modules.f90. The factor 0.7 was the possible reason for 2-delta-x-waves, which were observed since version 2.1d.

Calculation of deformation tensor re-designed (production_e).

STOP statements replaced by call of new subroutine local_stop, where MPI_FINALIZE is called before STOP in case of a parallel environment.

ISO2D parameter dp set to true for ibm hosts.

tend=p added in pres after calling sor method.

advec_s_bc, buoyancy, check_open, check_parameters, close_file, coriolis, diffusion_e, diffusivities, fft_for_1d_decomp, flow_statistics, init_3d_model, init_dvrp, init_grid, init_particles, init_pegrid, parin, plot_2d, poisfft, pres, read_var_list, user_interface, 

new:
local_stop, production_e

03/03/03 SR 2.5a N Particle velocities are also stored in array particles in case of zero density ratio. Steering of variables by dvrp browser included. advec_particles, init_dvrp, modules,



C AVS data format changed from float to xdr_float (needed on linux machines
due to the little/big endian problem). Updates for new version of dvrp software (e.g. using module dvrp is now mandatory).
close_file, init_dvrp, leap_frog, plot_dvrp



B Error in particle inertia part removed (exp_arg must not contain the timestep).
Error in calculation of the vertical flux of resolved scale energy (profile 57) removed. Displacement for integers in mpi_particle_type reduced from 16 to 8 on ibm.
advec_particles, flow_statistics, init_particles
12/03/03 SR 2.6 N Version optimized for NEC-SX6 parallel-vector machines. 

There are two main changes. A new vectorizable routine prognostic_equations_vec is added, where cache optimizations are undone. The tendency subroutines called by prognostic_equations_vec contain the full 3d-loops (compared with the cache-optimized versions where only loops over k are carried out). Additionally, a new pressure solver poisfft_hybrid_vec is added, which requires a 1d-domain-decomposition (like poisfft_hybrid). In this solver, a very fast FFT from Clive Temperton (ECMWF) is  optionally available by choosing fft_method = 'temperton-algorithm'

Additional changes in flow_statistics to allow better vectorization.

Small changes in routines diffusivities and pres which caused run time errors on IBM and NEC due to compiler problems.

Reading of environment variable tasks_per_node moved from routine poisfft_hybrid to routine parin.

Changes in makefile: modules and user_interface now depend on the f90 files, dependency of singleton added, LDFLAGS moved to the end of PROG rule, temperton_fft added.

advec_s_pw, advec_s_up, advec_u_pw, advec_u_up, advec_v_pw, advec_v_up, advec_w_pw, advec_w_up, buoyancy, calc_precipitation, calc_radiation, check_parameters, coriolis, cpu_log, diffusion_e, diffusion_s, diffusion_u, diffusion_v, diffusion_w, diffusivities, fft_for_1d_decomp, header, impact_of_latent_heat, init_pegrid, leap_frog, local_tremain, modules, parin, poisfft, poisfft_hybrid, pres, production_e, prognostic_equations, user_interface

new:
temperton_fft




B tend=p added in routine pres after calling sor method. pres
14/03/03 SR 2.6a N Additional vector optimization for NEC-SX6.

Optional system-specific random number generator available (for equal distributed numbers).

Define strings in all relevant subroutines changed from 

     #if defined( define_string )

to

     #if defined( __define_string )

Caution: The relevant subroutines are NOT listed in the right column!

asselin_filter, check_parameters, diffusion_e, diffusivities, disturb_field, global_min_max, header, local_tremain, modules, parin, poisfft_hybrid, pres, read_var_list, write_var_list



B Error in particle boundary condition removed (velocity must be inverted in case of reflection) advec_particles
16/04/03 SR 2.6b N Time series output of Monin Obukhov length.

Temperton fft can now be used for all hosts and every domain decomposition. Abort in case of Temperton fft, if number of gridpoints along x and/or y contain illegal factors. 

Index values for the extrema found in global_min_max are limited to the range 0..nx, 0..ny (on IBM machines -1 and nx+1/ny+1 occured which produced different RUN_CONTROL output compared to other machines).

Output format of iteration count in routine run_control enlarged.

check_parameters, fft_for_1d_decomp, global_min_max, init_pegrid, modules, plot_ts, poisfft, run_control, temperton_fft



B Additional checks of variables hybrid_solver and host (hybrid_solver must not be used for a 2d-domain-decomposition, host should be set by setting environment variable localhost).

Header output for mixing length limitations revised.

check_parameters, header
09/05/03 SR 2.7 N Version running on Linux Clusters using MPICH and Intel FORTRAN compiler (ifc)

So far, only absolutely neccessary changes have been done (possible optimizations will follow in a later version):

New time measurements for Linux (ifc) environment added.

Some parameters in MPI calls had to be modified (arrays had to be replaced by the first element of the regarding array) in order to fulfill f90 type and rank requirements, since on the MUK-cluster a FORTRAN90-version of MPI is used. This also required to replace "mpif.h" by using a special module (named mpi).  I did not find out how character strings can be send with MPI. Therefore, these strings are transformed to integers, before they are send, and transformed back to characters on the receiving PE.

On the MUK-cluster, only PE0 is able to read the values of environment variables. Therefore, these values are communicated via broadcast to the other PEs.

1d-decomposition is set as the default on Linux Clusters. Character strings have to be transformed to integer

Smaller changes:

Batch_scp paths for IBM, NEC and Linux added

check_open, cpu_log, flow_statistics, init_pegrid, local_getenv, modules, palm, poisfft_hybrid



B Measurements on IBM are now using function irtc, which allows correct measurements for jobs running over the 24:00 timeline. cpu_log, local_tremain, local_tremain_ini
01/08/03 SR 2.7a B Check that the number processors is also an integral divisor of the number  of gridpoints along y in case of a 1d-decomposition along x.

Error concerning the multinode-version of poisfft_hybrid removed. It was caused by the Linux changes of version 2.7 and led to program abort due to MPI errors in MPI_ALLTOALL.

No abort on t3e-systems if system specific routines are used.

Array sums is initialized in init_3d_model before the first call of pres. Call of init_cloud_physics moved before init_particles.

fft_1dd_init, init_3d_model, init_pegrid, poisfft_hybrid



N Interactive steering of dvrp-graphic features by dvrp-browser plugin extended (position of slicers, threshold values of isosurfaces) . New dvrp_graphics package parameter slicer_range_limits_dvrp. init_dvrp, modules, package_parin, plot_dvrp
29/10/03 SR 2.7b N In the multigrid method, on a defined level, data are gathered on PE0 and further calculations are carried out only on this PE. New d3par parameter mg_switch_to_pe0_level.

Particle groups implemented. New data type particle_groups_type and data type particle_type modified. Version numbers are output on the particle files. New particles_par parameter diameter. Parameter name uniform_psize changed to dvrp_psize. Parameters density_ratio is now an array.

advec_particles, check_open, exchange_horiz, header, init_particles, init_pegrid, modules, package_parin, parin, poismg



C Random number generator from numerical recipes is now rewritten as a module. In restart runs the seed of this generator is set to the last values of the previous run in order to keep the sequence of the random numbers. New binary version 2.2.

Buffer is flushed for file RUN_CONTROL immediately after every output on IBM and Linux cluster.

advec_particles, disturb_field, init_3d_model, init_particles, random_function, random_gauss, run_control, write_3d_binary



E In routine buoyance, horizontal mean temperature is now taken from array hom instead of array sums (otherwise inconsistence in case of using more than one statistical region).

Information about the time of the last timestep change added to the restart file.

System call of batch_scp on IBM corrected.

buoyancy, check_open, read_var_list, write_var_list
28/01/04 SR 2.8 N Runge-Kutta schemes (2nd and 3rd order) for time integration implemented. The third order scheme is the new default. It allows the timestep to be 0.9*CFL, which is much larger than for the former default leapfrog scheme.  The intermediate steps, which are part of these schemes, are realized by an additional loop within routine time_integration (currently leap_frog, but this routine will be renamed soon). Steering variables at and bt have been replaced by array sct. , which is particularly used in the prognostic equations. Values of this steering array are calculated within the new routine timestep_scheme_steering.

When using Runge-Kutta schemes,  the timestep increment is freely allowed to adjust after each timestep (the older schemes are using some restrictions, see routine timestep). Also, routine run_control is not automatically called in case of timestep changes, when Runge-Kutta schemes are switched on.

The old leapfrog scheme is still implemented and should produce the same results as in the previous version(s)!

Depending on the user experience, further adjustments to the Runge-Kutta schemes may be necessary within the next minor versions. 

Attention:
The content of the testresult-file (run control output) has significantly changed due to the new default timestep scheme!

check_parameters, header, init_3d_model, init_rankine, leap_frog, modules, prognostic_equations, read_var_list, swap_timelevel, timestep, write_var_list

new:
timestep_scheme_steering




C Output of particle infos in subroutine allocate_prt_memory on demand only.

Type log (used for cpu time measurements) changed to logpoint due to name conflict with FORTRAN intrinsic log.

advec_particles, cpu_log, 



E In case of opening unit 80, a barrier is set only for the first call of check_open (from routine init_particles), in order to avoid the possibility of hanging jobs, which may occur if unit 80 is opened within routine allocate_prt_memory. check_open
30/01/04 SR 2.8a N In order to prepare the code for the simulation of flow around buildings (to be realized in one of the next major versions), the lowerindex nzb has been replaced by a two-dimensional array nzb_2d in many of the three-dimensional loops. So far, all elements of this array are set to nzb.

This change should not effect simulation results and cpu time in any way.

advec_s_pw, advec_s_up, advec_u_pw, advec_u_up, advec_v_pw, advec_v_up, advec_w_pw, advec_w_up, asselin_filter, buoyancy, calc_liquid_water_content, calc_precipitation, calc_radiation, coriolis, diffusion_e, diffusion_s, diffusion_u, diffusion_v, diffusion_w, diffusivities, impact_of_latent_heat, init_3d_model, modules, production_e, prognostic_equations, 



C Euler step informations are removed in the run control output in case of Runge-Kutta schemes. Also, the timestep increment limitation is removed for the Runge-Kutta schemes.

Attention:
The content of the testresult-file (run control output) has changed due to this modification.

run_control, timestep



E Velocity gradients at the surface calculated in routine production_e are now limited. In case of inhomogeneous surface heating small diffusivities (km) sometimes caused very small timesteps due to this problem. production_e
30/04/04
SR
2.8b
N
The number of processors along the respective directions of the virtual processor grid must not be integral divisors of the number of gridpoints along x, y or z any more. In case that they are not integral divisors, the subdomains on the processors at the upper borders of the virtual grid will contain less gridpoints than the other subdomains (which are all of equal size). Although this causes a load imbalance, only runs with very small numbers of processors will feel a significant decrease of performance. In order to be able to carry out the transpositions, array d (and also working array tend) have the same size on all processors. This new feature is switched on by setting the inipar-parameter grid_matching = 'match', which is the default now.

Routine poisfft is now the default for solving the Poisson-equation for the case of 2d- as well as 1d-domain-decompositions and has been changed from a subroutine to a module. This routine now includes most of the optimization from the (Ketelsen) routine poisfft_hybrid and additionally allows 1d-decompositions along y.
ATTENTION: Routine poisfft_hybrid might still be faster when running PALM on more than one node of the IBM-Regatta, although the difference is not very big. The poisfft-code for a decomposition along y has still a poor performance and needs further optimization.

The spectra-package is now able to use all the available FFT-methods (selected by inipar-parameter fft_method ). Performance of the transpositions needed for calculating the spectra has been improved by using new direct transpositions from z to y and from y to x instead of two subsequent transpositions.

Runge-Kutta scheme implemented in the 1d-version (init_1d_model). Diffusion quantity arrays are allocated for previous timelevels only in case of leapfrog scheme.

This version is adapted for use on COMPAQ-DECALPHA systems (main changes: calling of SHAPE function within subroutine arguments (e.g. singleton) causes compiler errors, particle data type modified for better alignment, DATA statements from modules removed because they caused internal compiler errors).
advec_s_ups, advec_u_ups, advec_v_ups, advec_w_ups, calc_spectra, check_parameters, fft_xy, header, init_1d_model, init_3d_model, init_particles, init_pegrid, modules, parin, plot_2d, poisfft, prandtl_fluxes, pres, read_var_list, singleton, spline_x, spline_y, spline_z, time_integration, transpose, write_3d_binary, write_var_list



C
FFT routines renamed. Module fft_for_1d_decomp renamed fft_xy. Subroutine leap_frog renamed time_integration.

For all transpositions, MPI_ALLTOALL is used instead of MPI_ALLTOALLV. The names of many transposition indices have been changed.

Inipar-parameter impulse_advec renamed momentum_advec.

Reading of restart data is moved from init_3d_model to new routine read_3d_binary.

All former transpose routines collected into one file.
check_parameters, flow_statistics, header, init_1d_model, init_3d_model, init_advec, init_pegrid, palm, parin, poisfft, poisfft_hybrid, production_e, prognostic_equations, read_var_list, sline_x, spline_y, write_var_list

new:

fft_xy, read_3d_binary, time_integration, transpose

deleted:
fft_for_1d_decomp, leap_frog, transpose_xy, transpose_xz, transpose_yx, transpose_yz, transpose_zx, transpose_zy



E
Small errors in the particle-package removed, which sometimes caused aborts of restart runs on IBM.

Small error in calculating the time series profiles removed (array bounds of some arrays needed for profil-output had to be increased).

Function fft (singleton method) replaced by subroutine fftn due to problems with 64-bit mode on IBM
advec_particles, check_parameters,
26/03/05
SR
2.9
N
Non-cyclic lateral boundary conditions implemented.
Dirichlet/Neumann conditions at the inflow/outflow can be used either along x or along y. Cyclic conditions have to be used along the remaining direction. In case of non-cyclic conditions along x (y), the upper bound nxl (nyn) of array u (v) is enlarged by one gridpoint on the rightmost (northest) processor. These array index informations are passed to routine exchange_horiz and disturb_field by two additional arguments. Boundary conditions are set with new inipar-parameters bc_lr and bc_ns. A damping layer is needed at the outflow in order to damp reflections (see new parameters km_damp_max and outflow_damping_width) and an additional disturbance is imposed near the inflow (see new parameters  inflow_disturbance_begin and inflow_disturbance_end). Non-cyclic conditions require the use of the multigrid-method for solving the Poisson equation. Argument range added to routine boundary_conds (where the non-cyclic conditions are set).

Minor changes:
Output of parameter datform on &GLOBAL-Namelist (iso2d). This is set to 'big_endian' on IBM- and NEC-machines.

Number of particle groups are additionally calculated in header (formerly unknown).
advec_u_pw, advec_u_up, advec_v_pw, advec_v_up, boundary_conds, calc_liquid_water_content, check_parameters, close_file, coriolis, diffusion_u, diffusion_v, diffusion_w, diffusivities, disturb_field, exchange_horiz, header, init_3d_model, init_pegrid, init_pt_anomaly, init_rankine, modules, parin, poismg, pres, prognostic_equations, read_var_list, sor, time_integration, timestep, write_var_list



C
pres is called by default only at the last Runge-Kutta-substep, which spares a big amount of CPU-time (see also new parameter call_psolver_at_all_substeps). No pressure term in the momentum equations in case of Runge-Kutta-schemes.
Attention:
The content of the testresult-file (run control output) has changed due to this modification.

Program speedup by removing the exchange of ghost points in routines calc_liquid_water_content, diffusivities, .

Default value for residual limit increased from 1E-6 to 1E-4. Routine prolong in poismg simplified (one call of exchange_horiz spared).
calc_liquid_water_content, diffusivities, modules, poismg, pres, time_integration, timestep_scheme_steering



E
Calculation of vertical particle velocity (with inertia) corrected, exp_arg had a wrong sign.

(e)**1.5 replaced by e*SQRT(e) in init_1d_model because of wrong results on NEC machines (as a side effect, the new calculation is much faster).

Two errors concerning switch to PE0 (multigrid method) removed from init_pegrid.

Two errors in plot_2d removed: local_2d is allocated with upper bound nzt (former nzt+1 was wrong), additional barrier in case that PE0 gathers data from the other PEs, this barrier had to be set at the end of the gathering because otherwise PE0 sometimes received wrong data on tag 0.

Use of module indices removed from routine split_1dd in poisfft since it caused errors in case of nx /= ny and a 1D-decomposition along y.

Wrong re-definition of dx2 and dy2 in sor removed.
advec_particles, init_1d_model, init_pegrid, plot_2d, poisfft, sor
23/04/05
SR
2.9a
C
Additional check for incompatibilities between non-cyclic lateral boundary conditions and other schemes.

fcl_factor renamed cfl_factor.

New local array sums_ll declared in flow_statistics instead of temporarily using sums_l.

crmax (maximum number of crosses allowed in a plot of vertical profiles) increased from 20 to 100.
calc_spectra, check_parameters, flow_statistics, header, modules, parin, timestep



E
Revised calculation of output time counters regarding a possible decrease of the output time interval in case of restart runs.

Error removed in Dirichlet bottom boundary conditions for pt and q in case of Runge-Kutta schemes.

batch_scp calls (needed for determining correct filenames for AVS-fld-files) is given the remote username as an additional argument.

Default setting of outflow_damping_width corrected.

Initial horizontal velocities at the lowest grid levels in the 1d-model are set to a very small value in order to avoid wrong results and the resulting too small timesteps.

Implicit counters i renamed i9 in modules due to declaration conflicts with i in other subroutines (reported by Portland compiler).
advec_particles, boundary_conds, check_open, check_parameters, init_1d_model, modules, time_integration
18/05/05
SR
2.10
N
NetCDF support implemented.
Profiles, time series, spectra, cross-sections, 3d-volume-data and particle data can now be output in NetCDF format (file handles 101-108). This output can be switched on with the new d3par-parameter data_output_format. So far, this support is available on IBM systems at HLRN and on the NEC system at DKRZ. Output in format suitable for graphic software profil, iso2d and avs is still possible.

Output of cross sections for qv, vpt and lwp (liquid water path).
advec_particles, check_open, check_parameters, close_file, header, modules, parin, plot_1d, plot_2d, plot_3d, plot_spectra, plot_ts, read_var_list, write_var_list

new:
netcdf



C
In calc_spectra, n is assigned the number of spectra (formerly, the number of spectra was given by n-1)

Abort if poisfft_hybrid is called in a non-parallel environment.

Default values of spectra package parameters pl_spectra and spectra_direction changed.
calc_spectra, check_parameters, modules, pres, read_3d_binary, write_3d_binary



E
Missing argument in ffty (non-parallel case) added.

Error in output of particle inertia information removed.
poisfft, header
05/07/05
SR
2.11
N/C
New cloud physics code implemented, which explicitly simulates single cloud droplets, including droplet growth by condensation and collision. Using this code requires to set the new inipar-parameter cloud_droplets = .TRUE. and to use the particle package (by using the mrun-option -p particles). Coupling between droplets and the thermodynamic quantities (potential temperature and specific humidity) is regarded in the new subroutine interaction_droplets_ptq. The real number of droplets in a grid cell can steered by the new inipar-parameter initial_weighting_factor.

This code will be further extended and probably changed in the near future, so the current status is that of a test version!

In connection with this new code implementation, several changes has been made to the existing code:

The particle (droplet) size is now given as radius (previously diameter). E.g., the name of the respective package parameter diameter has changed to radius.

The current weighting_factor and radius are stored as additional particle attributes in the type structure particle_type. The MPI data type mpi_particle_type, needed for exchanging particles between subdomains, has been changed correspondingly.

gas_constant is renamed r_d, latent_heat is renamed r_v, the allowed string length of pl2d and pl3d is increased from 6 to 10.

New 3d arrays ql_1, ql_2, ql_v and ql_vp, ql is now a pointer, new pointer ql_c.

Data format for unit 85 (particle data output) is changed.
advec_particles.f90 check_open.f90, header.f90, init_3d_model.f90, init_cloud_physics.f90, init_particles.f90, modules.f90, package_parin.f90, parin.f90, plot_2d.f90, plot_3d.f90, read_var_list.f90, time_integration.f90, write_var_list.f90

new:
interaction_droplets_ptq.f90



C
Default value of call_psolver_at_all_substeps is .TRUE., because small scale waves occured with .FALSE. modules.f90



E
2d- and 3d-NetCDF-files are not opened if there is no output for them (pl2d = pl3d = ' ').

No opening of units 101 - 108 in case of missing NetCDF support.
check_open.f90, netcdf.f90
30/06/05
GS
2.11a
C
Scalars ug and vg have been changed into arrays in order to allow the specification of a geostrophic wind that depends on height (baroclinicity). The initial profiles of the the u- and v-component of the geostrophic wind are initialized by specifying the new initialization parameters ug_surface, ug_vertical_gradient, ug_vertical_gradient_level and vg_surface, vg_vertical_gradient, vg_vertical_gradient_level, respectively.

The former initialization parameters ug and vg have been removed.
boundary_conds, check_parameters, coriolis, header, init_1d_model, modules, parin, prognostic_equations, read_var_list, spline_z, write_var_list



E
Error in the calculation of u_0 and v_0 in production_e.f90 removed.

Attention:
The content of the testresult-file (run control output) has changed due to this modification.
production_e
20/10/05 SR 2.11b N/C Collision process for droplet growth completed.

The droplet code is still under development and requires further speed optimization!

Number of particles really used is additionally output on the netcdf particle data file.
advec_particles, modules, netcdf
      E The last PE in a row is not allowed to have more grid points than the other PES (only less). Jobs crashed in these cases that the last PE has more grid points. Therefore, the number of gridpoints along x (and y) must now meet a special condition which is checked in case that grid_matching = 'match' is used.

Error removed in calculating y axis data for yz cross sections. Error in output of netcdf yz-cross-sections removed.

2*r replaced by r in the exponential term of the particle momentum equation.
Error in output of yz-slice information removed.
advec_particles, header, init_pegrid, netcdf, plot_2d
06/12/05 SR 2.11c N Output of ql profile is allowed in case of using cloud droplets. check_parameters, flow_statistics, modules
      E The horizontally averaged pt profile is calculated within buoyancy only in case of the first respective intermediate timestep. This is done in order to spare communication time and to produce identical model results with jobs which are calling flow_statistics at different time intervals.

Error in netcdf variable declaration for statistic_regions /= 0 removed.
buoyancy, netcdf
23/02/06 SR 3.0 N Version for simulating flows over topography (buildings, mountains, etc.)

Besides the implementation of topography, this version contains several other improvements. The most important ones are listed here:
  • The cloud droplet code has been optimized (it now runs about 100 times faster than before).
  • Output of time-averaged 2d/3d data (sections / volume data) is possible.
  • Output of 2d-section data averaged along the direction normal to this section can be done.
  • The user-interface has been extended in order to allow 2d-/3d-output of user-defined quantities.
The kind of topography to be used is controlled by the new initialization parameter topography. Allowed values are 'flat', 'single_building', and 'read_from_file'. The user can define his own special topography setting within the new user-interface routine user_init_grid. For the single building case, the building (rectangular) size and position can be controlled with parameters building_height, building_length_x, building_length_y, building_wall_left, and building_wall_south.

The topography code is already optimized (vectorized) for use on the NEC-SX6.

Output of time-averaged 2d-/3d-data can be enabled by appending the string '_av' to the respective output quantities given with the new runtime-parameter data_output (see further below for the (name)changes in the parameters for data output). Example: if a time-average of the vertical cross-section of potential temperature is required, the user has to set data_output = 'pt_xy_av'. In case of NetCDF-output, the time-averaged data is output to additional files (seperate from the non-averaged output). The length of the averaging interval and the temporal distance of the single instantaneous sections (volumes) entering into this averaging is controlled by the new runtime-parameters averaging_intervaland dt_averaging_input.

An average normal to the direction of a cross-section can be output by setting the respective cross section index to -1 (example: section_xz = -1).

New user-interface routines allowing the output of user-defined quantities are user_check_data_output, user_define_netcdf_grid, user_data_output_2d, user_data_output_3d, and user_3d_data_averaging. These quantities can be calculated using the new CASE-Entry 'after integration' in the user-interface routine user_actions.

Optimization of the cloud droplet code is mainly done by re-sorting all particles after every timestep in a way that now all particles which belong to the same grid-box (k,j,i) are stored consecutively (wthout any strides) in memory.

Code adaptation for the IBM at the department of atmospheric sciences, Yonsei university (ibmy). Affected subroutines: fft_xy, netcdf,

Further new features:
  • Use of particle tails now requires setting of the new package-parameter use_particle_tails. The new parameter skip_particles_for_tail allows only every skip_particle_for_tail'th parameter having a tail.
  • Initialization-parameter e_min can be used to set a lower limit for the subgrid-scale TKE (i.e. to guarantee a lower limit for the flow's Reynolds-number).
  • A conservation of the volume flow (through the complete xz- and yz-sections  of the total domain) can be enforced by the new initialization-parameter conserve_volume_flow.
  • Data output can be skipped for a given time interval from simulation start (t=0) using new runtime-parameters skip_time_dopr, skip_time_dosp, skip_time_do2d_xy, skip_time_do2d_xz, skip_time_do2d_yz, and skip_time_do3d.
  • By default, NetCDF output is now using 64-bit offset format (large file support) on most machines. The user can switch to the 32-bit offset format with the new runtime-parameter netcdf_64bit. Units 111:113  and 116 are opened for NetCDF output of time-averaged 2d-3d data.
  • Data logging routines are added (see file data_log.f90), which can be used for debugging purposes. Output is done on unit 20 (local file name/directory name DATA_LOG).
  • New routine exchange_horiz_2d_int for  ghostpoint exchange of 2d-integer arrays. Routine exchange_horiz_2d is extended for non-cyclic boundary conditions in the non-parallel case.
  • Two different methods for calculating the mixing length and the dissipation can be used in the 1d-model. These are steered by the new initializing-parameters dissipation_1d and mixing_length_1d.
advec_particles, advec_s_pw, advec_s_up, advec_u_pw, advec_u_up, advec_v_pw, advec_v_up, advec_w_pw, advec_w_up, boundary_conds, buoyancy, check_open, check_parameters, close_file, coriolis, diffusion_e, diffusion_s, diffusion_u, diffusion_v, diffusion_w, diffusivities, disturb_field, disturb_heatflux, exchange_horiz_2d, exchange_horiz, flow_statistics, header, init_1d_model, init_3d_model, init_grid, init_particles, init_pegrid, modules, netcdf, package_parin, parin, poisfft, prandtl_fluxes, pres, production_e, prognostic_equations, read_3d_binary, read_var_list, run_control, time_integration, user_interface, write_3d_binary, write_var_list

new:
average_3d_data, data_log, sum_up_3d_data

renamed:
plot_dvrp -> data_output_dvrp
plot_spectra -> data_output_spectra
plot_ts -> data_output_tseries
plot_1d -> data_output_profiles
plot_2d -> data_output_2d
plot_3d -> data_output_3d
      C In parallel mode, all PEs are opening the same copy of the NAMELIST-parameter file PARIN, i.e. from now on, the file attribute "npe" must be removed from the corresponding file connection statements in the mrun configuration file (.mrun.config).

A large number of parameter names and local filenames have been changed (see list below). Most of them are parameters concerning data output (do). The former denotations "plot" and "pl" have been changed to "data_output" and "do", respectively. In addition to these changes, the names of many internal PALM parameters and variables have also changed.

Restart runs now require setting of the initialization-parameter initializing_actions = 'read_restart_data'.

Concerning particles, the parameters psl, psr, pdx, etc., which are controlling the position of the particle source and the number of particles within this source, are now 1d-arrays. This allows to define different particle sources for different particle groups.

The number of particle groups has to be set by the user (no automatic calculation from the value of parameter density_ratio any more). Variable idum in type particle_type has been renamed tail_id.

Output for units 15 and 17 flushed in init_1d_model.

List of changed parameter names:
Old name New name Comments
average_period_pl1d averaging_interval_pr
average_period_pr1d --- averaging of printed profiles is not possible any more
average_period_sp averaging_interval_sp
dt_average dt_averaging_input_pr
dt_pl1d dt_dopr
dt_plsp dt_dosp
dt_plts dt_dots
dt_pr1d dt_dopr_listing
dt_pl2d_xy dt_do2d_xy
dt_pl2d_xz dt_do2d_xz
dt_pl2d_yz dt_do2d_yz
ebene_xy section_xy
ebene_xz section_xz
ebene_yz section_yz
nz_plot3d nz_do3d
plts data_output_ts
pl1d data_output_pr
pl2d data_output old parameters pl2d and pl3d are joined to new parameter data_output
pl3d data_output
pl2d_at_begin do2d_at_begin
pl3d_at_begin do3d_at_begin
pl3d_compress do3d_compress
pl3d_precision do3d_comp_prec
pl_spectra data_output_sp
read_fields_from_prior_run read_restart_data
write_plot2d_on_each_pe data_output_2d_on_each_pe
z_max_pl1d z_max_do1d
z_max_pl2d z_max_do2d
advec_particles, advec_s_bc, check_parameters, check_open, close_file, data_output_dvrp,  data_output_profiles, data_output_tseries, data_output_2d, data_output_3d, flow_statistics, header, init_1d_model, init_3d_model, init_dvrp, init_particles, init_pegrid, init_slope, modules, netcdf, package_parin, palm, parin, print_1d, read_var_list, run_control, time_integration, timestep, write_compressed, write_var_list
      E Error in the particle release (defaults of psl, psr, psn, pss) at the PE boundary has been fixed.

Error removed getting the variable ids for _ext for profiles and time series.

NetCDF calls in data_output_tseries only if NetCDF output is switched on.

Output of the initial geostrophic wind profile corrected.

Maximum possibel timestep for Runge-Kutta-schemes reduced, error concerning initialization of  l_black removed (both in init_1d_model).
advec_particles, data_output_tseries, header, init_particles, init_1d_model, netcdf
26/04/06 SR 3.0a N OpenMP parallelization for the cache-optimized PALM code.

OpenMP directives have been added where necessary. Still further tuning will be necessary to get optimum performance. Other parts of the code than the cache-optimized code still need to be (OpenMP-) parallelized.

So far, threads are activated by setting the mrun-options -T <number_of_threads> and -O, i.e. when the Option -O is given, the -T argument is interpreted as the number of threads to be used. When -O is given, the number of MPI tasks per node is automatically set to 1.

Beside the additional compiler directives, the main code change required by the OpenMP parallelization is that global sums are now calculated by first calculating local sums on each thread. As the next step, local sums are calculated from these thread sums for each MPI task before the global sum is computed via MPI_ALLREDUCE.  For storing the thread sums, arrays sums_l and sums_l_l now have three dimensions where the number of elements of the third dimension is equal to the number of OpenMP threads.
buoyancy, cpu_statistics, diffusivities, flow_statistics, header, init_pegrid, init_3d_model, modules, prandtl_fluxes, pres, production_e
  ML   C Provisional correction for Piacsek & Williams advection scheme: keep u and v zero one layer below the topography (in case of set_1d-model_profiles). init_3d_model
  SR/ML   E In flow_statistics.f90 nzb_s_inner(j,i) replaced by nzb in determination of z_i.

Errors removed in the computation of the diabatic mixing length (init_1d_model.f90).

Error removed in extend mode when checking whether the selected cross sections match those in the already existing NetCDF file (netcdf.f90).
flow_statistics, init_1d_model, netcdf
02/06/06 SR 3.0b N This version is adapted to the Sun Fire X4600 system at TIT (to be used by setting the mrun option -h lctit). cpu_log, init_1d_model, local_tremain, locaö_tremain_ini, run_control
      C NetCDF 2d- and 3d-datasets now contain the exact coordinates of the variables along x and y with respect to the positions where they are defined on the staggered grid.

NetCDF datasets of cross sections (e.g. DATA_2D_XY_NETCDF) are only opened if the respective cross sections are really requested by the settings of parameter data_output.
check_parameters, data_output_2d, data_output_3d, modules, netcdf, user_interface
      E p is assigned to to_be_resorted instead of w data_output_2d
04/08/06 SR 3.1 N Subgrid scale velocities can (optionally) be included for calculating the particle advection, using the method of Weil et al. (2004, JAS, 61, 2877-2887). This method is switched on by the new particle package parameter use_sgs_for_particles. This also forces the Euler/upstream method to be used for time advancement of the TKE (see new parameter use_upstream_for_tke). The minimum timestep during the sub-timesteps is controlled by parameter dt_min_part. The data type particle_type has additional new attributes e_m, dt_sum, and speed_x/y/z_sgs.

Other new particle features: output of particle quantities as timeseries in NetCDF format (on local file DATA_1D_PTS_NETCDF). The output time interval is controlled by parameter dt_dopts. Particle advection can be switched off after some time using parameter end_time_prel. More than one particle per point can be started with parameter particles_per_point.

Additional parameter in function random_gauss which limits the range of the created random numbers to five times the standard deviation (=1). Seeds iran and iran_part are stored for restart runs.
advec_particles, check_open, check_parameters, close_file, diffusion_e, disturb_heatflux, header, init_3d_model, init_particles, modules, netcdf, package_parin, parin, prognostic_equations, random_gauss, read_3d_binary, read_var_list, time_integration, write_3d_binary, write_var_list

new:
data_output_ptseries
      C Call of subroutine header is moved after call of subroutine init_3d_model. Generation of run_description_header is moved from routines palm and header to check_parameters. Determination of the number of particle groups is moved from header to init_particles.

izuf renamed iran.
check_parameters, disturb_field, disturb_heatflux, header, init_particles, modules, palm
      E Variables do2d_unit and do3d_unit now defined as 2d-arrays. Before, in some cases, the units of variables in the 2d- and 3d-NetCDF-files have been wrong.

In routine poisfft default setting of the thread number tn in case of not using OpenMP.

Error removed in the non-parallel part of routine flow_statistics (number of arguments of array sums_l).

Error removed which appeared if the user had decreased the value of dt_dopr within a job chain and if simulataneous time averaging of profiles was switched on.
check_parameters, data_output_profiles, flow_statistics, modules, netcdf, poisfft
22/08/06 SR 3.1a N In case of more than one particle group, seperate output of timeseries for each of the groups.

New initial parameter dz_max, which limits the vertical gridspacing in case of a vertically stretched grid.
data_output_ptseries, header, init_grid, modules, netcdf, parin, read_var_list, write_var_list
      C Disturbances are imposed only for the last Runge-Kutta-substep.
Attention:
The content of the testresult-file (run control output) has changed due to this modification.

Output of xz and yz cross sections now up to gridpoint nzt+1.

Default settings of particle start positions changed.
check_open, close_file, data_output_2d, init_particles, netcdf, time_integration
      E Initialisation of all tendency arrays (t.._m) needed for the Runge-Kutta schemes.

Bugfix in index array initialization for line- or point-like topography structures.

Bugfix: yv coordinates are now used for yz cross sections, where neccessary.
init_3d_model, init_grid, netcdf
12/02/07 SR 3.1b  N Attention:
This is the last revision before switching to the subversion revison control system!

Informative output to the job protocoll in case of restarts.
check_for_restart
         C Values of environment variables are now read from local file ENVPAR instead of getting them by a system call.

For interpolation in advec_particles, allways level k is used.
advec_particles, local_tremain_ini, parin
      E Several bugfixes in advec_particles: new particles are released only if end_time_prel > simulated_time, transfer of particles when x < -0.5*dx (0.0 before), etc., index i,j used instead of cartesian (x,y) coordinate to check for transfer because this failed under very rare conditions, calculation of number of particles with same radius as the current particle (cloud droplet code).

Allocation of tail_mask and new_tail_id in case of restart-runs. "__" added in a cpp-directive. (both init_particles)
advec_particles, init_particles
02/03/07 SR 3.1c N A heatflux can be prescribed at the top with new inipar parameters top_heatflux and use_top_fluxes. New 2d-arrays qswst, qswst_m, tswst and tswst_m are used to store this flux. Use of fluxes are controlled with new index variable nzt_diff. A Neumann boundary condition for temperature can be applied under these conditions.

Additionally, a Dirichlet condition for temperature can be used at the top.

Attention:
The content of the testresult-file (run control output) has changed due to this modification.
check_parameters, diffusion_s, flow_statistics, header, init_grid, init_3d_model, modules, parin, production_e, prognostic_equations, read_var_list, read_3d_binary, swap_timelevel, write_var_list, write_3d_binary
      C Prognostic equations for all scalars are now solved up to gridpoint nzt (formerly nzt-1). Boundary conditions for scalars at top adjusted respectively (now applied only at nzt+1).

The default top boundary condition for temperature has been renamed to 'initial_gradient'.

Calls of dvrp_output_local, which were commented out for a long time, are now activated for all streams.
advec_s_pw, boundary_conds, calc_precipitation, check_parameters, diffusion_e, diffusion_s, impact_of_latent_heat, init_dvrp, init_pt_anomaly, modules, production_e, prognostic_equations, spline_z
      E Bugfix: 3d-array p is not a pointer any more.

Bugfix in init_particles: MPI_REAL argument in MPI_ALLREDUCE replaced by
MPI_INTEGER (caused error on NEC only).

Bugfix:  ddzw now dimensioned 1:nzt"+1".
diffusion_e, diffusion_s, diffusion_u, diffusion_v, diffusion_w, init_particles, modules
29/03/07 SR 3.2 N The code, including scripts, utility programs and documentation is now under full control of subversion. This revision includes a quite large number of new features, changes and bugfixes.
 
Particle reflection from vertical walls implemented. The particle SGS model is also adjusted to these vertical walls.

Wall functions for vertical walls now include diabatic conditions. New subroutines wall_fluxes, wall_fluxes_e. New 4D-array rif_wall.

Calculation/output of precipitation amount, precipitation rate and z0 (by setting "pra*", "prr*", "z0*" with data_output). The time interval on which the precipitation amount is defined is set by new d3par-parameter precipitation_amount_interval.

New inipar-parameter loop_optimization to control the loop optimization method.

New inipar-parameter pt_reference. If given, this value is used as the reference that is used in the buoyancy terms (otherwise, the instantaneous horizontally averaged temperature is used).

New d3par-parameter netcdf_64bit_3d to switch on and especially to switch off 64bit offset for 3D files only (because still some pd-software is unable to handle 64bit offset).

New d3par-parameter dt_max to set the maximum value allowed for the timestep.

New user interfaces user_advec_particles and user_init_3d_model. The new initializing action "by_user" calls user_init_3d_model and allows the initial setting of all 3d arrays under complete user control.

Samples added to the user interface which show how to add user-define time series quantities.

Topography height informations are stored on arrays zu_s_inner and zw_w_inner and output to the 2d/3d NetCDF files.

Unit 9 is generally opened for output of debug informations (file DEBUG_<pe#>).
Makefile

advec_particles, average_3d_data, buoyancy, calc_precipitation, check_open, check_parameters, data_output_2d, diffusion_e, diffusion_u, diffusion_v, diffusion_w, diffusivities, header, impact_of_latent_heat, init_particles, init_3d_model, modules, netcdf, parin, production_e, read_var_list, read_3d_binary, sum_up_3d_data, user_interface, write_var_list, write_3d_binary

new:
wall_fluxes
      C General revision of non-cyclic horizontal boundary conditions:
Radiation boundary conditions are now used instead of Neumann conditions at the outflow (calculation needs velocity values for t-dt, which are stored on new arrays u_m_l, u_m_r, etc.). The calculation of a mean outflow is not needed any more. Volume flow control is added for the outflow boundary (currently only for the north boundary!!). Additional gridpoints along x and y (uxrp, vynp) are not needed any more and removed from the code. Routine "boundary_conds" now operates on timelevel t+dt and is not split in two parts (main, uvw_outflow) any more. Neumann boundary conditions are used at inflow/outflow in case of non-cyclic boundary conditions for all 2d-arrays that are handled by exchange_horiz_2d.
Attention: the non-cyclic boundary conditions of this revision still contain some bugs which will be removed within the next revisions!

The Bott-Chlond-scheme can now be used in the vectorization-optimized part (NEC, etc.). Equations are solved up to gridpoint nzt.

The FFT-method for solving the Poisson-equation is now working with Neumann boundary conditions both at the bottom and the top. This requires adjustments of the tridiagonal coefficients and subtracting the horizontally averaged mean from the vertical velocity field.

Particles-package is now part of the default code ("-p particles" is not needed as an mrun-option any more). New attribute age_m in particle_type.

Move call of user_actions( 'after_integration' ) below the increments of times
and counters. user_actions is now called for each statistic region and has as an argument the number of the respective region (sr).

d3par-parameter data_output_ts is removed. Timeseries output for profil-software removed. Timeseries are now switched on by setting dt_dots. Timeseries data is collected in flow_statistics.

Initial velocities at nzb+1 are regarded for volume flow control in case they have been set zero before (to avoid small timesteps); see new internal parameters u/v_nzb_p1_for_vfc.

q is not allowed any more to become negative (prognostic_equations).

poisfft_init is only called if the fft-solver is switched on (init_pegrid).

d3par-parameter moisture renamed to humidity.

Subversion global revision number is read from mrun and added to the run description header and to the run control (_rc) file.

__vtk directives removed from main program.

The uitility routine interpret_config now reads PALM environment variables from NAMELIST instead using the system call GETENV.
advec_s_bc, advec_u_pw, advec_u_up, advec_v_pw, advec_v_up, asselin_filter, boundary_conds, check_parameters, coriolis, data_output_dvrp, data_output_ptseries, data_output_ts, data_output_2d, data_output_3d, diffusion_u, diffusion_v, disturb_field, exchange_horiz, exchange_horiz_2d, flow_statistics, header, init_grid, init_particles, init_pegrid, init_rankine, init_pt_anomaly, init_1d_model, init_3d_model, modules, palm, package_parin, parin, poisfft, poismg, prandtl_fluxes, pres, production_e, prognostic_equations, read_var_list, read_3d_binary, sor, swap_timelevel, time_integration, write_var_list, write_3d_binary
      E Bugfix: preset of tendencies te_em, te_um, and te_vm (init_1d_model).

Bugfix in sample for reading user defined data from restart file (user_init).

Bugfix in setting diffusivities for cases with the outflow damping layer extending over more than one subdomain (init_3d_model).

Check for possible negative humidities in the initial humidity profile.

In Makefile, default suffixes are removed from the suffix list to avoid calling of m2c in cases of existing .mod files (problem on Linux systems).
Makefile

check_parameters, init_1d_model, init_3d_model, user_interface
19/04/07 SR 3.2a C All system relevant compile and link options (e.g. cpp-directives, library paths, etc.) as well as the host identifier (local_host) are now specified in the configuration file. This generally allows to install PALM on any kind of Linux-, IBM-AIX-, or NEC-SX-system by adding appropriate settings to the configuration file. A description is added to chapter 5.0 of the online documentation.

Scripts are also running under the public domain ksh.

Filetransfer by ftp removed (options -f removed from mrun and mbuild).

Call of (system-)FLUSH routine moved to new routine local_flush.

return_addres and return_username are read from ENVPAR-NAMELIST-file instead of using local_getenv.

Preprocessor strings for different linux clusters changed to "lc", some preprocessor directives renamed (new: __intel_openmp_bug which is used for the bug in tridia_1dd and the OpenMP directives). Preprocessor directives for old systems have been removed.

advec_particles, check_open, cpu_log, cpu_statistics, data_output_dvrp, flow_statistics, header, init_dvrp, init_particles, init_1d_model, init_dvrp, init_pegrid, local_getenv, local_system, local_tremain, local_tremain_ini, modules, palm, parin, run_control

new:
local_flush

changed scripts:
mbuild, mrun, .mrun.config.default
30/05/07 SR 3.2b N Calculation and output of user-defined profiles: New userpar-parameter data_output_pr_user. In the user-interface, routine user_check_data_output_pr has been added and user_statistics has been extended.
check_parameters, flow_statistics, modules, parin, read_var_list, user_interface, write_var_list
      C In pres.f90, division through dt_3d replaced by multiplication of the inverse. For performance optimisation, this is done in the loop calculating the divergence instead of using a seperate loop.

Variables var_hom and var_sum are both renamed pr_palm.
data_output_profiles, flow_statistics, init_3d_model, modules, parin, pres, read_var_list, run_control, time_integration
      E Bugfix: work_fft*_vec removed from some PRIVATE-declarations (poisfft).

Bugfix: field_chr renamed field_char (user_interface).

Bugfix: output of use_upstream_for_tke (header).
header, poisfft, user_interface
21/06/07 SR 3.3 N This version allows runs for the ocean. These runs can be switched on with the new inipar-parameter ocean.

Setting this switch has several effects:
  • An additional prognostic equation for salinity is solved.
  • Potential temperature in buoyancy and stability-related terms is replaced by potential density.
  • Potential density is calculated from the equation of state for seawater after each timestep, using the algorithm proposed by Jackett et al. (2006, J. Atmos. Oceanic Technol., 23, 1709-1728).
    So far, only the initial hydrostatic pressure is entered into this equation.
  • z=0 (sea surface) is assumed at the model top (vertical grid index k=nzt on the w-grid), with negative values of z indicating the depth.
  • Initial profiles are constructed (e.g. from pt_vertical_gradient / pt_vertical_gradient_level) starting from the sea surface, using surface values given by pt_surface, sa_surface, ug_surface, and vg_surface.
  • Zero salinity flux is used as default boundary condition at the bottom of the sea.
  • If switched on, random perturbations are by default imposed to the upper model domain from zu(nzt*2/3) to zu(nzt-3).
Relevant new inipar-parameters to be exclusively used for steering ocean runs are bc_sa_t, bottom_salinityflux, sa_surface, sa_vertical_gradient, sa_vertical_gradient_level, and top_salinityflux.

Salinity (sa) and potential density (rho) are included as new 2d/3d output quantities. Vertical profiles of salinity (sa), salinity fluxes (w"sa", w*sa*, wsa), and potential density (rho) can also be output.
advec_s_bc, average_3d_data,
boundary_conds, buoyancy,
check_parameters, data_output_2d,
data_output_3d, diffusion_e,
diffusivities, flow_statistics, header,
init_grid, init_3d_model, modules,
netcdf, parin, production_e,
prognostic_equations,
read_3d_binary,  read_var_list,
sum_up_3d_data, swap_timelevel,
time_integration, user_interface,
write_var_list, write_3d_binary

new:
eqn_state_seawater, init_ocean
      C Inipar-parameter use_pt_reference renamed use_reference.
Internal variable hydro_press renamed hyp, routine calc_mean_pt_profile renamed calc_mean_profile.

The format of the RUN_CONTROL file has been adjusted for ocean runs.

Defaults of .._vertical_gradient_levels changed from -1.0 to -9999999.9.
advec_particles, buoyancy, 
calc_liquid_water_content,
check_parameters, diffusion_e,
diffusivities, header,
init_cloud_physics, modules,
production_e, prognostic_equations,
run_control
      E Bugfix: height above topography instead of height above level k=0 is used for calculating the mixing length (diffusion_e and diffusivities).

Bugfix: error in boundary condition for TKE removed (advec_s_bc).
advec_s_bc, diffusion_e, diffusivities
05/10/07 SR 3.4 N This version allows runs for a coupled atmosphere-ocean LES.

Coupling frequency is controlled by new d3par-parameter dt_coupling, the coupling mode ('atmosphere_to_ocean' or 'ocean_to_atmosphere') for the respective processes is read from environment variable coupling_mode, which is set by the mpiexec-command in mrun. Communication between the two models is handled by the intercommunicator comm_inter. Local files opened by the ocean model get the additional suffic "_O". Saturation at k=nzb is assumed for the atmosphere coupled to ocean. Usage of the coupled model is described in the new section 3.8.

A momentum flux can be set as top boundary condition using the new inipar parameters top_momentumflux_u|v.

Non-cyclic boundary conditions can be used along all horizontal directions.

Quantities w*p* and w"e can be output as vertical profiles.

Initial profiles are reset to constant profiles in case that initializing_actions /= 'set_constant_profiles'. (init_rankine)

Optionally calculate km and kh from initial TKE e_init.
boundary_conds, check_open, check_parameters, diffusion_u, diffusion_v, flow_statistics, header, init_pegrid, init_rankine, init_3d_model, local_stop, modules, palm, parin, prandtl_fluxes, pres, prognostic_equations, read_var_list, read_3d_binary, swap_timelevel, timestep, time_integration, write_var_list, write_3d_binary

new:
surface_coupler
      C Remaining variables iran changed to iran_part (advec_particles, init_particles).

In case that the presure solver is not called for every Runge-Kutta substep (call_psolver_at_all_substeps = .F.), it is called after the first substep instead of the last. In that case, random perturbations are also added to the velocity field after the first substep.

Initialization of km, kh = 0.00001 for ocean = .T. (for ocean = .F. it remains 0.01).

Allow data_output_pr = 'q', 'wq', 'w"q"', 'w*q*' for humidity = .T. (instead of cloud_physics = .T.).
advec_particles, check_for_restart, check_parameters, init_particles, init_3d_model, time_integration
      E Bugs from code parts for non-cyclic boundary conditions are removed:
loops for u and v are starting from index nxlu, nysv, respectively. The radiation boundary condition is used for every Runge-Kutta substep. Velocity phase speeds for the radiation boundary conditions are calculated for the first Runge-Kutta substep only and reused for the further substeps. New arrays c_u, c_v, and c_w are defined for this purpose. Several index errors are removed from the radiation boundary condition code parts. Upper bounds for calculating u_0 and v_0 (in production_e) are nxr+1 and nyn+1 because otherwise these values are not available in case of non-cyclic boundary conditions.

+dots_num_palm in module user, +module netcdf_control in user_init (both in user_interface)

Bugfix: wrong sign removed from the buoyancy production term in the case use_reference = .T. (production_e)

Bugfix: Error message concerning output of particle concentration (pc) modified (check_parameters).

Bugfix: Rayleigh damping for ocean fixed.
advec_u_pw, advec_u_up, advec_v_pw, advec_v_up, boundary_conds, buoyancy, check_parameters, coriolis, diffusion_u, diffusion_v, header, init_pegrid, init_3d_model, modules, poismg, production_e, prognostic_equations, user_interface
29/11/07 SR 3.4a N Pressure boundary conditions for vertical walls added to the multigrid solver. They are applied using new wall flag arrays (wall_flags_..) which are defined for each grid level. New argument gls added to routine user_init_grid (user_interface).

Plant canopy model of Watanabe (2004,BLM 112,307-341) added.  It can be switched on by the new inipar parameter plant_canopy. The inipar parameter canopy_mode can be used to prescribe a plant canopy type. The default case is a homogeneous plant canopy. Heterogeneous distributions of the leaf area density and the canopy drag coefficient can be defined in the new routine user_init_plant_canopy (user_interface). The inipar parameters lad_surface, lad_vertical_gradient and lad_vertical_gradient_level can be used to prescribe the vertical profile of leaf area density. The inipar parameter drag_coefficient determines the canopy drag coefficient. Finally, the inipar parameter pch_index determines the index of the upper boundary of the plant canopy. 

Allow new case bc_uv_t = 'dirichlet_0' for channel flow.

For unknown variables (CASE DEFAULT) call new subroutine user_data_output_dvrp.

Frequence of sorting particles can be controlled with new particles_par parameter dt_sort_particles. Sorting is moved from the SGS timestep loop in advec_particles after the end of this loop.
advec_particles, check_parameters, data_output_dvrp, header, init_3d_model, init_grid, init_particles, init_pegrid, modules, package_parin, parin, read_var_list, user_interface, write_var_list

new:
plant_canopy_model
      C Redefine initial nzb_local as the actual total size of topography (later the extent of topography in nzb_local is reduced by 1dx at the E topography walls and by 1dy at the N topography walls to form the basis for nzb_s_inner); for consistency redefine 'single_building' case.

Vertical profiles now based on nzb_s_inner; they are divided by ngp_2dh_s_inner (scalars, procucts of scalars) and ngp_2dh (staggered velocity components and their products, procucts of scalars and velocity components), respectively.

Allow two instead of one digit to specify isosurface and slicer variables.

Status of 3D-volume NetCDF data file only depends on switch netcdf_64bit_3d (check_open).

prognostic_equations include the respective wall_*flux in the parameter list of calls of diffusion_s. Same as before, only the values of wall_heatflux(0:4) can be assigned. At present,  wall_humidityflux, wall_qflux, wall_salinityflux, and wall_scalarflux are kept zero. diffusion_s uses the respective wall_*flux instead of wall_heatflux. This update serves two purposes: - it avoids errors in calculations with humidity/scalar/salinity and prescribed non-zero wall heatflux, - it prepares PALM for a possible assignment of wall fluxes of humidity/scalar/salinity in a future releases.
buoyancy, check_open, data_output_dvrp, diffusion_s, diffusivities, flow_statistics, header, init_3d_model, init_dvrp, init_grid, modules, prognostic_equations
      E Bugfix: summation of sums_l_l (diffusivities).

Several bugfixes in the ocean part: Initial density rho is calculated (init_ocean). Error in initializing u_init and v_init removed (check_parameters). Calculation of density flux now starts from nzb+1 (production_e).

Bugfix: pleft/pright changed to pnorth/psouth in sendrecv of particle tail numbers along y, small bugfixes in the SGS part (advec_particles).

Bugfix: model_string needed a default value (combine_plot_fields).

Bugfix: wavenumber calculation for even nx in routines maketri (poisfft).

Bugfix: assignment of fluxes at walls.

Bugfix: absolute value of f must be used when calculating the Blackadar mixing length (init_1d_model).
advec_particles, check_parameters, combine_plot_fields, diffusion_s, diffusivities, init_ocean, init_1d_model, poisfft, production_e
16/09/08 SR 3.5 N The turbulence recycling method of Kataoka and Mizuno (2002) has been implemented. A precursor run must be used to initialize the main run with a quasi-stationary turbulence state (see initializing_actions = 'read_data_for_recycling'). The precursor run may have a smaller domain than the main run, which helps to spare CPU time. New parameters for steering the recycling method are turbulent_inflow, recycling_width, inflow_damping_height, and inflow_damping_width.

One new feature related with the recycling method is that within a job chain, the number of processors or the virtual processor grid is now allowed to change. Such a change modifies the domain decomposition and requires that the PEs within a restart job have to know from which file(s) they have to read their restart data (up to now there has been a one to one relation between the restart files and the PEs, i.e. processor 4 only reads data from file "_0004"). Informations about the subdomain index limits of all restart files are now written by PE0 (to file "_0000"). Also, all variables containing global values which do not depend on the subdomain are written to file "_0000" only. In a restart run, this file is opened by all PEs to read these global values and the informations about the subdomain index limits. Depending on these index limit information it is decided which restart file(s) has/have to be opened in order to read the complete data necessary for the current subdomain.

The plant canopy model has been extended to the prognostic equation for the potential temperature and passive scalar, as proposed by Watanabe (2004) and Shaw and Schumann (1992).

Particle advection adjusted for ocean runs.

Grid coarsening (multigrid method) is limited to subdomains (i.e. no collection of data on PE0 for further coarsening beyond subdomains) if mg_switch_to_pe0_level = -1.

User-defined spectra are allowed.

Output of q*2 profile and w"u"0, w"v"0, w"q"0 timeseries are added.

Argument -c introduced to most of the subjob calls, which allows the user to choose his own job catalog by setting the environment variable job_catalog in the configuration file (default is ~/job_queue).

This release contains first adjustments for the SGI-ICE system at HLRN.

Attention: On the SGI-ICE, runs are still aborted or are hanging for unknown reasons from time to time.
advec_particles, calc_spectra, check_open, check_parameters, data_output_spectra, header, init_3d_model, init_particles, init_pegrid, modules, netcdf, parin, particle_boundary_conds, plant_canopy_model, prognostic_equations, read_3d_binary, read_var_list, time_integration, user_interface, write_3d_binary, write_var_list

new:
inflow_turbulence
      C Allow 100 spectra levels instead of 10 for consistency with define_netcdf_header. (calc_spectra, data_output_spectra, header, modules, netcdf)

Modification of the integrated version of the profile function for momentum for unstable stratification; more consistent flux definitions. (wall_fluxes, production_e)

Strict grid matching along z is not needed for mg-solver. (check_parameters)

Sharp lateral edges of the plant canopy introduced. Before, smaller values of the leaf area density than those applied in the prognostic equation for the vertical velocity were used in the prognostic equations for u and v at the boundaries of the plant_canopy. (init_3d_model)

-cross_ts_* (read_var_list, write_var_list), -dopr_time_count, old profil parameters (cross_..., dopr_crossindexprofile_***) removed. (read_3d_binary, write_3d_binary)

Reading/writing of hom_sum, volume_flow_area, volume_flow_initial is moved from ..._3d_binary to ..._var_list, hom_sum is now allocated in parin instead of init_3d_model.

npex, npey moved from inipar- to d3par-namelist. (parin)

myid_char14 removed. This was used, since on IBM machines the process rank could be changed when the final communicator was defined, in order to save the preliminary processor-id for opening the binary output file for restarts (unit 14), because otherwise a mismatch occured when reading this file in a restart job. This mechanism is not needed any more since the subdomain informations are now contained in file _0000. (parin, check_open)

Transpositions for the 2D domain decomposition have been optimized by using f_inv as an automatic array instead of providing the memory by a dummy argument. This spares one copy loop per transposition. Order of indices in the 3D loops in some of the transpose routines have been rearranged for better cache utilization. Both have been suggested by Roland Richter (SGI) as part of the HLRN-II benchmark process. (transpose)

Workaround for getting information about the coupling mode, because the -env Option (mpiexec) is not available on many systems (SGI, lcxt4 at  Bergen Center for Computational Science). (palm, mrun)

Format adjustments in order to allow CPU# > 999. (cpu_statistics)

Provisional solution for run_control_1d output: add 'CALL check_open( 15 )'. (init_1d_model)

Subdomain data are read into temporary arrays pf_tmp / pf3d_tmp in order to avoid INTEL compiler warnings about (automatic) creation of temporary arrays. (combine_plot_fields)

Time coordinate t=0 stored on netcdf-file only if an output is required for this time for at least one of the profiles.

Output of spline timeseries splptx etc. removed. (flow_statistics)
advec_s_ups, advec_u_ups, advec_v_ups, advec_w_ups, calc_spectra, check_open, check_parameters, cpu_statistics, data_output_profiles, data_output_spectra, flow_statistics, header, init_1d_model, init_3d_model, modules, netcdf, palm, parin, poisfft, production_e, read_3d_binary, read_var_list, transpose, wall_fluxes, write_3d_binary, write_var_list
      E Bugfix: modification of the calculation of the vertical turbulent momentum fluxes u'w' and v'w'; the first usws that is computed corresponds to -u'w'/u* and not as priorily assumed to (-u'w')**0.5, the first vsws that is computed corresponds to -v'w'/u* and not as priorily assumed to (-v'w')**0.5. Therefore, the intermediate result for usws has to be multiplied by -u* instead by itself in order to get u'w'. Accordingly, the intermediate result for vsws has to be multiplied by -u* instead by itself in order to get v'w'. As u* is calculated for the position of a scalar an additional interpolation of u* to the position of u and v, respectively, is necessary. As u* is not determined for the ghost points on each PE, an additional exchange of information from neighbouring PEs is necessary. Tests showed that this decreases u* by some 10% and increases TKE and momentum fluxes by some 10% because friction was underestimated and momentum fluxes were wrongly calculated due to the bug. (prandtl_fluxes, wall_fluxes)

Attention:
The content of the testresult-file (run control output) has changed due to the above bugfix.

Bugfix: change definition of us_wall from 1D to 2D. 

Bugfix: exponent = 1.0 / ( ny + 1.0 ) in calc_spectra_x. (calc_spectra)

Bugfix: calculation of horizontal fluxes at vertical walls. (diffusion_s)

Bugfix: zero assignments to tendency arrays in case of restarts. (init_3d_model)

Bugfix: wrong grid level used in multigrid solver, routine restrict. (poismg)

Bugfix: Construction of vertical profiles when 10 gradients have been specified in the parameter list (ug, vg, pt, q, sa, lad). (check_parameters)

Bugfix: Determination of the number of vertical levels for which spectra shall be output - 100 replaced by 10. (netcdf)

Bugfix: Lengths of those characters that are needed for the output of the characteristic levels of potential temperature, the geostrophic wind, scalar concentration, humidity and leaf area density to the header file has been increased. (header)

Bugfix: add ')' in user_read_restart_data.

The leaf area density is explicitly set to its surface value at k=0. (check_parameters)

Introduce prefix_chr to ensure unique dvrp_file path.

small bugfixes for user_interface sample code (comments):
- initialize ustvst with 0.0 as it is now computed only until nxr and nyn
- two ALLOCATE statements moved from user_read_restart_data back to user_init
- remove 'READ (13) u2_av' statement in user_read_restart_data

Bugfix: remove IF statement in plant_canopy_model_ij.

Bugfix: divide sums(k,8) (e) and sums(k,34) (e*) by ngp_2dh_s_inner(k,sr) (like other scalars). (flow_statistics)

Bugfix: dopr_time_count was written on the binary file, which caused that NetCDF files newly created by restart files (no append of existing files!) contained uneccessary time levels. (read_3d_binary, write_3d_binary)

Bugfix: extra '*' removed in user_statistics sample code. (user_interface)

Bugfix: a stop command was missing in some cases of the parallel branch. (local_stop)

Bugfix in volume flow control for non-cyclic boundary conditions. (pres)

Bugfix: misplaced #endif directives. (combine_plot_fields)

Data are collected from PE0 in an ordered sequence which seems to avoid hanging of processes on SGI-ICE. (cpu_statistics)
calc_spectra, check_parameters, cpu_statistics, diffusion_s, flow_statistics, init_3d_model, init_dvrp, local_stop, plant_canopy_model, poismg, prandtl_fluxes, pres, read_3d_binary, user_interface, wall_fluxes, write_3d_binary
 Attention: If make is not used for compiling, all routines have to be re-compiled after every change to modules.f90!
 
 

2.0  How to change the model source code

Currently (Sep 23rd, 2009), only selected users are allowed to commit changes to the repository. Some rules about how to check in modified code will be added later.

3.0  Description of selected parts of the model source code

This section gives links to existing descriptions about special parts of the PALM code, e.g. special numerical methods that are used (including mathematical background). Documents are stored in single files with different formats. Some are in German (sorry for the inconvenience).