PALM   a parallelized LES model

technical/numerical documentation
 
 
 
Current fixed release: 3.7a
Last change of this document:  $Id: technical_documentation.html 493 2010-03-01 08:30:24Z raasch $

Table of contents
 
 


1.0  History of changes

The following table documents all changes to the PALM repository since 2009. Older changes can be found here.

Entries in the fifth column specify the type of the change: change  of existing code (C), new code (N) or bugfix (B). More detailed information about the changes may sometimes be found in the header comment lines of the respective routines.
 
date
 author
 svn revision
 latest release
 type
 description
 affected routines
05/02/10
 SR
 485

ngp_3d, ngp_3d_inner changed to 64 bit. Otherwise, on large grids > 2*10**9 grid points some global statistics (e.g. total kinetic energy) have wrong values.
 init_3d_model, modules
05/02/10
 SR
 484
B
 Typo in file headers removed ("current" instead of "actual").
 many
05/02/10
 SR
 483
 3.7a
 N
 compare_palm_logs is additionally compiled with mbuild -u . (Makefile in trunk/UTIL)

make options (mopts) to be set by configuration file implemented. (mrun, mbuild)

humidity = .T. can be used for runs with topography. wall_humidityflux and
wall_scalarflux are the corresponding new inipar parameters.
(check_parameters, init_3d_model, parin)

Large scale vertical motion (subsidence/ascent) can be added as additional source to the prognostic equation for potential temperature. See new parameters large_scale_subsidence, ws_vertical_gradient, and ws_vertical_gradient_level. (check_parameters, header, Makefile, modules, parin, prognostic_equations, read_var_list, subsidence, write_var_list)

Scripts have been updated for Cray-XT5 machine (lcxt5m) and Hitachi-machine (lckyoto).

A simple method for installing and running palm (with limited features) has been added. (Makefile, new scripts palm_simple_install, palm_simple_run)

Masked data output has been implemented but it is not documented in the user manual so far. This feature can already use NetCDF4 file format. (check_open, check_parameters, close_file, modules, netcdf, parin, time_integration, user_parin, new: data_output_mask, init_masks, user_data_output_mask)
 check_open, check parameters, close_file, header, init_3d_model, Makefile, mbuild, modules, mrun, netcdf, parin, prognostic_equations, read_var_list, time_integration, user_parin, write_var_list

new: data_output_mask,  init_masks, palm_simple_install, palm_simple_run, subsidence, user_data_output_mask




C
 2d-decomposition is default for Cray-XT machines. (init_pegrid)

var_ts is replaced by dots_max. (modules, init_3d_model)

Every cloud droplet has now an own weighting factor and can be deleted due to collisions. Condensation and collision of cloud droplets are adjusted accordingly. (advec_particles)

Collision efficiency for large cloud droplets has changed according to table of Rogers and Yau. (collision_efficiency)
 advec_particles, collision_efficiency, init_3d_model, init_pegrid, modules




B
 Bugfix for generating serial jobs. (subjob)

Bugfix: index problem concerning gradient_level indices removed. (header)

Dimension of array stat in cascade change to prevent type problems with mpi2 libraries. (poisfft_hybrid)

Loop was split to make runs reproducible when using ifort compiler. (disturb_field)

Bugfix: exchange of ghost points for prho included. (time_integration)

Bugfix in calculation of time-averaged surface heatfluxes. (sum_up_3d_data)

Bugfix in calculation of precipitation_rate. (calc_precipitation)

Bugfix: initial data assignments to some dvrp arrays changed due to error messages from gfortran compiler. (modules)

Bugfix in calculation of cloud droplet velocity. (advec_particles)

Bugfix in transfer of particles at south/left edge. (advec_particles)

Bugfix in calculation of collision efficiency. (collision_efficiency)
 advec_particles, calc_precipitation, collision_efficiency, disturb_field, header, modules, poisfft_hybrid, subjob, sum_up_3d_data, time_integration
23/09/09
 SR
 392
 3.7
 N
 Most important changes: atmosphere-ocean coupling has been improved and steering of dvr-software is more user-friendly now. Beside that, the release contains a lot of optimizations, changes, and bugfixes which have accumulated over the last months.

The atmosphere-ocean coupling allows to use independent precursor runs in order to account for different spin-up times. The time when coupling has to be started is given by new inipar parameter coupling_start_time. The precursor ocean run has to be started using new mrun option "-y" in order to add the appendix "_O" to all output files. (check_for_restart, check_parameters, data_output_2d, data_output_3d, data_output_profiles, data_output_ptseries, data_output_spectra, data_output_tseries, header, init_coupling, modules, mrun, parin, read_var_list, surface_coupler, time_integration, write_var_list)

Clipping of dvrp output implemented. Default colourtable for particles implemented, particle attributes (color, dvrp_size) can be set with new parameters particle_color, particle_dvrpsize, color_interval, dvrpsize_interval. Slicer attributes (dvrp) are set with new routine set_slicer_attributes_dvrp and are controlled with existing parameters slicer_range_limits. (init_dvrp, data_output_dvrp, modules, set_slicer_attributes_dvrp,  user_data_output_dvrp)

Polygon reduction for topography and ground plate isosurface. Reduction level for buildings can be chosen with parameter cluster_size. (init_dvrp)

Variables bc_lr / bc_ns  in most subroutines replaced by LOGICAL variables bc_lr_cyc,
bc_ns_cyc for speed optimization. This gives a significant speedup of more than 10%. (check_parameters, diffusion_u, diffusion_v, diffusion_w, modules)

Scripts have been adapted for machine lck (Yonsei Univ.). (mrun, mbuild, subjob)

Additional timestep criterion in case of simulations with plant canopy. (timestep)

Check for illegal entries in section_xy|xz|yz that exceed nz+1|ny+1|nx+1. (check_parameters)

External pressure gradient can be used as driving force. (check_parameters, header, init_3d_model, modules, parin, prognostic_equations, read_var_list, write_var_list)

New topography case 'single_street_canyon'. (header, init_grid, modules, parin, read_var_list, user_check_parameters, user_header, user_init_grid, write_var_list)

Option to predefine a target bulk velocity for conserve_volume_flow. (check_parameters, header, init_3d_model, modules, parin, read_var_list, write_var_list)

Option for user defined 2D data output in xy cross sections at z=nzb+1. (data_output_2d, user_data_output_2d)

xy cross section output of surface heatfluxes (sensible and latent). (average_3d_data, check_parameters, data_output_2d, modules, read_3d_binary, sum_up_3d_data, write_3d_binary)		 average_3d_data, check_for_restart, check_parameters, data_output_2d, data_output_3d, data_output_dvrp, data_output_profiles, data_output_ptseries, data_output_spectra, data_output_tseries, diffusion_u, diffusion_v, diffusion_w, init_coupling, init_dvrp, init_grid, init_3d_model, header, mbuild, modules, mrun, package_parin, parin, prognostic_equations, read_3d_binary, read_var_list, subjob, sum_up_3d_data, surface_coupler, timestep, time_integration, user_check_parameters, user_data_output_2d, user_data_output_dvrp, user_header, user_init_grid, write_3d_binary, write_var_list

new: set_particle_attributes, set_slicer_attributes_dvrp




C
 Output of messages replaced by message handling routine. (advec_particles, advec_s_bc, buoyancy, calc_spectra, check_for_restart, check_open, coriolis, cpu_log, data_output_2d, data_output_3d, data_output_dvrp, data_output_profiles, data_output_spectra, fft_xy, flow_statistics, header,  init_1d_model, init_3d_model, init_dvrp, init_grid, init_particles, init_pegrid, netcdf, parin, plant_canopy_model, poisfft_hybrid, poismg, read_3d_binary, read_var_list, surface_coupler, temperton_fft, timestep, user_actions, user_data_output_dvrp, user_dvrp_coltab, user_init_grid, user_init_plant_canopy, user_parin, user_read_restart_data, user_spectra)

Output of NetCDF messages with aid of message handling routine. (check_open, close_file, data_output_2d, data_output_3d, data_output_profiles, data_output_ptseries, data_output_spectra,  data_output_tseries, netcdf)

__lcmuk changed to __lc to avoid problems with Intel compiler on sgi-ice. (poisfft)

For extended NetCDF files, the updated title attribute includes an update of time_average_text where appropriate. (netcdf)

In case of restart runs without extension, initial profiles are not written to NetCDF-file anymore. So far, time axis always started at t=0. (data_output_profiles, modules, read_var_list, write_var_list)

initializing_actions='read_data_for_recycling' renamed to 'cyclic_fill'. A cyclic fill can now
be used independent of turbulent_inflow. (check_parameters, header, init_3d_model)

2 NetCDF error numbers changed. (data_output_3d)

A link to the webspage appendix_a.html is printed for further information about possible reasons of the error that appeared. (message)

Temperature gradient criterion for estimating the boundary layer height replaced by the gradient criterion of Sullivan et al. (1998). (flow_statistics)

NetCDF unit attribute in timeseries output in case of statistic regions added. (netcdf)

Maximum number of tails is calculated from maximum number of particles and skip_particles_for_tail. (init_particles)

Value of vertical_particle_advection is allowed to differ for each particle group. (advec_particles, header, modules)

First constant in array den also defined as type double. (eqn_state_seawater)

Parameter dvrp_psize moved from particles_par to dvrp_graphics_par. (package_parin)

topography_grid_convention moved from userpar to inipar (check_parameters, header, parin, read_var_list, user_check_parameters, user_header, user_init_grid, user_parin, write_var_list)

Default value of grid_matching changed to 'strict'. (modules)

Adjustments for runs on lcxt4 (necessary due to a software update on CRAY) and for coupled runs on ibmy. (mrun, subjob)		 advec_particles, advec_s_bc, buoyancy, calc_spectra, check_for_restart, check_open, check_parameters, close_file, coriolis, cpu_log, data_output_2d, data_output_3d, data_output_dvrp, data_output_profiles, data_output_ptseries, data_output_spectra, data_output_tseries, eqn_state_seawater, fft_xy, flow_statistics, header, init_1d_model, init_3d_model, init_dvrp, init_grid, init_particles, init_pegrid, message, mrun, netcdf, package_parin, parin, plant_canopy_model, poisfft, poisfft_hybrid, poismg, read_3d_binary, read_var_list, subjob, surface_coupler, temperton_fft, timestep, user_actions,user_check_parameters, user_data_output_dvrp, user_dvrp_coltab, user_header, user_init_grid, user_init_plant_canopy, user_parin, user_read_restart_data, user_spectra, write_var_list




B
 Bugfix: Initial hydrostatic pressure profile in case of ocean runs is now calculated in 5 iteration steps. (init_ocean)

Bugfix: wrong sign in buoyancy production of ocean part in case of not using the reference density (only in 3D routine production_e). (production_e)

Bugfix: output of averaged 2d/3d quantities requires that an avaraging interval has been set, respective error message is included. (check_parameters)

Bugfix: Output on unit 14 only if requested by write_binary. (user_last_actions)

Bugfix to avoid zero division by km_neutral. (production_e)

Bugfix for extended NetCDF files: In order to avoid 'data mode' errors if updated attributes are larger than their original size, NF90_PUT_ATT is called in 'define mode' enclosed by NF90_REDEF and NF90_ENDDEF calls. This implies a possible performance loss; an alternative strategy would be to ensure equal attribute size in a job chain. (netcdf)

Bugfix: correction of initial volume flow for non-flat topography. (init_3d_model)

Bugfix: zero initialization of arrays within buildings for 'cyclic_fill'. (init_3d_model)

Bugfix: to_be_resorted => s_av for time-averaged scalars. (data_output_2d, data_output_3d)

Bugfix: avoid that ngp_2dh_s_inner becomes zero. (init_3_model)

Typographical error: unit of wpt in dots_unit. (modules)

Bugfix: error in check, if particles moved further than one subdomain length. This check must not be applied for newly released particles. (advec_particles)

Bugfix: several tail counters are initialized, particle_tail_coordinates is only written to file if its third index is > 0, arrays for tails are allocated with a minimum size of 10 tails if there is no tail initially. (advec_particles, init_particles)

Bugfix: pressure included for profile output. (check_parameters)

Bugfix: Type of count and count_rate changed to default INTEGER on NEC machines. (cpu_log)

Bugfix: output of particle time series only if particle advection is switched on. (time_integration)

Bugfix: qsws was calculated in case of constant heatflux = .FALSE.. (prandtl_fluxes)

Bugfix: averaging along z is not allowed for 2d quantities (e.g. u* and z0). (data_output_2d)

Typographical errors. (netcdf)

If the inversion height calculated by the prerun is zero, inflow_damping_height must be explicitly specified. (init_3d_model)

Small bugfix concerning 3d 64bit netcdf output format. (header)

Bugfix: dt_fixed removed from the restart file, because otherwise, no change from a fixed to a variable timestep would be possible in restart runs. (read_var_list, write_var_list)

Bugfix: initial setting of time_coupling in coupled restart runs. (time_integration)		 advec_particles, check_parameters, cpu_log, data_output_2d, data_output_3d, header, init_3d_model, init_particles, init_ocean, modules, netcdf, prandtl_fluxes, production_e, read_var_list, time_integration, user_last_actions, write_var_list
02/02/09
 SR
 228
 3.6
 N
 This release contains further additions, changes, and bugfixes for the SGI-ICE system, as well as important changes like the splitting of the user-interface into single files, a complete revision of the pre-compiling mechanism allowing to use different make-depositories simultaneously, and a revision of the reading-mechanism from restart files, which has no more restrictions in case that the previous and current run have different domain/subdomain sizes. A new handling of all kind of output messages is introduced with this release and will be applied to all existing messages within the next releases.

New features are:

Restart runs on SGI-ICE are working. (mrun)
2d-decomposition is default on SGI-ICE systems. (init_pegrid)

Ocean-atmosphere coupling realized with MPI-1. mrun adjusted for this case (-Y option). Check that PALM is called with mrun-option "-K parallel" for coupling. Adjustments in mrun, mbuild, and subjob for lcxt4.

DVRP arguments changed to single precision, mode pathlines added.

User can add additional routines in files user_additional_routines.

User can check user parameters and deduce further quantities in user_check_parameters		 check_for_restart, check_parameters, data_output_dvrp, init_dvrp, init_pegrid, local_stop, modules, package_parin, palm, surface_coupler, timestep

Makefile, mbuild, mrun, subjob

new:
init_coupling, user_additional_routines, user_check_parameters




C
 User interface has been split into single files.

Type of variables count and count_rate changed to INTEGER(8) in order to avoid "out of range" problems, which result in measured negative time intervals. (cpu_log, local_tremain, local_tremain_ini)

Reading mechanism from restart files completely revised. The subdomain/total domain size is now allowed to vary arbitrarily between the current and previous run. (read_var_list, read_3d_binary, user_read_restart_data)

Precompilation mechanism (mbuild) completely revised: the source-tarfile remains in the source directory (trunk/SOURCE). One depository per block (given in the config-file) is created. Always all files from the source directory are copied to the respective depository. No additional file checks are done any more (mrun version 2.1). The depository name used contains the conditions given by mrun-option "-K". (mbuild, mrun)

Output messages (including required stop of execution) can now be handled with the new subroutine message. All output messages will be replaced by this routine step by step within the next revisions.

Neumann boundary condition at k=nzb is explicitly set for better reading, although this has been already done in boundary_conds. (advec_s_bc)

Origin of the xy-coordinate system shifted from the center of the first grid cell (indices i=0, j=0) to the south-left corner of this cell. (netcdf) Topography definition according to new user parameter topography_grid_convention. (init_grid, modules, user_header, user_init_grid, user_parin)		 advec_s_bc, check_parameters, cpu_log, init_grid, local_stop, local_tremain, local_tremain_ini, modules, netcdf, read_3d_binary, read_var_list, user_read_restart_data

Makefile, mbuild, mrun

removed:
user_interface

new:
message, user_3d_data_averaging, user_actions, user_advec_particles, user_check_data_output, user_check_data_output_pr, user_data_output_2d, user_data_output_3d, user_data_output_dvrp, user_define_netcdf_grid, user_dvrp_coltab, user_header, user_init_3d_model, user_init, user_init_grid, user_init_particles, user_init_plant_canopy, user_last_actions, user_parin, user_parin, user_particle_attributes, user_read_restart_data, user_spectra, user_statistics




B
 Bugfix: array d is reallocated in case that multigrid is used. (calc_spectra)

Bugfixes for nonparallel execution. (check_for_restart, cpu_statistics, inflow_turbulence, timestep)

Size of pf3d adjusted to the required output size (1 gridpoint less, along all three dimensions), because output of a subset of the data (pf3d(nxa:nxe...)) in the NF90_PUT_VAR statement caused segmentation fault with the INTEL compiler. (combine_plot_fields)

Bugfix: error in zu index in case of section_xy=-1. (header)

Inconsistency removed: as the thermal stratification is not taken into account for the evaluation of the wall fluxes at vertical walls, the eddy viscosity km must not be used for the evaluation of the velocity gradients dudy, dwdy, dvdx and dwdx. (production_e)

Bugfix in calculating k index in case of oceans runs. (sort_particles)

Bugfix: no output of particle concentration and radius unless particles have been started. (data_output_2d)

Bugfix: reading of spectrum_x|y from restart files ignored if total numbers of grid points do not match. (read_3d_binary)

Bugfix: abort in case that absolute temperature is below zero. (init_cloud_physics)		 advec_particles, calc_spectra, check_for_restart, cpu_statistics, data_output_2d, header, inflow_turbulence, init_cloud_physics, production_e, read_3d_binary, timestep

combine_plot_fields

 
 

2.0  How to change the source code and how to document modifications

Currently (Sep 23rd, 2009), only selected users are allowed to commit changes to the repository. Rules given below are preliminary, until we have switched to the "trac"-System.

  1. Change source code in your current working copy. Document changes in the respective file headers unter "Current revisions:". Additionally, document all changes in a local file (e.g. ~/palm/current_version/CURRENT_MODIFICATIONS), including the files that have been changed. Classify changes in three cases, new (N), changed (C), and bugfix (B). Try to be as short and as precise as possible in describing the changes!

  2. Test the changed version, unless it works sufficiently.

  3. Commit the changes to the repository

       svn commit -m "message string" trunk

    where "message string" is a short, meaningful summary of the changes.

    Alternatively, in case that you have made quite a lot of changes, you can also give the contents of your file CURRENT_MODIFICATIONS as the log message:

       svn commit -F CURRENT_MODIFICATIONS trunk

  4. Enter the changes into the technical documentation (under .../trunk/DOC/tec/technical_documentation.html).

  5. Move the change comments in the file headers from "Current revisions:" to "Former revisions:". Set the current svn id in front of that.

    Example:

    before move:

    ! Current revisions:
    ! -----------------
    ! new dummy argument abcd
    !
    ! Former revisions:
    ! -----------------
    ! $Id: technical_documentation.html 493 2010-03-01 08:30:24Z raasch $
    !
    ! 407 2009-12-01 15:01:15Z maronga
    ! var_ts is replaced by dots_max

    after move:

    ! Current revisions:
    ! -----------------
    !
    !
    ! Former revisions:
    ! -----------------
    ! $Id: technical_documentation.html 493 2010-03-01 08:30:24Z raasch $
    !
    ! 377 2008-08-30 11:03:41Z raasch
    !     new dummy argument abcd
    !
    ! 359 2008-06-01 15:01:15Z maronga
    ! var_ts is replaced by dots_max

  6. Commit the changes in the technical documentation and the file headers:

       svn commit -m "last commit documented" trunk


3.0  Description of selected parts of the model source code

This section gives links to existing descriptions about special parts of the PALM code, e.g. special numerical methods that are used (including mathematical background). Documents are stored in single files with different formats. Some are in German (sorry for the inconvenience).