source: palm/trunk/SOURCE/time_integration.f90 @ 2271

Last change on this file since 2271 was 2271, checked in by sward, 4 years ago

error messages and numbers updated

  • Property svn:keywords set to Id
File size: 42.2 KB
Line 
1!> @file time_integration.f90
2!------------------------------------------------------------------------------!
3! This file is part of PALM.
4!
5! PALM is free software: you can redistribute it and/or modify it under the
6! terms of the GNU General Public License as published by the Free Software
7! Foundation, either version 3 of the License, or (at your option) any later
8! version.
9!
10! PALM is distributed in the hope that it will be useful, but WITHOUT ANY
11! WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR
12! A PARTICULAR PURPOSE.  See the GNU General Public License for more details.
13!
14! You should have received a copy of the GNU General Public License along with
15! PALM. If not, see <http://www.gnu.org/licenses/>.
16!
17! Copyright 1997-2017 Leibniz Universitaet Hannover
18!------------------------------------------------------------------------------!
19!
20! Current revisions:
21! ------------------
22!
23!
24! Former revisions:
25! -----------------
26! $Id: time_integration.f90 2271 2017-06-09 12:34:55Z sward $
27! Start timestep message changed
28!
29! 2259 2017-06-08 09:09:11Z gronemeier
30! Implemented synthetic turbulence generator
31!
32! 2233 2017-05-30 18:08:54Z suehring
33!
34! 2232 2017-05-30 17:47:52Z suehring
35! Adjustments to new topography and surface concept
36! Modify passed parameters for disturb_field
37!
38! 2178 2017-03-17 11:07:39Z hellstea
39! Setting perturbations at all times near inflow boundary is removed
40! in case of nested boundaries
41!
42! 2174 2017-03-13 08:18:57Z maronga
43! Added support for nesting with cloud microphysics
44!
45! 2118 2017-01-17 16:38:49Z raasch
46! OpenACC directives and related code removed
47!
48! 2050 2016-11-08 15:00:55Z gronemeier
49! Implement turbulent outflow condition
50!
51! 2031 2016-10-21 15:11:58Z knoop
52! renamed variable rho to rho_ocean
53!
54! 2011 2016-09-19 17:29:57Z kanani
55! Flag urban_surface is now defined in module control_parameters,
56! removed commented CALLs of global_min_max.
57!
58! 2007 2016-08-24 15:47:17Z kanani
59! Added CALLs for new urban surface model
60!
61! 2000 2016-08-20 18:09:15Z knoop
62! Forced header and separation lines into 80 columns
63!
64! 1976 2016-07-27 13:28:04Z maronga
65! Simplified calls to radiation model
66!
67! 1960 2016-07-12 16:34:24Z suehring
68! Separate humidity and passive scalar
69!
70! 1957 2016-07-07 10:43:48Z suehring
71! flight module added
72!
73! 1919 2016-05-27 14:51:23Z raasch
74! Initial version of purely vertical nesting introduced.
75!
76! 1918 2016-05-27 14:35:57Z raasch
77! determination of time step moved to the end of the time step loop,
78! the first time step is now always calculated before the time step loop (i.e.
79! also in case of restart runs)
80!
81! 1914 2016-05-26 14:44:07Z witha
82! Added call for wind turbine model
83!
84! 1878 2016-04-19 12:30:36Z hellstea
85! Synchronization for nested runs rewritten
86!
87! 1853 2016-04-11 09:00:35Z maronga
88! Adjusted for use with radiation_scheme = constant
89!
90! 1849 2016-04-08 11:33:18Z hoffmann
91! Adapted for modularization of microphysics
92!
93! 1833 2016-04-07 14:23:03Z raasch
94! spectrum renamed spectra_mod, spectra related variables moved to spectra_mod
95!
96! 1831 2016-04-07 13:15:51Z hoffmann
97! turbulence renamed collision_turbulence
98!
99! 1822 2016-04-07 07:49:42Z hoffmann
100! icloud_scheme replaced by microphysics_*
101!
102! 1808 2016-04-05 19:44:00Z raasch
103! output message in case unscheduled radiation calls removed
104!
105! 1797 2016-03-21 16:50:28Z raasch
106! introduction of different datatransfer modes
107!
108! 1791 2016-03-11 10:41:25Z raasch
109! call of pmci_update_new removed
110!
111! 1786 2016-03-08 05:49:27Z raasch
112! +module spectrum
113!
114! 1783 2016-03-06 18:36:17Z raasch
115! switch back of netcdf data format for mask output moved to the mask output
116! routine
117!
118! 1781 2016-03-03 15:12:23Z raasch
119! some pmc calls removed at the beginning (before timeloop),
120! pmc initialization moved to the main program
121!
122! 1764 2016-02-28 12:45:19Z raasch
123! PMC_ACTIVE flags removed,
124! bugfix: nest synchronization after first call of timestep
125!
126! 1762 2016-02-25 12:31:13Z hellstea
127! Introduction of nested domain feature
128!
129! 1736 2015-12-04 08:56:33Z raasch
130! no perturbations added to total domain if energy limit has been set zero
131!
132! 1691 2015-10-26 16:17:44Z maronga
133! Added option for spin-ups without land surface and radiation models. Moved calls
134! for radiation and lan surface schemes.
135!
136! 1682 2015-10-07 23:56:08Z knoop
137! Code annotations made doxygen readable
138!
139! 1671 2015-09-25 03:29:37Z raasch
140! bugfix: ghostpoint exchange for array diss in case that sgs velocities are used
141! for particles
142!
143! 1585 2015-04-30 07:05:52Z maronga
144! Moved call of radiation scheme. Added support for RRTM
145!
146! 1551 2015-03-03 14:18:16Z maronga
147! Added interface for different radiation schemes.
148!
149! 1496 2014-12-02 17:25:50Z maronga
150! Added calls for the land surface model and radiation scheme
151!
152! 1402 2014-05-09 14:25:13Z raasch
153! location messages modified
154!
155! 1384 2014-05-02 14:31:06Z raasch
156! location messages added
157!
158! 1380 2014-04-28 12:40:45Z heinze
159! CALL of nudge_ref added
160! bc_pt_t_val and bc_q_t_val are updated in case nudging is used
161!
162! 1365 2014-04-22 15:03:56Z boeske
163! Reset sums_ls_l to zero at each timestep
164! +sums_ls_l
165! Calculation of reference state (previously in subroutine calc_mean_profile)
166
167! 1342 2014-03-26 17:04:47Z kanani
168! REAL constants defined as wp-kind
169!
170! 1320 2014-03-20 08:40:49Z raasch
171! ONLY-attribute added to USE-statements,
172! kind-parameters added to all INTEGER and REAL declaration statements,
173! kinds are defined in new module kinds,
174! old module precision_kind is removed,
175! revision history before 2012 removed,
176! comment fields (!:) to be used for variable explanations added to
177! all variable declaration statements
178! 1318 2014-03-17 13:35:16Z raasch
179! module interfaces removed
180!
181! 1308 2014-03-13 14:58:42Z fricke
182! +netcdf_data_format_save
183! For masked data, parallel netcdf output is not tested so far, hence
184! netcdf_data_format is switched back to non-paralell output.
185!
186! 1276 2014-01-15 13:40:41Z heinze
187! Use LSF_DATA also in case of Dirichlet bottom boundary condition for scalars
188!
189! 1257 2013-11-08 15:18:40Z raasch
190! acc-update-host directive for timestep removed
191!
192! 1241 2013-10-30 11:36:58Z heinze
193! Generalize calc_mean_profile for wider use
194! Determine shf and qsws in dependence on data from LSF_DATA
195! Determine ug and vg in dependence on data from LSF_DATA
196! 1221 2013-09-10 08:59:13Z raasch
197! host update of arrays before timestep is called
198!
199! 1179 2013-06-14 05:57:58Z raasch
200! mean profiles for reference state are only calculated if required,
201! small bugfix for background communication
202!
203! 1171 2013-05-30 11:27:45Z raasch
204! split of prognostic_equations deactivated (comment lines), for the time being
205!
206! 1128 2013-04-12 06:19:32Z raasch
207! asynchronous transfer of ghost point data realized for acc-optimized version:
208! prognostic_equations are first called two times for those points required for
209! the left-right and north-south exchange, respectively, and then for the
210! remaining points,
211! those parts requiring global communication moved from prognostic_equations to
212! here
213!
214! 1115 2013-03-26 18:16:16Z hoffmann
215! calculation of qr and nr is restricted to precipitation
216!
217! 1113 2013-03-10 02:48:14Z raasch
218! GPU-porting of boundary conditions,
219! openACC directives updated
220! formal parameter removed from routine boundary_conds
221!
222! 1111 2013-03-08 23:54:10Z raasch
223! +internal timestep counter for cpu statistics added,
224! openACC directives updated
225!
226! 1092 2013-02-02 11:24:22Z raasch
227! unused variables removed
228!
229! 1065 2012-11-22 17:42:36Z hoffmann
230! exchange of diss (dissipation rate) in case of turbulence = .TRUE. added
231!
232! 1053 2012-11-13 17:11:03Z hoffmann
233! exchange of ghost points for nr, qr added
234!
235! 1036 2012-10-22 13:43:42Z raasch
236! code put under GPL (PALM 3.9)
237!
238! 1019 2012-09-28 06:46:45Z raasch
239! non-optimized version of prognostic_equations removed
240!
241! 1015 2012-09-27 09:23:24Z raasch
242! +call of prognostic_equations_acc
243!
244! 1001 2012-09-13 14:08:46Z raasch
245! all actions concerning leapfrog- and upstream-spline-scheme removed
246!
247! 849 2012-03-15 10:35:09Z raasch
248! advec_particles renamed lpm, first_call_advec_particles renamed first_call_lpm
249!
250! 825 2012-02-19 03:03:44Z raasch
251! wang_collision_kernel renamed wang_kernel
252!
253! Revision 1.1  1997/08/11 06:19:04  raasch
254! Initial revision
255!
256!
257! Description:
258! ------------
259!> Integration in time of the model equations, statistical analysis and graphic
260!> output
261!------------------------------------------------------------------------------!
262 SUBROUTINE time_integration
263 
264
265    USE advec_ws,                                                              &
266        ONLY:  ws_statistics
267
268    USE arrays_3d,                                                             &
269        ONLY:  diss, dzu, e, e_p, nr, nr_p, prho, pt, pt_p, pt_init, q_init, q,&
270               ql, ql_c, ql_v, ql_vp, qr, qr_p, q_p, ref_state, rho_ocean,     &
271               s, s_p, sa_p, tend, u, u_p, v, vpt, v_p, w, w_p
272
273    USE calc_mean_profile_mod,                                                 &
274        ONLY:  calc_mean_profile
275
276    USE control_parameters,                                                    &
277        ONLY:  advected_distance_x, advected_distance_y, average_count_3d,     &
278               averaging_interval, averaging_interval_pr,                      &
279               bc_lr_cyc, bc_ns_cyc, bc_pt_t_val,                              &
280               bc_q_t_val, call_psolver_at_all_substeps, cloud_droplets,       &
281               cloud_physics, constant_flux_layer, constant_heatflux,          &
282               create_disturbances, dopr_n, constant_diffusion, coupling_mode, &
283               coupling_start_time, current_timestep_number,                   &
284               disturbance_created, disturbance_energy_limit, dist_range,      &
285               do_sum, dt_3d, dt_averaging_input, dt_averaging_input_pr,       &
286               dt_coupling, dt_data_output_av, dt_disturb, dt_do2d_xy,         &
287               dt_do2d_xz, dt_do2d_yz, dt_do3d, dt_domask,dt_dopts, dt_dopr,   &
288               dt_dopr_listing, dt_dots, dt_dvrp, dt_run_control,              &
289               end_time, first_call_lpm, galilei_transformation, humidity,     &
290               intermediate_timestep_count, intermediate_timestep_count_max,   &
291               land_surface, large_scale_forcing,                              &
292               loop_optimization, lsf_surf, lsf_vert, masks,                   &
293               microphysics_seifert, mid, nest_domain,                         &
294               neutral, nr_timesteps_this_run, nudging,                        &
295               ocean, passive_scalar,                                          &
296               prho_reference, pt_reference, pt_slope_offset, random_heatflux, &
297               run_coupled, simulated_time, simulated_time_chr,                &
298               skip_time_do2d_xy, skip_time_do2d_xz, skip_time_do2d_yz,        &
299               skip_time_do3d, skip_time_domask, skip_time_dopr,               &
300               skip_time_data_output_av, sloping_surface,                      &
301               stop_dt, terminate_coupled, terminate_run, timestep_scheme,     &
302               time_coupling, time_do2d_xy, time_do2d_xz, time_do2d_yz,        &
303               time_do3d, time_domask, time_dopr, time_dopr_av,                &
304               time_dopr_listing, time_dopts, time_dosp, time_dosp_av,         &
305               time_dots, time_do_av, time_do_sla, time_disturb, time_dvrp,    &
306               time_run_control, time_since_reference_point,                   &
307               turbulent_inflow, turbulent_outflow, urban_surface,             &
308               use_initial_profile_as_reference,                               &
309               use_single_reference_value, u_gtrans, v_gtrans, virtual_flight, &
310               ws_scheme_mom, ws_scheme_sca
311
312    USE cpulog,                                                                &
313        ONLY:  cpu_log, log_point, log_point_s
314
315    USE flight_mod,                                                            &
316        ONLY:  flight_measurement
317
318
319    USE indices,                                                               &
320        ONLY:  nbgp, nx, nxl, nxlg, nxr, nxrg, nyn, nyng, nys, nysg, nzb, nzt
321
322    USE interaction_droplets_ptq_mod,                                          &
323        ONLY:  interaction_droplets_ptq
324
325    USE interfaces
326
327    USE kinds
328
329    USE land_surface_model_mod,                                                &
330        ONLY:  lsm_energy_balance, lsm_soil_model,                             &
331               skip_time_do_lsm
332
333    USE ls_forcing_mod,                                                        &
334        ONLY:  ls_forcing_surf, ls_forcing_vert
335
336    USE microphysics_mod,                                                      &
337        ONLY: collision_turbulence
338
339    USE nudge_mod,                                                             &
340        ONLY:  calc_tnudge, nudge_ref
341
342    USE particle_attributes,                                                   &
343        ONLY:  particle_advection, particle_advection_start,                   &
344               use_sgs_for_particles, wang_kernel
345
346    USE pegrid
347
348    USE pmc_interface,                                                         &
349        ONLY:  nested_run, nesting_mode, pmci_datatrans,                       &
350               pmci_ensure_nest_mass_conservation, pmci_synchronize
351
352    USE production_e_mod,                                                      &
353        ONLY:  production_e_init
354
355    USE progress_bar,                                                          &
356        ONLY:  finish_progress_bar, output_progress_bar
357
358    USE prognostic_equations_mod,                                              &
359        ONLY:  prognostic_equations_cache, prognostic_equations_vector
360
361    USE radiation_model_mod,                                                   &
362        ONLY: dt_radiation, force_radiation_call, radiation, radiation_control,&
363              skip_time_do_radiation, time_radiation
364
365    USE spectra_mod,                                                           &
366        ONLY: average_count_sp, averaging_interval_sp, calc_spectra, dt_dosp,  &
367              skip_time_dosp
368
369    USE statistics,                                                            &
370        ONLY:  flow_statistics_called, hom, pr_palm, sums_ls_l, u_max,         &
371               u_max_ijk, v_max, v_max_ijk, w_max, w_max_ijk
372
373    USE surface_layer_fluxes_mod,                                              &
374        ONLY:  surface_layer_fluxes
375
376    USE surface_mod,                                                           &
377        ONLY:  surf_def_h, surf_lsm_h, surf_usm_h
378
379    USE urban_surface_mod,                                                     &
380        ONLY:  usm_material_heat_model, usm_material_model,                    &
381               usm_radiation, usm_surface_energy_balance
382
383    USE synthetic_turbulence_generator_mod,                                    &
384        ONLY:  stg_main, use_synthetic_turbulence_generator
385
386    USE user_actions_mod,                                                      &
387        ONLY:  user_actions
388
389    USE wind_turbine_model_mod,                                                &
390        ONLY:  wind_turbine, wtm_forces
391
392    IMPLICIT NONE
393
394    CHARACTER (LEN=9) ::  time_to_string          !<
395
396    REAL(wp) ::  dt_3d_old  !< temporary storage of timestep to be used for
397                            !< steering of run control output interval
398
399!
400!-- At beginning determine the first time step
401    CALL timestep
402
403!
404!-- Synchronize the timestep in case of nested run.
405    IF ( nested_run )  THEN
406!
407!--    Synchronization by unifying the time step.
408!--    Global minimum of all time-steps is used for all.
409       CALL pmci_synchronize
410    ENDIF
411
412!
413!-- Determine and print out the run control quantities before the first time
414!-- step of this run. For the initial run, some statistics (e.g. divergence)
415!-- need to be determined first.
416    IF ( simulated_time == 0.0_wp )  CALL flow_statistics
417    CALL run_control
418
419!
420!-- Data exchange between coupled models in case that a call has been omitted
421!-- at the end of the previous run of a job chain.
422    IF ( coupling_mode /= 'uncoupled'  .AND.  run_coupled )  THEN
423!
424!--    In case of model termination initiated by the local model the coupler
425!--    must not be called because this would again cause an MPI hang.
426       DO WHILE ( time_coupling >= dt_coupling  .AND.  terminate_coupled == 0 )
427          CALL surface_coupler
428          time_coupling = time_coupling - dt_coupling
429       ENDDO
430       IF (time_coupling == 0.0_wp  .AND.                                      &
431           time_since_reference_point < dt_coupling )                          &
432       THEN
433          time_coupling = time_since_reference_point
434       ENDIF
435    ENDIF
436
437#if defined( __dvrp_graphics )
438!
439!-- Time measurement with dvrp software 
440    CALL DVRP_LOG_EVENT( 2, current_timestep_number )
441#endif
442
443    CALL location_message( 'starting timestep-sequence', .TRUE. )
444!
445!-- Start of the time loop
446    DO  WHILE ( simulated_time < end_time  .AND.  .NOT. stop_dt  .AND. &
447                .NOT. terminate_run )
448
449       CALL cpu_log( log_point_s(10), 'timesteps', 'start' )
450!
451!--    Determine ug, vg and w_subs in dependence on data from external file
452!--    LSF_DATA
453       IF ( large_scale_forcing .AND. lsf_vert )  THEN
454           CALL ls_forcing_vert ( simulated_time )
455           sums_ls_l = 0.0_wp
456       ENDIF
457
458!
459!--    Set pt_init and q_init to the current profiles taken from
460!--    NUDGING_DATA
461       IF ( nudging )  THEN
462           CALL nudge_ref ( simulated_time )
463!
464!--        Store temperature gradient at the top boundary for possible Neumann
465!--        boundary condition
466           bc_pt_t_val = ( pt_init(nzt+1) - pt_init(nzt) ) / dzu(nzt+1)
467           bc_q_t_val  = ( q_init(nzt+1) - q_init(nzt) ) / dzu(nzt+1)
468       ENDIF
469
470!
471!--    Execute the user-defined actions
472       CALL user_actions( 'before_timestep' )
473
474!
475!--    Calculate forces by wind turbines
476       IF ( wind_turbine )  THEN
477
478          CALL cpu_log( log_point(55), 'wind_turbine', 'start' )
479
480          CALL wtm_forces
481
482          CALL cpu_log( log_point(55), 'wind_turbine', 'stop' )
483
484       ENDIF       
485       
486!
487!--    Start of intermediate step loop
488       intermediate_timestep_count = 0
489       DO  WHILE ( intermediate_timestep_count < &
490                   intermediate_timestep_count_max )
491
492          intermediate_timestep_count = intermediate_timestep_count + 1
493
494!
495!--       Set the steering factors for the prognostic equations which depend
496!--       on the timestep scheme
497          CALL timestep_scheme_steering
498
499!
500!--       Calculate those variables needed in the tendency terms which need
501!--       global communication
502          IF ( .NOT. use_single_reference_value  .AND. &
503               .NOT. use_initial_profile_as_reference )  THEN
504!
505!--          Horizontally averaged profiles to be used as reference state in
506!--          buoyancy terms (WARNING: only the respective last call of
507!--          calc_mean_profile defines the reference state!)
508             IF ( .NOT. neutral )  THEN
509                CALL calc_mean_profile( pt, 4 )
510                ref_state(:)  = hom(:,1,4,0) ! this is used in the buoyancy term
511             ENDIF
512             IF ( ocean )  THEN
513                CALL calc_mean_profile( rho_ocean, 64 )
514                ref_state(:)  = hom(:,1,64,0)
515             ENDIF
516             IF ( humidity )  THEN
517                CALL calc_mean_profile( vpt, 44 )
518                ref_state(:)  = hom(:,1,44,0)
519             ENDIF
520
521          ENDIF
522
523          IF ( .NOT. constant_diffusion )  CALL production_e_init
524          IF ( ( ws_scheme_mom .OR. ws_scheme_sca )  .AND.  &
525               intermediate_timestep_count == 1 )  CALL ws_statistics
526!
527!--       In case of nudging calculate current nudging time scale and horizontal
528!--       means of u, v, pt and q
529          IF ( nudging )  THEN
530             CALL calc_tnudge( simulated_time )
531             CALL calc_mean_profile( u, 1 )
532             CALL calc_mean_profile( v, 2 )
533             CALL calc_mean_profile( pt, 4 )
534             CALL calc_mean_profile( q, 41 )
535          ENDIF
536
537!
538!--       Solve the prognostic equations. A fast cache optimized version with
539!--       only one single loop is used in case of Piascek-Williams advection
540!--       scheme. NEC vector machines use a different version, because
541!--       in the other versions a good vectorization is prohibited due to
542!--       inlining problems.
543          IF ( loop_optimization == 'cache' )  THEN
544             CALL prognostic_equations_cache
545          ELSEIF ( loop_optimization == 'vector' )  THEN
546             CALL prognostic_equations_vector
547          ENDIF
548
549!
550!--       Particle transport/physics with the Lagrangian particle model
551!--       (only once during intermediate steps, because it uses an Euler-step)
552!--       ### particle model should be moved before prognostic_equations, in order
553!--       to regard droplet interactions directly
554          IF ( particle_advection  .AND.                         &
555               simulated_time >= particle_advection_start  .AND. &
556               intermediate_timestep_count == 1 )  THEN
557             CALL lpm
558             first_call_lpm = .FALSE.
559          ENDIF
560
561!
562!--       Interaction of droplets with temperature and specific humidity.
563!--       Droplet condensation and evaporation is calculated within
564!--       advec_particles.
565          IF ( cloud_droplets  .AND.  &
566               intermediate_timestep_count == intermediate_timestep_count_max )&
567          THEN
568             CALL interaction_droplets_ptq
569          ENDIF
570
571!
572!--       Exchange of ghost points (lateral boundary conditions)
573          CALL cpu_log( log_point(26), 'exchange-horiz-progn', 'start' )
574
575          CALL exchange_horiz( u_p, nbgp )
576          CALL exchange_horiz( v_p, nbgp )
577          CALL exchange_horiz( w_p, nbgp )
578          CALL exchange_horiz( pt_p, nbgp )
579          IF ( .NOT. constant_diffusion )  CALL exchange_horiz( e_p, nbgp )
580          IF ( ocean )  THEN
581             CALL exchange_horiz( sa_p, nbgp )
582             CALL exchange_horiz( rho_ocean, nbgp )
583             CALL exchange_horiz( prho, nbgp )
584          ENDIF
585          IF ( humidity )  THEN
586             CALL exchange_horiz( q_p, nbgp )
587             IF ( cloud_physics .AND. microphysics_seifert )  THEN
588                CALL exchange_horiz( qr_p, nbgp )
589                CALL exchange_horiz( nr_p, nbgp )
590             ENDIF
591          ENDIF
592          IF ( cloud_droplets )  THEN
593             CALL exchange_horiz( ql, nbgp )
594             CALL exchange_horiz( ql_c, nbgp )
595             CALL exchange_horiz( ql_v, nbgp )
596             CALL exchange_horiz( ql_vp, nbgp )
597          ENDIF
598          IF ( wang_kernel  .OR.  collision_turbulence  .OR.                &
599               use_sgs_for_particles )  THEN
600             CALL exchange_horiz( diss, nbgp )
601          ENDIF
602          IF ( passive_scalar )  CALL exchange_horiz( s_p, nbgp )
603
604          CALL cpu_log( log_point(26), 'exchange-horiz-progn', 'stop' )
605
606!
607!--       Boundary conditions for the prognostic quantities (except of the
608!--       velocities at the outflow in case of a non-cyclic lateral wall)
609          CALL boundary_conds
610!
611!--       Swap the time levels in preparation for the next time step.
612          CALL swap_timelevel
613
614          IF ( nested_run )  THEN
615
616             CALL cpu_log( log_point(60), 'nesting', 'start' )
617!
618!--          Domain nesting. The data transfer subroutines pmci_parent_datatrans
619!--          and pmci_child_datatrans are called inside the wrapper
620!--          subroutine pmci_datatrans according to the control parameters
621!--          nesting_mode and nesting_datatransfer_mode.
622!--          TO_DO: why is nesting_mode given as a parameter here?
623             CALL pmci_datatrans( nesting_mode )
624
625             IF ( TRIM( nesting_mode ) == 'two-way' .OR.                       &
626                  nesting_mode == 'vertical' )  THEN
627!
628!--             Exchange_horiz is needed for all parent-domains after the
629!--             anterpolation
630                CALL exchange_horiz( u, nbgp )
631                CALL exchange_horiz( v, nbgp )
632                CALL exchange_horiz( w, nbgp )
633                IF ( .NOT. neutral )  CALL exchange_horiz( pt, nbgp )
634
635                IF ( humidity )  THEN
636
637                   CALL exchange_horiz( q, nbgp )
638
639                   IF ( cloud_physics  .AND.  microphysics_seifert )  THEN
640!                        CALL exchange_horiz( qc, nbgp )
641                       CALL exchange_horiz( qr, nbgp )
642!                        CALL exchange_horiz( nc, nbgp )
643                       CALL exchange_horiz( nr, nbgp )
644                   ENDIF
645
646                ENDIF
647
648                IF ( passive_scalar )  CALL exchange_horiz( s, nbgp )
649                IF ( .NOT. constant_diffusion )  CALL exchange_horiz( e, nbgp )
650             ENDIF
651!
652!--          Correct the w top-BC in nest domains to ensure mass conservation.
653!--          This action must never be done for the root domain. Vertical
654!--          nesting implies mass conservation.
655             IF ( nest_domain )  THEN
656                CALL pmci_ensure_nest_mass_conservation
657             ENDIF
658
659             CALL cpu_log( log_point(60), 'nesting', 'stop' )
660
661          ENDIF
662
663!
664!--       Temperature offset must be imposed at cyclic boundaries in x-direction
665!--       when a sloping surface is used
666          IF ( sloping_surface )  THEN
667             IF ( nxl ==  0 )  pt(:,:,nxlg:nxl-1) = pt(:,:,nxlg:nxl-1) - &
668                                                    pt_slope_offset
669             IF ( nxr == nx )  pt(:,:,nxr+1:nxrg) = pt(:,:,nxr+1:nxrg) + &
670                                                    pt_slope_offset
671          ENDIF
672
673!
674!--       Impose a turbulent inflow using the recycling method
675          IF ( turbulent_inflow )  CALL  inflow_turbulence
676
677!
678!--       Impose a turbulent inflow using synthetic generated turbulence
679         IF ( use_synthetic_turbulence_generator ) THEN
680            CALL  stg_main
681         ENDIF
682
683!
684!--       Set values at outflow boundary using the special outflow condition
685          IF ( turbulent_outflow )  CALL  outflow_turbulence
686
687!
688!--       Impose a random perturbation on the horizontal velocity field
689          IF ( create_disturbances  .AND.                                      &
690               ( call_psolver_at_all_substeps  .AND.                           &
691               intermediate_timestep_count == intermediate_timestep_count_max )&
692          .OR. ( .NOT. call_psolver_at_all_substeps  .AND.                     &
693               intermediate_timestep_count == 1 ) )                            &
694          THEN
695             time_disturb = time_disturb + dt_3d
696             IF ( time_disturb >= dt_disturb )  THEN
697                IF ( disturbance_energy_limit /= 0.0_wp  .AND.                 &
698                     hom(nzb+5,1,pr_palm,0) < disturbance_energy_limit )  THEN
699                   CALL disturb_field( 'u', tend, u )
700                   CALL disturb_field( 'v', tend, v )
701                ELSEIF ( ( .NOT. bc_lr_cyc  .OR.  .NOT. bc_ns_cyc )            &
702                     .AND. .NOT. nest_domain )  THEN
703!
704!--                Runs with a non-cyclic lateral wall need perturbations
705!--                near the inflow throughout the whole simulation
706                   dist_range = 1
707                   CALL disturb_field( 'u', tend, u )
708                   CALL disturb_field( 'v', tend, v )
709                   dist_range = 0
710                ENDIF
711                time_disturb = time_disturb - dt_disturb
712             ENDIF
713          ENDIF
714
715!
716!--       Reduce the velocity divergence via the equation for perturbation
717!--       pressure.
718          IF ( intermediate_timestep_count == 1  .OR. &
719                call_psolver_at_all_substeps )  THEN
720             CALL pres
721          ENDIF
722
723!
724!--       If required, compute liquid water content
725          IF ( cloud_physics )  THEN
726             CALL calc_liquid_water_content
727          ENDIF
728!
729!--       If required, compute virtual potential temperature
730          IF ( humidity )  THEN
731             CALL compute_vpt
732          ENDIF
733
734!
735!--       Compute the diffusion quantities
736          IF ( .NOT. constant_diffusion )  THEN
737
738!
739!--          Determine surface fluxes shf and qsws and surface values
740!--          pt_surface and q_surface in dependence on data from external
741!--          file LSF_DATA respectively
742             IF ( ( large_scale_forcing .AND. lsf_surf ) .AND. &
743                 intermediate_timestep_count == intermediate_timestep_count_max )&
744             THEN
745                CALL ls_forcing_surf ( simulated_time )
746             ENDIF
747
748!
749!--          First the vertical (and horizontal) fluxes in the surface
750!--          (constant flux) layer are computed
751             IF ( constant_flux_layer )  THEN
752                CALL cpu_log( log_point(19), 'surface_layer_fluxes', 'start' )
753                CALL surface_layer_fluxes
754                CALL cpu_log( log_point(19), 'surface_layer_fluxes', 'stop' )
755             ENDIF
756
757!
758!--          If required, solve the energy balance for the surface and run soil
759!--          model. Call for horizontal as well as vertical surfaces
760             IF ( land_surface .AND. simulated_time > skip_time_do_lsm)  THEN
761
762                CALL cpu_log( log_point(54), 'land_surface', 'start' )
763!
764!--             Call for horizontal upward-facing surfaces
765                CALL lsm_energy_balance( .TRUE., -1 )
766                CALL lsm_soil_model( .TRUE., -1 )
767!
768!--             Call for northward-facing surfaces
769                CALL lsm_energy_balance( .FALSE., 0 )
770                CALL lsm_soil_model( .FALSE., 0 )
771!
772!--             Call for southward-facing surfaces
773                CALL lsm_energy_balance( .FALSE., 1 )
774                CALL lsm_soil_model( .FALSE., 1 )
775!
776!--             Call for eastward-facing surfaces
777                CALL lsm_energy_balance( .FALSE., 2 )
778                CALL lsm_soil_model( .FALSE., 2 )
779!
780!--             Call for westward-facing surfaces
781                CALL lsm_energy_balance( .FALSE., 3 )
782                CALL lsm_soil_model( .FALSE., 3 )
783
784                CALL cpu_log( log_point(54), 'land_surface', 'stop' )
785             ENDIF
786
787!
788!--          If required, solve the energy balance for urban surfaces and run
789!--          the material heat model
790             IF (urban_surface) THEN
791                CALL cpu_log( log_point(74), 'urban_surface', 'start' )
792                CALL usm_surface_energy_balance
793                IF ( usm_material_model )  THEN
794                   CALL usm_material_heat_model
795                ENDIF
796                CALL cpu_log( log_point(74), 'urban_surface', 'stop' )
797             ENDIF
798
799!
800!--          Compute the diffusion coefficients
801             CALL cpu_log( log_point(17), 'diffusivities', 'start' )
802             IF ( .NOT. humidity ) THEN
803                IF ( ocean )  THEN
804                   CALL diffusivities( prho, prho_reference )
805                ELSE
806                   CALL diffusivities( pt, pt_reference )
807                ENDIF
808             ELSE
809                CALL diffusivities( vpt, pt_reference )
810             ENDIF
811
812             CALL cpu_log( log_point(17), 'diffusivities', 'stop' )
813
814          ENDIF
815
816!
817!--       If required, calculate radiative fluxes and heating rates
818          IF ( radiation .AND. intermediate_timestep_count                     &
819               == intermediate_timestep_count_max .AND. simulated_time >    &
820               skip_time_do_radiation )  THEN
821
822               time_radiation = time_radiation + dt_3d
823
824             IF ( time_radiation >= dt_radiation .OR. force_radiation_call )   &
825             THEN
826
827                CALL cpu_log( log_point(50), 'radiation', 'start' )
828
829                IF ( .NOT. force_radiation_call )  THEN
830                   time_radiation = time_radiation - dt_radiation
831                ENDIF
832
833                CALL radiation_control
834
835                CALL cpu_log( log_point(50), 'radiation', 'stop' )
836
837                IF (urban_surface)  THEN
838                   CALL cpu_log( log_point(75), 'usm_radiation', 'start' )
839                   CALL usm_radiation
840                   CALL cpu_log( log_point(75), 'usm_radiation', 'stop' )
841                ENDIF
842
843             ENDIF
844          ENDIF
845
846       ENDDO   ! Intermediate step loop
847
848!
849!--    Increase simulation time and output times
850       nr_timesteps_this_run      = nr_timesteps_this_run + 1
851       current_timestep_number    = current_timestep_number + 1
852       simulated_time             = simulated_time   + dt_3d
853       simulated_time_chr         = time_to_string( simulated_time )
854       time_since_reference_point = simulated_time - coupling_start_time
855
856
857
858       IF ( simulated_time >= skip_time_data_output_av )  THEN
859          time_do_av         = time_do_av       + dt_3d
860       ENDIF
861       IF ( simulated_time >= skip_time_do2d_xy )  THEN
862          time_do2d_xy       = time_do2d_xy     + dt_3d
863       ENDIF
864       IF ( simulated_time >= skip_time_do2d_xz )  THEN
865          time_do2d_xz       = time_do2d_xz     + dt_3d
866       ENDIF
867       IF ( simulated_time >= skip_time_do2d_yz )  THEN
868          time_do2d_yz       = time_do2d_yz     + dt_3d
869       ENDIF
870       IF ( simulated_time >= skip_time_do3d    )  THEN
871          time_do3d          = time_do3d        + dt_3d
872       ENDIF
873       DO  mid = 1, masks
874          IF ( simulated_time >= skip_time_domask(mid) )  THEN
875             time_domask(mid)= time_domask(mid) + dt_3d
876          ENDIF
877       ENDDO
878       time_dvrp          = time_dvrp        + dt_3d
879       IF ( simulated_time >= skip_time_dosp )  THEN
880          time_dosp       = time_dosp        + dt_3d
881       ENDIF
882       time_dots          = time_dots        + dt_3d
883       IF ( .NOT. first_call_lpm )  THEN
884          time_dopts      = time_dopts       + dt_3d
885       ENDIF
886       IF ( simulated_time >= skip_time_dopr )  THEN
887          time_dopr       = time_dopr        + dt_3d
888       ENDIF
889       time_dopr_listing          = time_dopr_listing        + dt_3d
890       time_run_control   = time_run_control + dt_3d
891
892!
893!--    Data exchange between coupled models
894       IF ( coupling_mode /= 'uncoupled'  .AND.  run_coupled )  THEN
895          time_coupling = time_coupling + dt_3d
896
897!
898!--       In case of model termination initiated by the local model
899!--       (terminate_coupled > 0), the coupler must be skipped because it would
900!--       cause an MPI intercomminucation hang.
901!--       If necessary, the coupler will be called at the beginning of the
902!--       next restart run.
903          DO WHILE ( time_coupling >= dt_coupling .AND. terminate_coupled == 0 )
904             CALL surface_coupler
905             time_coupling = time_coupling - dt_coupling
906          ENDDO
907       ENDIF
908
909!
910!--    Execute user-defined actions
911       CALL user_actions( 'after_integration' )
912
913!
914!--    If Galilei transformation is used, determine the distance that the
915!--    model has moved so far
916       IF ( galilei_transformation )  THEN
917          advected_distance_x = advected_distance_x + u_gtrans * dt_3d
918          advected_distance_y = advected_distance_y + v_gtrans * dt_3d
919       ENDIF
920
921!
922!--    Check, if restart is necessary (because cpu-time is expiring or
923!--    because it is forced by user) and set stop flag
924!--    This call is skipped if the remote model has already initiated a restart.
925       IF ( .NOT. terminate_run )  CALL check_for_restart
926
927!
928!--    Carry out statistical analysis and output at the requested output times.
929!--    The MOD function is used for calculating the output time counters (like
930!--    time_dopr) in order to regard a possible decrease of the output time
931!--    interval in case of restart runs
932
933!
934!--    Set a flag indicating that so far no statistics have been created
935!--    for this time step
936       flow_statistics_called = .FALSE.
937
938!
939!--    If required, call flow_statistics for averaging in time
940       IF ( averaging_interval_pr /= 0.0_wp  .AND.  &
941            ( dt_dopr - time_dopr ) <= averaging_interval_pr  .AND.  &
942            simulated_time >= skip_time_dopr )  THEN
943          time_dopr_av = time_dopr_av + dt_3d
944          IF ( time_dopr_av >= dt_averaging_input_pr )  THEN
945             do_sum = .TRUE.
946             time_dopr_av = MOD( time_dopr_av, &
947                                    MAX( dt_averaging_input_pr, dt_3d ) )
948          ENDIF
949       ENDIF
950       IF ( do_sum )  CALL flow_statistics
951
952!
953!--    Sum-up 3d-arrays for later output of time-averaged 2d/3d/masked data
954       IF ( averaging_interval /= 0.0_wp  .AND.                                &
955            ( dt_data_output_av - time_do_av ) <= averaging_interval  .AND. &
956            simulated_time >= skip_time_data_output_av )                    &
957       THEN
958          time_do_sla = time_do_sla + dt_3d
959          IF ( time_do_sla >= dt_averaging_input )  THEN
960             CALL sum_up_3d_data
961             average_count_3d = average_count_3d + 1
962             time_do_sla = MOD( time_do_sla, MAX( dt_averaging_input, dt_3d ) )
963          ENDIF
964       ENDIF
965
966!
967!--    Calculate spectra for time averaging
968       IF ( averaging_interval_sp /= 0.0_wp  .AND.  &
969            ( dt_dosp - time_dosp ) <= averaging_interval_sp  .AND.  &
970            simulated_time >= skip_time_dosp )  THEN
971          time_dosp_av = time_dosp_av + dt_3d
972          IF ( time_dosp_av >= dt_averaging_input_pr )  THEN
973             CALL calc_spectra
974             time_dosp_av = MOD( time_dosp_av, &
975                                 MAX( dt_averaging_input_pr, dt_3d ) )
976          ENDIF
977       ENDIF
978
979!
980!--    Call flight module and output data
981       IF ( virtual_flight )  THEN
982          CALL flight_measurement
983          CALL data_output_flight
984       ENDIF
985
986!
987!--    Profile output (ASCII) on file
988       IF ( time_dopr_listing >= dt_dopr_listing )  THEN
989          CALL print_1d
990          time_dopr_listing = MOD( time_dopr_listing, MAX( dt_dopr_listing, &
991                                                           dt_3d ) )
992       ENDIF
993
994!
995!--    Graphic output for PROFIL
996       IF ( time_dopr >= dt_dopr )  THEN
997          IF ( dopr_n /= 0 )  CALL data_output_profiles
998          time_dopr = MOD( time_dopr, MAX( dt_dopr, dt_3d ) )
999          time_dopr_av = 0.0_wp    ! due to averaging (see above)
1000       ENDIF
1001
1002!
1003!--    Graphic output for time series
1004       IF ( time_dots >= dt_dots )  THEN
1005          CALL data_output_tseries
1006          time_dots = MOD( time_dots, MAX( dt_dots, dt_3d ) )
1007       ENDIF
1008
1009!
1010!--    Output of spectra (formatted for use with PROFIL), in case of no
1011!--    time averaging, spectra has to be calculated before
1012       IF ( time_dosp >= dt_dosp )  THEN
1013          IF ( average_count_sp == 0 )  CALL calc_spectra
1014          CALL data_output_spectra
1015          time_dosp = MOD( time_dosp, MAX( dt_dosp, dt_3d ) )
1016       ENDIF
1017
1018!
1019!--    2d-data output (cross-sections)
1020       IF ( time_do2d_xy >= dt_do2d_xy )  THEN
1021          CALL data_output_2d( 'xy', 0 )
1022          time_do2d_xy = MOD( time_do2d_xy, MAX( dt_do2d_xy, dt_3d ) )
1023       ENDIF
1024       IF ( time_do2d_xz >= dt_do2d_xz )  THEN
1025          CALL data_output_2d( 'xz', 0 )
1026          time_do2d_xz = MOD( time_do2d_xz, MAX( dt_do2d_xz, dt_3d ) )
1027       ENDIF
1028       IF ( time_do2d_yz >= dt_do2d_yz )  THEN
1029          CALL data_output_2d( 'yz', 0 )
1030          time_do2d_yz = MOD( time_do2d_yz, MAX( dt_do2d_yz, dt_3d ) )
1031       ENDIF
1032
1033!
1034!--    3d-data output (volume data)
1035       IF ( time_do3d >= dt_do3d )  THEN
1036          CALL data_output_3d( 0 )
1037          time_do3d = MOD( time_do3d, MAX( dt_do3d, dt_3d ) )
1038       ENDIF
1039
1040!
1041!--    Masked data output
1042       DO  mid = 1, masks
1043          IF ( time_domask(mid) >= dt_domask(mid) )  THEN
1044             CALL data_output_mask( 0 )
1045             time_domask(mid) = MOD( time_domask(mid),  &
1046                                     MAX( dt_domask(mid), dt_3d ) )
1047          ENDIF
1048       ENDDO
1049
1050!
1051!--    Output of time-averaged 2d/3d/masked data
1052       IF ( time_do_av >= dt_data_output_av )  THEN
1053          CALL average_3d_data
1054          CALL data_output_2d( 'xy', 1 )
1055          CALL data_output_2d( 'xz', 1 )
1056          CALL data_output_2d( 'yz', 1 )
1057          CALL data_output_3d( 1 )
1058          DO  mid = 1, masks
1059             CALL data_output_mask( 1 )
1060          ENDDO
1061          time_do_av = MOD( time_do_av, MAX( dt_data_output_av, dt_3d ) )
1062       ENDIF
1063
1064!
1065!--    Output of particle time series
1066       IF ( particle_advection )  THEN
1067          IF ( time_dopts >= dt_dopts  .OR. &
1068               ( simulated_time >= particle_advection_start  .AND. &
1069                 first_call_lpm ) )  THEN
1070             CALL data_output_ptseries
1071             time_dopts = MOD( time_dopts, MAX( dt_dopts, dt_3d ) )
1072          ENDIF
1073       ENDIF
1074
1075!
1076!--    Output of dvrp-graphics (isosurface, particles, slicer)
1077#if defined( __dvrp_graphics )
1078       CALL DVRP_LOG_EVENT( -2, current_timestep_number-1 )
1079#endif
1080       IF ( time_dvrp >= dt_dvrp )  THEN
1081          CALL data_output_dvrp
1082          time_dvrp = MOD( time_dvrp, MAX( dt_dvrp, dt_3d ) )
1083       ENDIF
1084#if defined( __dvrp_graphics )
1085       CALL DVRP_LOG_EVENT( 2, current_timestep_number )
1086#endif
1087
1088!
1089!--    If required, set the heat flux for the next time step at a random value
1090       IF ( constant_heatflux  .AND.  random_heatflux )  THEN
1091          IF ( surf_def_h(0)%ns >= 1 )  CALL disturb_heatflux( surf_def_h(0) )
1092          IF ( surf_lsm_h%ns    >= 1 )  CALL disturb_heatflux( surf_lsm_h    )
1093          IF ( surf_usm_h%ns    >= 1 )  CALL disturb_heatflux( surf_usm_h    )
1094       ENDIF
1095
1096!
1097!--    Execute user-defined actions
1098       CALL user_actions( 'after_timestep' )
1099
1100!
1101!--    Determine size of next time step. Save timestep dt_3d because it is
1102!--    newly calculated in routine timestep, but required further below for
1103!--    steering the run control output interval
1104       dt_3d_old = dt_3d
1105       CALL timestep
1106
1107!
1108!--    Synchronize the timestep in case of nested run.
1109       IF ( nested_run )  THEN
1110!
1111!--       Synchronize by unifying the time step.
1112!--       Global minimum of all time-steps is used for all.
1113          CALL pmci_synchronize
1114       ENDIF
1115
1116!
1117!--    Computation and output of run control parameters.
1118!--    This is also done whenever perturbations have been imposed
1119       IF ( time_run_control >= dt_run_control  .OR.                     &
1120            timestep_scheme(1:5) /= 'runge'  .OR.  disturbance_created ) &
1121       THEN
1122          CALL run_control
1123          IF ( time_run_control >= dt_run_control )  THEN
1124             time_run_control = MOD( time_run_control, &
1125                                     MAX( dt_run_control, dt_3d_old ) )
1126          ENDIF
1127       ENDIF
1128
1129!
1130!--    Output elapsed simulated time in form of a progress bar on stdout
1131       IF ( myid == 0 )  CALL output_progress_bar
1132
1133       CALL cpu_log( log_point_s(10), 'timesteps', 'stop' )
1134
1135
1136    ENDDO   ! time loop
1137
1138    IF ( myid == 0 )  CALL finish_progress_bar
1139
1140#if defined( __dvrp_graphics )
1141    CALL DVRP_LOG_EVENT( -2, current_timestep_number )
1142#endif
1143
1144    CALL location_message( 'finished time-stepping', .TRUE. )
1145
1146 END SUBROUTINE time_integration
Note: See TracBrowser for help on using the repository browser.