Home


About us
PALM
PALM-4U
Installation
Release notes
Documentation
Getting Help
Gallery
Developers area

source: palm/trunk/SCRIPTS/.mrun.config.imuk_gpu @ 1119

Last change on this file since 1119 was 1111, checked in by raasch, 12 years ago

New:
---

GPU porting of pres, swap_timelevel. Adjustments of openACC directives.
Further porting of poisfft, which now runs completely on GPU without any
host/device data transfer for serial an parallel runs (but parallel runs
require data transfer before and after the MPI transpositions).
GPU-porting of tridiagonal solver:
tridiagonal routines split into extermal subroutines (instead using CONTAINS),
no distinction between parallel/non-parallel in poisfft and tridia any more,
tridia routines moved to end of file because of probable bug in PGI compiler
(otherwise "invalid device function" is indicated during runtime).
(cuda_fft_interfaces, fft_xy, flow_statistics, init_3d_model, palm, poisfft, pres, prognostic_equations, swap_timelevel, time_integration, transpose)
output of accelerator board information. (header)

optimization of tridia routines: constant elements and coefficients of tri are
stored in seperate arrays ddzuw and tric, last dimension of tri reduced from 5 to 2,
(init_grid, init_3d_model, modules, palm, poisfft)

poisfft_init is now called internally from poisfft,
(Makefile, Makefile_check, init_pegrid, poisfft, poisfft_hybrid)

CPU-time per grid point and timestep is output to CPU_MEASURES file
(cpu_statistics, modules, time_integration)

Changed:

resorting from/to array work changed, work now has 4 dimensions instead of 1 (transpose)
array diss allocated only if required (init_3d_model)

pressure boundary condition "Neumann+inhomo" removed from the code
(check_parameters, header, poisfft, poisfft_hybrid, pres)

Errors:

bugfix: dependency added for cuda_fft_interfaces (Makefile)
bugfix: CUDA fft plans adjusted for domain decomposition (before they always
used total domain) (fft_xy)
  • Property svn:keywords set to Id
File size: 11.8 KB
Line 
1#$Id: .mrun.config.imuk_gpu 1111 2013-03-08 23:54:10Z raasch $
2#column 1          column 2                                   column 3
3#name of variable  value of variable (~ must not be used)     scope
4#----------------------------------------------------------------------------
5%mainprog          palm.f90
6%base_directory    $HOME/palm/current_version
7%base_data         ~/palm/current_version/JOBS
8%source_path       $base_directory/trunk/SOURCE
9%add_source_path   $base_directory/USER_CODE/$fname
10%depository_path   $base_directory/MAKE_DEPOSITORY
11#
12%host_identifier   inferno      lcmuk
13#
14# pure MPI version
15%remote_username   <replace by your IMUK username>               lcmuk parallel pgi
16%tmp_user_catalog  /localdata                                    lcmuk parallel pgi
17%compiler_name     mpif90                                        lcmuk parallel pgi
18%compiler_name_ser pgf90                                         lcmuk parallel pgi
19%cpp_options       -Mpreprocess:-DMPI_REAL=MPI_DOUBLE_PRECISION:-DMPI_2REAL=MPI_2DOUBLE_PRECISION:-D__nopointer   lcmuk parallel pgi
20%mopts             -j:4                                          lcmuk parallel pgi
21%fopts             -Mcray=pointer:-fastsse:-r8                   lcmuk parallel pgi
22%lopts             -Mcray=pointer:-fastsse:-r8                   lcmuk parallel pgi
23#
24# pure MPI version with debug options
25%remote_username   <replace by your IMUK username>               lcmuk parallel pgidbg
26%tmp_user_catalog  /localdata                                    lcmuk parallel pgidbg
27%compiler_name     mpif90                                        lcmuk parallel pgidbg
28%compiler_name_ser pgf90                                         lcmuk parallel pgidbg
29%cpp_options       -Mpreprocess:-DMPI_REAL=MPI_DOUBLE_PRECISION:-DMPI_2REAL=MPI_2DOUBLE_PRECISION:-D__nopointer   lcmuk parallel pgidbg
30%mopts             -j:4                                          lcmuk parallel pgidbg
31%fopts             -Mcray=pointer:-O0:-C:-g:-Mbounds:-Mchkstk:-traceback:-r8   lcmuk parallel pgidbg
32%lopts             -Mcray=pointer:-O0:-C:-g:-Mbounds:-Mchkstk:-traceback:-r8   lcmuk parallel pgidbg
33#
34# pure GPU version
35%remote_username   <replace by your IMUK username>                       lcmuk pgigpu
36%tmp_user_catalog  /localdata                                            lcmuk pgigpu
37%compiler_name     pgf90                                                 lcmuk pgigpu
38%compiler_name_ser pgf90                                                 lcmuk pgigpu
39%cpp_options       -Mpreprocess:-D__nopointer:-D__openacc:-D__cuda_fft   lcmuk pgigpu
40%mopts             -j:4                                                  lcmuk pgigpu
41%fopts             -acc:-ta=nvidia,4.1:-Minfo=acc:-Mcray=pointer:-fastsse:-r8:-Mcuda    lcmuk pgigpu
42%lopts             -acc:-ta=nvidia,4.1:-Minfo=acc:-Mcray=pointer:-fastsse:-r8:-Mcuda:-L/localdata/opt/pgi/linux86-64/2012/cuda/4.1/lib64:-lcufft    lcmuk pgigpu
43#
44# MPI+GPU
45%remote_username   <replace by your IMUK username>               lcmuk parallel pgigpu
46%tmp_user_catalog  /localdata                                    lcmuk parallel pgigpu
47%compiler_name     mpif90                                        lcmuk parallel pgigpu
48%compiler_name_ser pgf90                                         lcmuk parallel pgigpu
49%cpp_options       -Mpreprocess:-DMPI_REAL=MPI_DOUBLE_PRECISION:-DMPI_2REAL=MPI_2DOUBLE_PRECISION:-D__nopointer:-D__openacc:-D__cuda_fft   lcmuk parallel pgigpu
50%mopts             -j:4                                          lcmuk parallel pgigpu
51%fopts             -acc:-ta=nvidia,4.1:-Minfo=acc:-Mcray=pointer:-fastsse:-r8:-Mcuda    lcmuk parallel pgigpu
52%lopts             -acc:-ta=nvidia,4.1:-Minfo=acc:-Mcray=pointer:-fastsse:-r8:-Mcuda:-L/localdata/opt/pgi/linux86-64/2012/cuda/4.1/lib64:-lcufft   lcmuk parallel pgigpu
53#
54%write_binary                true                             restart
55#
56#----------------------------------------------------------------------------
57# INPUT-commands, executed before running PALM - lines must start with "IC:"
58#----------------------------------------------------------------------------
59#IC:
60#
61#----------------------------------------------------------------------------
62# ERROR-commands - executed when program terminates abnormally
63#----------------------------------------------------------------------------
64EC:[[ \$locat = execution ]]  &&  cat  RUN_CONTROL
65EC:pwd; ls -al
66#
67#----------------------------------------------------------------------------
68# OUTPUT-commands - executed when program terminates normally
69#----------------------------------------------------------------------------
70#
71# Combine 1D- and 3D-profile output (these files are not usable for plotting)
72OC:[[ -f LIST_PROFIL_1D     ]]  &&  cat  LIST_PROFIL_1D  >>  LIST_PROFILE
73OC:[[ -f LIST_PROFIL        ]]  &&  cat  LIST_PROFIL     >>  LIST_PROFILE
74#
75# Combine all particle information files
76OC:[[ -f PARTICLE_INFOS/_0000 ]]  &&  cat  PARTICLE_INFOS/* >> PARTICLE_INFO
77#
78#----------------------------------------------------------------------------
79# List of input-files
80#----------------------------------------------------------------------------
81PARIN                    in:job        d3#      $base_data/$fname/INPUT    _p3d
82PARIN                    in:job        d3f      $base_data/$fname/INPUT    _p3df
83TOPOGRAPHY_DATA          in:locopt     d3#:d3f  $base_data/$fname/INPUT    _topo
84BININ                    in:loc:flpe   d3f      $base_data/$fname/RESTART  _d3d
85PARTICLE_RESTART_DATA_IN in:loc:flpe   prtf     $base_data/$fname/RESTART  _rprt
86DATA_1D_PR_NETCDF        in:locopt     prf      $base_data/$fname/OUTPUT   _pr     nc
87DATA_1D_SP_NETCDF        in:locopt     spf      $base_data/$fname/OUTPUT   _sp     nc
88DATA_1D_TS_NETCDF        in:locopt     tsf      $base_data/$fname/OUTPUT   _ts     nc
89DATA_1D_PTS_NETCDF       in:locopt     ptsf     $base_data/$fname/OUTPUT   _pts    nc
90DATA_2D_XY_NETCDF        in:locopt     xyf      $base_data/$fname/OUTPUT   _xy     nc
91DATA_2D_XY_AV_NETCDF     in:locopt     xyf      $base_data/$fname/OUTPUT   _xy_av  nc
92DATA_2D_XZ_NETCDF        in:locopt     xzf      $base_data/$fname/OUTPUT   _xz     nc
93DATA_2D_YZ_NETCDF        in:locopt     yzf      $base_data/$fname/OUTPUT   _yz     nc
94DATA_3D_NETCDF           in:locopt     3df      $base_data/$fname/OUTPUT   _3d     nc
95DATA_PRT_NETCDF          in:locopt:pe  prtf     $base_data/$fname/OUTPUT   _prt
96#
97#----------------------------------------------------------------------------
98# List of output-files
99#----------------------------------------------------------------------------
100BINOUT                    out:loc:flpe restart   $base_data/$fname/RESTART  _d3d
101PARTICLE_RESTART_DATA_OUT out:loc:flpe prt#:prtf $base_data/$fname/RESTART  _rprt
102#
103RUN_CONTROL       out:loc:tr   d3#     $base_data/$fname/MONITORING  _rc
104RUN_CONTROL       out:loc:tra  d3f     $base_data/$fname/MONITORING  _rc
105HEADER            out:loc:tr   d3#     $base_data/$fname/MONITORING  _header
106HEADER            out:loc:tra  d3f     $base_data/$fname/MONITORING  _header
107CPU_MEASURES      out:loc:tr   d3#     $base_data/$fname/MONITORING  _cpu
108CPU_MEASURES      out:loc:tra  d3f     $base_data/$fname/MONITORING  _cpu
109#
110DATA_1D_PR_NETCDF    out:loc:tr   pr#:prf   $base_data/$fname/OUTPUT  _pr      nc
111DATA_1D_SP_NETCDF    out:loc:tr   sp#:spf   $base_data/$fname/OUTPUT  _sp      nc
112DATA_1D_TS_NETCDF    out:loc:tr   ts#:tsf   $base_data/$fname/OUTPUT  _ts      nc
113DATA_1D_PTS_NETCDF   out:loc:tr   pts#:ptsf $base_data/$fname/OUTPUT  _pts     nc
114DATA_2D_XY_NETCDF    out:loc:tr   xy#:xyf   $base_data/$fname/OUTPUT  _xy      nc
115DATA_2D_XY_AV_NETCDF out:loc:tr   xy#:xyf   $base_data/$fname/OUTPUT  _xy_av   nc
116DATA_2D_XZ_NETCDF    out:loc:tr   xz#:xzf   $base_data/$fname/OUTPUT  _xz      nc
117DATA_2D_XZ_AV_NETCDF out:loc:tr   xz#:xzf   $base_data/$fname/OUTPUT  _xz_av   nc
118DATA_2D_YZ_NETCDF    out:loc:tr   yz#:yzf   $base_data/$fname/OUTPUT  _yz      nc
119DATA_2D_YZ_AV_NETCDF out:loc:tr   yz#:yzf   $base_data/$fname/OUTPUT  _yz_av   nc
120DATA_3D_NETCDF       out:loc:tr   3d#:3df   $base_data/$fname/OUTPUT  _3d      nc
121DATA_3D_AV_NETCDF    out:loc:tr   3d#:3df   $base_data/$fname/OUTPUT  _3d_av   nc
122DATA_MASK_01_NETCDF    out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m01     nc
123DATA_MASK_01_AV_NETCDF out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m01_av  nc
124DATA_MASK_02_NETCDF    out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m02     nc
125DATA_MASK_02_AV_NETCDF out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m02_av  nc
126DATA_MASK_03_NETCDF    out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m03     nc
127DATA_MASK_03_AV_NETCDF out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m03_av  nc
128DATA_MASK_04_NETCDF    out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m04     nc
129DATA_MASK_04_AV_NETCDF out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m04_av  nc
130DATA_MASK_05_NETCDF    out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m05     nc
131DATA_MASK_05_AV_NETCDF out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m05_av  nc
132DATA_MASK_06_NETCDF    out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m06     nc
133DATA_MASK_06_AV_NETCDF out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m06_av  nc
134DATA_MASK_07_NETCDF    out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m07     nc
135DATA_MASK_07_AV_NETCDF out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m07_av  nc
136DATA_MASK_08_NETCDF    out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m08     nc
137DATA_MASK_08_AV_NETCDF out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m08_av  nc
138DATA_MASK_09_NETCDF    out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m09     nc
139DATA_MASK_09_AV_NETCDF out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m09_av  nc
140DATA_MASK_10_NETCDF    out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m10     nc
141DATA_MASK_10_AV_NETCDF out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m10_av  nc
142DATA_MASK_11_NETCDF    out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m11     nc
143DATA_MASK_11_AV_NETCDF out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m11_av  nc
144DATA_MASK_12_NETCDF    out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m12     nc
145DATA_MASK_12_AV_NETCDF out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m12_av  nc
146DATA_MASK_13_NETCDF    out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m13     nc
147DATA_MASK_13_AV_NETCDF out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m13_av  nc
148DATA_MASK_14_NETCDF    out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m14     nc
149DATA_MASK_14_AV_NETCDF out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m14_av  nc
150DATA_MASK_15_NETCDF    out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m15     nc
151DATA_MASK_15_AV_NETCDF out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m15_av  nc
152DATA_MASK_16_NETCDF    out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m16     nc
153DATA_MASK_16_AV_NETCDF out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m16_av  nc
154DATA_MASK_17_NETCDF    out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m17     nc
155DATA_MASK_17_AV_NETCDF out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m17_av  nc
156DATA_MASK_18_NETCDF    out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m18     nc
157DATA_MASK_18_AV_NETCDF out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m18_av  nc
158DATA_MASK_19_NETCDF    out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m19     nc
159DATA_MASK_19_AV_NETCDF out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m19_av  nc
160DATA_MASK_20_NETCDF    out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m20     nc
161DATA_MASK_20_AV_NETCDF out:loc:tr ma#:maf   $base_data/$fname/OUTPUT  _m20_av  nc
162DATA_PRT_NETCDF      out:loc:pe   prt#:prtf $base_data/$fname/OUTPUT  _prt     nc
163DATA_PRT_NETCDF      out:loc:trpe prt#:prtf $base_data/$fname/OUTPUT  _prt     nc
164DATA_DVR             out:loc:trpe dvr       $base_data/$fname/OUTPUT  _dvr
165#
166DVRP_LOG          out:loc:tr   dv#       $base_data/$fname/MONITORING  _dvrp_log
167DVRP_LOG          out:loc:tra  dvf       $base_data/$fname/MONITORING  _dvrp_log
168PARTICLE_INFO     out:loc:tr   pt#       $base_data/$fname/MONITORING  _prt_info
169PARTICLE_INFO     out:loc:tra  ptf       $base_data/$fname/MONITORING  _prt_info
170PARTICLE_DATA     out:loc:flpe prt#:prtf $base_data/$fname/OUTPUT      _prt_dat
171# A BLANK LINE MUST FOLLOW
Note: See TracBrowser for help on using the repository browser.
                                                                                                                                                                                                                                                                                                                                                                               
  | Impressum | ©Leibniz Universität Hannover |