Home

Context Navigation

iofiles

Timestamp:: Nov 20, 2018 4:17:51 PM (6 years ago)
Author:: kanani
Comment:: --

Legend:

: Unmodified
: Added
: Removed
: Modified

doc/app/iofiles

-                      v111
+                      v112
 The model works with a set of files, which are all located in the temporary working directory and which are either expected at the beginning of the run (the so-called input files) or which are produced during the run and/or at the end of the simulation (output files). The following table lists all default files addressed by the model. They are arranged according to the associated FORTRAN unit (unit number). The unit 90 is used for different files, which are opened only briefly and uniquely in the model and closed directly afterwards. Unit numbers 101 - 109, 111 - 113, 116, 118 and 121-160 refer to data files which have netCDF format. These are not opened by a FORTRAN-OPEN-statement but by a corresponding netCDF call ({{{NF90_CREATE}}} or {{{NF90_OPEN}}}). These files are only created on machines where a netCDF library is available. \\\\
 The file names described in the list correspond to the names indicated in the respective OPEN instruction, i.e. the files are expected and saved under these names in the temporary working directory of the model. By default, file names are always written in capital letters. The third column indicates whether it is an input or output file (I and/or O). The netCDF files can be both input and output files (I/O). If restart jobs shall append data to an existing netCDF file (created by a previous job of the job chain), the respective file has to be given as an input file (see more details about the [../netcdf PALM-netCDF-output]). \\\\
 On parallel computers, many of the files are read and/or written by the central processing element 0 (PE0) only. These files have processor-independent content (and therefore they can be read or written by other PEs just as well). However, certain files have processor-dependent content. For the binary output of data for restart runs (local file [#BINOUT BINOUT]), for example, each PE outputs only the data of its subdomain. So each processing element writes into its own file with its own file name. These files lie in a subdirectory of the temporary working directory. Their names are built from the underline ("_") and the four digit processor ID. The data written for restart runs would be e.g. on the files BINOUT/_0000 (PE0), BINOUT/_0001 (PE1), BINOUT/_0002 (PE2) etc. Such files, which have processor-dependent content on parallel computers, are marked in the following list by the fact, that to the file name a line ("/") is attached. If appropriate output files are to be copied through '''mrun''' to permanent files, and/or files with processor-dependent content are supposed to be copied as input files into the temporary working directory of the model, you have to indicate a special file attribute in the appropriate file connection statement (see [wiki:arpe arpe], [wiki:flpe flpe] in the '''mrun''' description). Then the permanent file name will be interpreted as a directory name, in which the input files are expected and/or to which output files are written. The file names in these directories are always named _0000, _0001, _0002 etc. \\\\
+On parallel computers, many of the files are read and/or written by the central processing element 0 (PE0) only. These files have processor-independent content (and therefore they can be read or written by other PEs just as well). However, certain files have processor-dependent content. For the binary output of data for restart runs (local file [#BINOUT BINOUT]), for example, each PE outputs only the data of its subdomain. So each processing element writes into its own file with its own file name. These files lie in a subdirectory of the temporary working directory. Their names are built from the underline ("_") and the four digit processor ID. The data written for restart runs would be e.g. on the files BINOUT/_0000 (PE0), BINOUT/_0001 (PE1), BINOUT/_0002 (PE2) etc. Such files, which have processor-dependent content on parallel computers, are marked in the following list by the fact, that to the file name a line ("/") is attached. If appropriate output files are to be copied through [wiki:doc/app/palmrun palmrun] to permanent files, and/or files with processor-dependent content are supposed to be copied as input files into the temporary working directory of the model, you have to indicate a special file attribute in the appropriate file connection statement (see [wiki:arpe arpe], [wiki:flpe flpe] in the [wiki:doc/app/palmrun palmrun] description). Then the permanent file name will be interpreted as a directory name, in which the input files are expected and/or to which output files are written. The file names in these directories are always named _0000, _0001, _0002 etc. \\\\
 For internal use, the model may open a set of further files, which are not defined by the user but contain no usable information and therefore are not included in this list. \\\\
 Depending on the configuration of the underlying file-system (e.g. Lustre) and the capacity of the I/O-hardware, simultaneous output to a larger number of files, i.e. if a larger number of PEs is used (>1000), may lead to severe problems as job aborts or even to a complete crash of the whole system. In order to circumvent this problem, the maximum number of parallel I/O streams (i.e. the number of files which are simultaneously written), can be limited by using the '''mrun'''-option {{{-w <max # of streams>}}}, where {{{<max # of streams>}}} should be smaller or equal 256 (on the SGI-ICE of HLRN-II). The (parallel) output is then done in a sequential order for blocks of 256 PEs. \\\\
+Depending on the configuration of the underlying file-system (e.g. Lustre) and the capacity of the I/O-hardware, simultaneous output to a larger number of files, i.e. if a larger number of PEs is used (>1000), may lead to severe problems as job aborts or even to a complete crash of the whole system. In order to circumvent this problem, the maximum number of parallel I/O streams (i.e. the number of files which are simultaneously written), can be limited by using the [wiki:doc/app/palmrun palmrun]-option {{{-w <max # of streams>}}}, where {{{<max # of streams>}}} should be smaller or equal 256 (on the SGI-ICE of HLRN-II). The (parallel) output is then done in a sequential order for blocks of 256 PEs. \\\\
 In case of [../examples/coupled coupled] atmosphere-ocean runs, both the atmosphere and the [../inipar#ocean ocean] executable use the same temporary working directory. However, each executable requires its own, unique set of files for I/O. In order to distinguish between atmosphere and ocean files, coupled atmosphere-ocean runs use the following filename convention. The atmosphere executable uses the set of normal filenames given in the table below. The ocean executable uses a set of modified filenames that have the string '_O' added to their normal name. The coupled ocean filenames are given in brackets in the table below where applicable. The string '_O' is simply appended to most filenames; exceptions are highlighted in bold face. ('''Note''': uncoupled ocean runs use the normal set of filenames without '_O'.) \\
 …
 {{{#!td
 Binary data, which are written by the model at the end of the run and possibly needed by [../runs restart runs] for the initialization. This output file is then read in as file [#BININ BININ]. It contains the [../inipar initial parameters] of the model run, arrays of the prognostic and diagnostic variables as well as those parameters determined so far during a job chain and variables for plots of horizontally averaged vertical profiles (see [../d3par#data_output_pr data_output_pr]). With runs on several processors it has to be noted that each processing element writes its own file and the file content is processor-dependent. A specification of the file format can be found in the description of the file [#BININ BININ]. \\\\
 The file BINOUT is written by the model only if, with the help of the '''mrun'''-configuration file, the environment variable {{{write_binary}}} is assigned to the {{{true}}} (see [../runs example]). \\\\
+The file BINOUT is written by the model only if, with the help of the [wiki:doc/app/palmrun palmrun]-configuration file, the environment variable {{{write_binary}}} is assigned to the {{{true}}} (see [../runs example]). \\\\
 With large grid point numbers the file BINOUT (or the files residing in directory BINOUT/) will be very large and should be stored (if available) on the archive system of the remote computer.
 }}}
 …
 {{{#!td
 This file contains data of the two-dimensional horizontal cross sections written by the model (see [../d3par#data_output data_output]). \\\\
 With parallel runs and choice of [../d3par#data_output_2d_on_each_pe data_output_2d_on_each_pe] = ''.T.'' each processing element writes the data of its subdomain to a separate file with the name PLOT2D_XY_<processor-Id>, whereby <processor-Id> is a four digit number (e.g. PLOT2D_XY_0000). After PALM has finished, the contents of these individual files are sampled into one final file using the program {{{combine_plot_fields.x}}} (automatically activated by '''mrun'''). The tool writes informative messages about the actions accomplished into the job protocol (e.g., even if no files were found). By the call of {{{combine_plot_fields.x}}} possibly existing files of the other cross sections (xz and/or yz) and three-dimensional arrays (see [#PLOT3D_DATA PLOT3D_DATA]) are also combined. \\\\
+With parallel runs and choice of [../d3par#data_output_2d_on_each_pe data_output_2d_on_each_pe] = ''.T.'' each processing element writes the data of its subdomain to a separate file with the name PLOT2D_XY_<processor-Id>, whereby <processor-Id> is a four digit number (e.g. PLOT2D_XY_0000). After PALM has finished, the contents of these individual files are sampled into one final file using the program {{{combine_plot_fields.x}}} (automatically activated by [wiki:doc/app/palmrun palmrun]). The tool writes informative messages about the actions accomplished into the job protocol (e.g., even if no files were found). By the call of {{{combine_plot_fields.x}}} possibly existing files of the other cross sections (xz and/or yz) and three-dimensional arrays (see [#PLOT3D_DATA PLOT3D_DATA]) are also combined. \\\\
 Further information about the output of plots of two-dimensional cross sections is found in the description of the run parameter [../d3par#data_output data_output].
 }}}
 …
 {{{#!td
 This file contains the data of the three-dimensional arrays (see [../d3par#data_output data_output]) in netCDF format. The specific netCDF format (e.g. classic, 64-bit offset, netcdf4, etc.) depends on the setting of d3par-parameter [../d3par#netcdf_data_format netcdf_data_format].\\\\
 With parallel runs and {{{netcdf_data_format}}} < ''5'', each processing element first writes the data of its subdomain in FORTRAN binary format to a separate file named PLOT3D_DATA_<Processor-Id>, where <Processor-Id> is a four digit number (e.g. PLOT3D_DATA_0000). After PALM has finished, contents of these files are collected by the helper routine {{{combine_plot_fields.x}}} (automatically executed by '''mrun'''). This tool expects the files of the individual processing elements under the names described above and outputs the combined data in netCDF format into the final file named PLOT3D_DATA. The tool writes informative messages about the accomplished actions into the job protocol (also if no files were found). Helper routine {{{combine_plot_fields.x}}} also collects files containing two-dimensional data (see e.g. [#PLOT2D_XY PLOT2D_XY]).\\\\
+With parallel runs and {{{netcdf_data_format}}} < ''5'', each processing element first writes the data of its subdomain in FORTRAN binary format to a separate file named PLOT3D_DATA_<Processor-Id>, where <Processor-Id> is a four digit number (e.g. PLOT3D_DATA_0000). After PALM has finished, contents of these files are collected by the helper routine {{{combine_plot_fields.x}}} (automatically executed by [wiki:doc/app/palmrun palmrun]). This tool expects the files of the individual processing elements under the names described above and outputs the combined data in netCDF format into the final file named PLOT3D_DATA. The tool writes informative messages about the accomplished actions into the job protocol (also if no files were found). Helper routine {{{combine_plot_fields.x}}} also collects files containing two-dimensional data (see e.g. [#PLOT2D_XY PLOT2D_XY]).\\\\
 With {{{netcdf_data_format}}} > ''4'' PALM directly writes the data of the subdomains into one single file using parallel I/O (MPI I/O), which makes a call of {{{combine_plot_fields.x}}} obsolete.
 }}}