Changes between Version 35 and Version 36 of doc/app/palmrun


Ignore:
Timestamp:
Oct 30, 2018 2:33:05 PM (7 years ago)
Author:
raasch
Comment:

--

Legend:

Unmodified
Added
Removed
Modified
  • doc/app/palmrun

    v35 v36  
    2525||-b   ||---  ||create a batch job  ||
    2626||-B   ||---  ||Do not delete the temporary working directory  ||
     27||-c   ||default ||configuration identifier  ||
    2728||-C   ||---  ||Tells that it is a {{{palmrun}}} call for a restart job that has been automatically created. This is an internal option but it can be used for manually generated restart runs, if the user likes to re-use the contents of the {{{SOURCES_FOR_RUN...}}} folder.  ||
    28 ||-d   ||test  ||Name of the run  ||
    2929||-F   ||---  ||Create a batch job file only, and do not submit it.  ||
    30 ||-h   ||default ||configuration identifier  ||
    3130||-k   ||false ||If set true, input files that have the {{{ln}}} attribute and that have been generated by a previous run within a job chain will be automatically deleted at the end of the run.  ||
    3231||-m   ||" "  ||memory in MByte to be requested in batch jobs per MPI task  ||
     
    3433||-O   ||1    ||OpenMP threads to be started per MPI task. Environment variable {{{OMP_NUM_THREADS}}} will be set to this value  ||
    3534||-q   ||none ||name of the job queue to which batch jobs will be submitted  ||
     35||-r   ||test  ||Name of the run  ||
    3636||-s   ||" "  ||List of subroutines (Fortran file names) from the SVN repository (under {{{.../trunk/SOURCES}}}) that shall be compiled for this run. Compiled files will be exclusively used for the run and not be put in the MAKE_DEPOSITORY. In case of {{{-s LM}}}, all files in the repository that have been modified by the used will be compiled.  ||
    3737||-t   ||" "  ||cpu time request for batch jobs in seconds  ||
     
    8585* Setting the environment variable {{{PALM_BIN}}} in shell-profile files (e.g. {{{.bashrc}}}) is not required any more.
    8686
    87 * The old configuration file {{{.mrun.config}}} has been split into two files {{{.palm.config.<configuration_identifier>}}} and {{{.palm.iofiles}}}, where {{{<configuration_identifier>}}} (short {{{<ci>}}}) is an arbitrary string that you can define. \\\\ "Configuration" means a setting for a specific computer with a specific compiler, compiler options, libraries, etc. \\ If you like to run '''PALM''' with different configurations, e.g. one with debug options switched on, and one with high optimization, you need to create separate files for each configuration, e.g. {{{.palm.config.optimized}}} and {{{.palm.config.debug}}}. This replaces the old block structure in {{{.mrun.config}}}. The configuration file to be used is defined by {{{palmrun}}}- or {{{palmbuild}}}-option {{{-h}}}., e.g. {{{palmrun ... -h optimized}}} will use {{{.palm.config.optimized}}}. You find examples of {{{.palm.config.<...>}}} files in your '''PALM''' copy under {{{.../trunk/SCRIPTS}}} \\\\ You will need only one file {{{.palm.iofiles}}} which contains the file connection statements to be used for all configurations. \\ The file attributes (second column in the file connection statements) have been partly changed. The second attribute, which was either {{{loc}}}, {{{locopt}}} or {{{job}}}, has been completely removed. Optional input files now require {{{inopt}}} as first attribute. Those input files to be send to the remote host require {{{tr}}} as second attribute (instead of {{{job}}}). {{{fl}}} and {{{flpe}}} must be changed to {{{ln}}} and {{{lnpe}}} respectively. \\\\ For output files, a wildcard {{{*}}} can be given as file activation string in the third column. In such a case, existing local output files will always be copied to their permanent position. No warning will be given if they do not exist. \\\\ Wildcards (*) are allowed for local names of output files (e.g. {{{BINOUT*}}}) and file extensions of input files (e.g. {{{_p3d*}}}). Using wildcards, only one file connection statement is required, e.g. for nested runs which require different input files for each domain ({{{_p3d, _p3d_N01, _p3d_N02}}}, etc.) or which generate different output files (e.g. {{{BINOUT, BINOUT_N01}}}, etc.). The additional extensions that are identified from the existing files (e.g. {{{_N01, _N02}}}) will be automatically added to the local filename (in case of input files) or to the file extension (in case of output files). \\\\ The utility program {{{interpret_config}}} has been removed. The configuration files are now directly interpreted by the shellscripts.
    88 
    89 * Only one call of {{{palmbuild}}} is required to compile for both the utilities and the PALM source code (there is no option {{{-u}}} anymore). The compiled routines (object files and executables) are put into folder {{{MAKE_DEPOSITORY_<configuration_identifier>}}}, where {{{<configuration_identifier>}}} equals the string given with {{{palmbuild}}}-option {{{-h}}}.
    90 
    91 * {{{palmrun}}} does not compile any more at the beginning of a batch job. The palm-executable for the batch-job (or for the interactive session) is created as part of the {{{palmrun}}}-call that you have manually entered at your terminal, and it is created before the batch-job is submitted. The executable is put into the folder {{{SOURCES_FOR_RUN_<run_identifier>}}}, where {{{<run_identifier>}}} is the string provided with {{{palmrun}}}-option {{{-d}}}. This folder is now put into the folder set with variable {{{fast_io_catalog}}} (see below for {{{fast_io_catalog}}}). If you do not use a user-interface, {{{palmrun}}} will not compile at all and will take the executable from folder {{{MAKE_DEPOSITORY_<configuration_identifier>}}} that has been generated with your last call of {{{palmbuild}}}. If {{{palmrun}}} cannot find the folder {{{MAKE_DEPOSITORY_<configuration_identifier>}}}, it will internally call {{{palmbuild}}} in order to generate it. If {{{palmrun}}} finds a folder {{{SOURCES_FOR_RUN_<run_identifier>}}} that has been generated by a previous call of {{{palmrun}}}, it will ask you if executables from that folder shall be used. This way, you can avoid to re-compile your user-interface with each call of {{{palmrun}}}. Automatically generated restart runs will always use executables from  {{{SOURCES_FOR_RUN_<run_identifier>}}}. \\\\ You may have to remove folders {{{SOURCES_FOR_RUN_...}}} manually from time to time, because they are not deleted automatically at the end of a job (or the last job of a restart job chain).
     87* The old configuration file {{{.mrun.config}}} has been split into two files {{{.palm.config.<configuration_identifier>}}} and {{{.palm.iofiles}}}, where {{{<configuration_identifier>}}} (short {{{<ci>}}}) is an arbitrary string that you can define. \\\\ "Configuration" means a setting for a specific computer with a specific compiler, compiler options, libraries, etc. \\ If you like to run '''PALM''' with different configurations, e.g. one with debug options switched on, and one with high optimization, you need to create separate files for each configuration, e.g. {{{.palm.config.optimized}}} and {{{.palm.config.debug}}}. This replaces the old block structure in {{{.mrun.config}}}. The configuration file to be used is defined by {{{palmrun}}}- or {{{palmbuild}}}-option {{{-c}}}., e.g. {{{palmrun ... -c optimized}}} will use {{{.palm.config.optimized}}}. You find examples of {{{.palm.config.<...>}}} files in your '''PALM''' copy under {{{.../trunk/SCRIPTS}}} \\\\ You will need only one file {{{.palm.iofiles}}} which contains the file connection statements to be used for all configurations. \\ The file attributes (second column in the file connection statements) have been partly changed. The second attribute, which was either {{{loc}}}, {{{locopt}}} or {{{job}}}, has been completely removed. Optional input files now require {{{inopt}}} as first attribute. Those input files to be send to the remote host require {{{tr}}} as second attribute (instead of {{{job}}}). {{{fl}}} and {{{flpe}}} must be changed to {{{ln}}} and {{{lnpe}}} respectively. \\\\ For output files, a wildcard {{{*}}} can be given as file activation string in the third column. In such a case, existing local output files will always be copied to their permanent position. No warning will be given if they do not exist. \\\\ Wildcards (*) are allowed for local names of output files (e.g. {{{BINOUT*}}}) and file extensions of input files (e.g. {{{_p3d*}}}). Using wildcards, only one file connection statement is required, e.g. for nested runs which require different input files for each domain ({{{_p3d, _p3d_N01, _p3d_N02}}}, etc.) or which generate different output files (e.g. {{{BINOUT, BINOUT_N01}}}, etc.). The additional extensions that are identified from the existing files (e.g. {{{_N01, _N02}}}) will be automatically added to the local filename (in case of input files) or to the file extension (in case of output files). \\\\ The utility program {{{interpret_config}}} has been removed. The configuration files are now directly interpreted by the shellscripts.
     88
     89* Only one call of {{{palmbuild}}} is required to compile for both the utilities and the PALM source code (there is no option {{{-u}}} anymore). The compiled routines (object files and executables) are put into folder {{{MAKE_DEPOSITORY_<configuration_identifier>}}}, where {{{<configuration_identifier>}}} equals the string given with {{{palmbuild}}}-option {{{-c}}}.
     90
     91* {{{palmrun}}} does not compile any more at the beginning of a batch job. The palm-executable for the batch-job (or for the interactive session) is created as part of the {{{palmrun}}}-call that you have manually entered at your terminal, and it is created before the batch-job is submitted. The executable is put into the folder {{{SOURCES_FOR_RUN_<run_identifier>}}}, where {{{<run_identifier>}}} is the string provided with {{{palmrun}}}-option {{{-r}}}. This folder is now put into the folder set with variable {{{fast_io_catalog}}} (see below for {{{fast_io_catalog}}}). If you do not use a user-interface, {{{palmrun}}} will not compile at all and will take the executable from folder {{{MAKE_DEPOSITORY_<configuration_identifier>}}} that has been generated with your last call of {{{palmbuild}}}. If {{{palmrun}}} cannot find the folder {{{MAKE_DEPOSITORY_<configuration_identifier>}}}, it will internally call {{{palmbuild}}} in order to generate it. If {{{palmrun}}} finds a folder {{{SOURCES_FOR_RUN_<run_identifier>}}} that has been generated by a previous call of {{{palmrun}}}, it will ask you if executables from that folder shall be used. This way, you can avoid to re-compile your user-interface with each call of {{{palmrun}}}. Automatically generated restart runs will always use executables from  {{{SOURCES_FOR_RUN_<run_identifier>}}}. \\\\ You may have to remove folders {{{SOURCES_FOR_RUN_...}}} manually from time to time, because they are not deleted automatically at the end of a job (or the last job of a restart job chain).
    9292
    9393* The option for giving the file activation strings is now {{{-a "d3# ..."}}} instead of {{{-r "d3# ..."}}}.
     
    197197 - Lines starting with {{{BD:}}}: Here you have to give the batch directives that are required by your batch-system. {{{palmrun}}} will replace wildcards in the following way:
    198198   * {{{ {{project_account}} }}}: To be used if you like to run the job under a specific account number. Is replaced by value provided with {{{palmrun}}}-option {{{-A}}}.
    199    * {{{ {{job_id}} }}}: The job's name. It will be formed by the run identifier provided with {{{palmrun}}}-option {{{-d}}} and a 5-digit random number, e.g. {{{-d example_cbl}}} may give {{{example_cbl.12345}}}.
     199   * {{{ {{job_id}} }}}: The job's name. It will be formed by the run identifier provided with {{{palmrun}}}-option {{{-r}}} and a 5-digit random number, e.g. {{{-r example_cbl}}} may give {{{example_cbl.12345}}}.
    200200   * {{{ {{cpu_hours}}, {{cpu_minutes}}, {{cpu_seconds}} }}}: Will be replaced based on the total CPU time in seconds provided with {{{palmrun}}}-option {{{-t}}}, .e.g. {{{-t 3666}}} will replace {{{ {{cpu_hours}}=1, {{cpu_minutes}}=1, {{cpu_seconds}=6 }}}.
    201201   * {{{ {{nodes}} }}}: The number of nodes requested by the job. It will be replaced by the result of {{{ totalcores / ( noMPIt * noOpenMPt )}}}, where {{{totalcores}}} is the total number of cores as requested with {{{palmrun}}}-option {{{-X}}}, {{{noMPIt}}} is the number of MPI-tasks to be started on each node, as given my {{{palmrun}}}-option {{{-T}}}, and {{{noOpenMPT}}} is the number of OpenMP-threads to be started per MPI-task, as given by {{{palmrun}}}-option {{{-O}}}.
    202202   * {{{ {{tasks_per_node}} }}}: The number of MPI-tasks to be started on each node, as given my {{{palmrun}}}-option {{{-T}}}.
    203    * {{{ {{jobfile}} }}}: Name of the job protocol file. The filename for jobs running on a remote host is created from {{{palmrun}}}-options {{{-h}}} and {{{-d}}}, e.g. for {{{palmrun -d example_cbl -h crayh ...}}} the job protocol file name will be {{{crayh_example_cbl}}}. For jobs running on a local host, the name part from option {{{-h}}} will be omitted.
     203   * {{{ {{jobfile}} }}}: Name of the job protocol file. The filename for jobs running on a remote host is created from {{{palmrun}}}-options {{{-c}}} and {{{-r}}}, e.g. for {{{palmrun -r example_cbl -c crayh ...}}} the job protocol file name will be {{{crayh_example_cbl}}}. For jobs running on a local host, the name part from option {{{-c}}} will be omitted.
    204204   * {{{ {{queue}} }}}: The name of the queue to which the job shall be submitted. Will be replaced by the value provided with {{{palmrun}}}-option {{{-q}}}, or, if {{{-q}}} is omitted, by the value of variable {{{defaultqueue}}} (see further above).
    205205   * {{{ {{previous_job}} }}}: The name of a previous job as given by {{{palmrun}}}-option {{{-W}}}. Can be used to set job dependencies.