Changes between Version 14 and Version 15 of doc/tec/testsuite
- Timestamp:
- Nov 22, 2018 11:37:43 AM (7 years ago)
Legend:
- Unmodified
- Added
- Removed
- Modified
-
doc/tec/testsuite
v14 v15 1 1 = PALM Tests = 2 2 3 PALM features a test suite to ensure its reliability while development goes forward. It can be executed locally with the following options:3 PALM features a test suite to ensure its reliability while development goes forward. It can be executed locally. For available options and their description type: 4 4 {{{ 5 5 #!sh 6 palmtest - h "<configuration>" -d "<list of test cases>" -X <max number of cores to execute on> -N "<test id>"6 palmtest --help 7 7 }}} 8 As these are all optional, the defaults are as follows:9 - configuration: default10 - list of test cases: all available11 - max number of cores to execute on: all available but not more that 3212 - test id: the current time13 8 14 9 == The Testserver == 15 10 16 The server responsible for testing can be found [https://testserver.palm-model.org/ here]. It is a 32 Core shared memory node running Jenkins on top of Ubuntu Server 16.04. The server runs the following test script based on plain PALM installation.11 The server responsible for testing can be found [https://testserver.palm-model.org/ here]. It is a 32 Core shared memory node running Jenkins on top of Ubuntu Server 16.04. The server runs the following test script based on a plain PALM installation. 17 12 {{{ 18 13 #!sh 19 14 #!/bin/bash 20 15 21 palmtest -h "default" -N "${BUILD_NUMBER}" -X 32 22 result_default=$? 16 palmtest --no-color --test-id "${BUILD_NUMBER}" 23 17 24 if [[ ${result_default} -ne 0 ]]; then 25 palmtest -h "debug" -N "${BUILD_NUMBER}_debug" -X 32 26 result_debug=$? 18 result=$? 19 20 if [[ ${result} -ne 0 ]]; then 21 echo "" 22 echo "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%" 23 echo "%%%%%%%%%%%%%%%%%%%% DUMPING palmtest.log %%%%%%%%%%%%%%%%%%%%%%%%" 24 echo "%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%" 25 cat ${WORKSPACE}/tests/${BUILD_NUMBER}/palmtest.log 26 exit 1 27 27 else 28 result_debug=029 fi30 31 if [[ ${result_default} -eq 0 ]] && [[ ${result_debug} -eq 0 ]]; then32 28 exit 0 33 else34 exit 135 29 fi 36 30 … … 39 33 == How to add a test case == 40 34 41 Any developer can add new test cases. This is as simple as adding a respective {{{*_p3d}}} file and {{{*_rc}}} file to the INSTALL directory in the PALM trunk. The {{{palmtest}}} script will detect all available {{{*_p3d}}} namelist files with matching {{{*_rc}}} in this directory, execute them using {{{palmrun}}} and compare them to the respective {{{*_rc}}} reverence file. Please make sure the reverence {{{*_rc}}} file is valid and reproducible. It is the developers responsibility to make sure that the newly added test succeeds at the time it is added! All methodical errors associated with a specific test case that arise later, are also the initial developers responsibility.35 Any developer can add new test cases. All test cases are stored within trunk/TESTS/cases. Please follow the layout of already existing cases and make sure the reference files in the MONITORING and OUTPUT sub-directories are valid and reproducible according to the parameters specified in its respective case_config.yml file. It is the developers responsibility to make sure that the newly added test case succeeds at the time it is added! All methodical errors associated with a specific test case that arise later, are also the initial developers responsibility. In case a commit breaks a test, the developer who performed that specific commit is responsible for fixing it. 42 36 43 37 == Rules for test case development == … … 45 39 Please to comply with the following Rules while developing new test cases: 46 40 47 1. The test case must be executable on 1, 2, 4, 8, 16 and 32cores.41 1. The test case must be executable at least on 4 cores. 48 42 2. The execution time of the test case on a single core should not exceed 10 seconds. 49 43 3. The test case is only allowed to be committed after the developer made sure that the test can run successful on the server. 50 4. If a test case creates problems that are not fixable within an acceptable timeframe, the test case will be removed. The test case may be added again, once all problems are solved.44 4. If a test case creates problems that are not fixable within an acceptable timeframe, the test case will be disabled. The test case may be added again, once all problems are solved. 51 45 52 46 == List of proposed test cases == 53 47 54 48 ||='''Name''' =||='''Description''' =||='''Status''' =||='''Responsible Developers''' =|| 55 ||example_cbl ||Simple good old convective boundary layer setup ||Implemented || || 56 ||ex_lsm_clearsky ||Basic LSM setup ||Implemented ||Katrin || 57 ||test_oceanml ||Basic ocean mixed layer setup ||Implemented ||Siggi || 49 || ex_lsm_clearsky ||Basic LSM setup ||Implemented ||Katrin || 58 50 || ||FFTW / MG setup ||Missing || || 59 || ||humidity setup (is used in ex_lsm_clearsky) ||Implemented || ||60 51 || ||passive scalar setup ||Missing || || 61 || ||topography setup ||Missing || ||62 52 || ||particle LCM setup - for passive tracers with SGS fluctuations ||Missing || Johannes || 63 53 || ||particle LCM setup - for cloud droplets including collision ||Missing || Johannes || … … 66 56 || ||non-cyclic turbulence generator setup ||Missing || || 67 57 || ||cyclic fill setup ||Missing || || 68 || ||restart setup ||Missing || ||69 58 || ||all output activated setup ||Missing || || 70 || ||USM + canopy setup ||Missing || ||71 || ||RRTMG setup ||Missing || ||72 59 || ||nesting setup ||Missing || || 73 || ||chemistry setup ||Missing || ||74 60 || ||Setup for testing the wind turbine model ||Missing || || 75 61 … … 77 63 78 64 ||='''Name''' =||='''Description''' =||='''Status''' =||='''Responsible Developers''' =|| 79 ||default ||Based on gfortran, MPICH2, FFTW and NetCDF with options: {{{-O3 -ffree-line-length-none}}} ||Implemented ||Helge ||80 ||debug ||Based on default but with debug options: {{{-O0 -Wall -Wextra -pedantic -fcheck=all -fbacktrace}}} ||Implemented ||Helge ||81 65 || ||different compiler: intel/cray/pgi ? ||Missing || || 82 66 || ||non-MPI - should be serial. i.e. without {{{-D__parallel}}} (Siggi) ||Missing || || … … 85 69 || ||nopointer ||Missing || || 86 70 || ||FFTW - only required after OpenACC is part of default code (Siggi) ||Missing || || 87 || ||RRTMG ||Missing || ||88 71 || ||KPP_CHEM ||Missing || ||
