WARRANTY, USAGE RESTRICTIONS, etc. You are not allowed to have access to this software unless you signed an agreement. See http://www.knmi.nl/hirlam If you have signed an agreement, warranty, utilisation rights, disclosure rights, etc., etc., are all strictly limited to the terms of the agreement. In particular, if the agreement does not mention an item, you are not entitled to it. For example: Unless the agreement says otherwise, you are not allowed to use (parts of) the HIRLAM software. Contact the HIRLAM system manager, Gerard Cats, cats@knmi.nl for further information. EXPERIENCED USERS. (skip this section if you are not an experienced user). Frequently I receive error reports from experienced users that are the consequence of not following up the instructions below. This section should help you not to fall into the pitfalls again! These are the currently known pitfalls: If attempting to install the documentation, do not forget to set the environment variable SRC2HTML (probably to `ALL'). INTRODUCTION. Please note that the HIRLAM system has been primarily designed for routine weather forecasting. Reaching high performance in routine, operational, conditions has higher priority than ease of installation. As a consequence, installation is not extremely simple.... It is assumed that on your computer system, there is somebody responsible for the installation of the HIRLAM system. In the following this person is referred to as 'the local HIRLAM system manager'. There is no objection against having more local system managers on one system, except perhaps for some wastage of disk space, and duplication of efforts. Your position may be one of three possibilities: you may have been chosen to be the local system manager, you may have chosen to be your own local system manager, or you may be a HIRLAM user, while somebody else is your local system manager. The description of the HIRLAM system assumes the last of these possibilities in the way it addresses you and references the local HIRLAM system manager. If you are a (the?) local system manager, then you should also perform the tasks that are described here as the local system manager's. It is assumed that the local system manager makes a test run before releasing the system (but of course after installation). This test run is required to complete the installation procedures: without it, the system will probably not be useable by somebody else. Below are: A description of the environment and utilities HIRLAM requires; Instructions how to install the HIRLAM system; Instructions how to install the test data set; Instructions how to start a HIRLAM run, and where to find its output; Some remarks on problem handling. REQUIRED ENVIRONMENT AND UTILITIES. The HIRLAM system needs a computer system running Unix, with Fortran (77, some 90 extensions, in particular, `automatic arrays') and C compilers. Some of the Unix utilities that are used by HIRLAM may not be available on your system, or not be of the required quality, flexibility, etc. etc. Known problems have been treated by putting them in environment variables, to be set by the local sytem manager; see the file Env_system (see below). Utilities that are not available will have to be emulated; the HIRLAM system provides emulations for them (as far as known). The (more or less standard) Unix utilities `make' and `rcs' must be at least at the level of the GNU versions. If they are not, the local system manager should install them in the subdirectory `bin' of a directory pointed to by the environment variable HL_UTIL. On some systems, GNU `make' may be available as `gmake'. Warning: On many a system, the `co' command from the rcs system is deficient, leading to Fortran compilation errors. Make sure you have `co' of at least GNU quality. Internally HIRLAM uses files in the HDF format, developed by NCSA. So the HDF utilities must be available on your system. See http://hdf.ncsa.uiuc.edu. HIRLAM uses the `original HDF', not HDF5. The local system manager should also install the utility `mandtg' in $HL_UTIL/bin. How to do this is explained below. INSTALLATION OF THE SYSTEM. You have un-tar-red a file containing at least: README This file Env_system A script to make your computer system known to HIRLAM Doc_system A read-me file to describe Env_system Install A script to install the HIRLAM system problems A file describing problems seen before. emulations A directory with scripts to emulate utilities used by HIRLAM but known to be unavailable on some Unix systems: Mkdir A script to emulate 'mkdir -p', i.e. recursive mkdir whence A script to locate executables in your path ja A dummy script for job accounting, may be replaced by something more useful for your system if you wish so qsub, qdel, and qstat emulate NQS (Network Queuing System) commands In addition, you may have created tar or gzip/tar files, with HIRLAM sources, data files, scripts, or a test data set. If you or the local system manager have not done so earlier, you should edit Env_system (please read Doc_system on this), and store it somewhere in a safe place. Afterwards, you may re-use this saved version. Then edit `Install' to make it recognise your version of `Env_system'. If parts of the HIRLAM system are to be installed on a system that cannot be addressed by Unix commands like `mkdir' or `cp' you will also have to edit Install accordingly (the affected lines are marked by a comment). If your HIRLAM export file contains the HTML documentation system, and you want to install this on the same computer as the HIRLAM system itself, you should now set the environment variable `SRC2HTML' (set to `ALL' to do all sources) and, if desired, set `LOCAL_HTML_REFS' (set to the full path of the local HTML file to be included in the index of the documentation system). For details, see the file Doc_install_HTML. By the way, installing the HIRLAM HTML documentation system is typically a task for the local HIRLAM system manager. Because it treats several thousands of files, it may take some time to complete, so you would not want to do it more often than necessary. Now execute the modified `Install'. If this completes OK, it will report the names of Makefile and Makefile_x on its last few lines of output. Remember those names, see below). This should complete the installation if somebody else than you is the local HIRLAM system manager; but his tasks are not compete yet: The rest of this section on installation of the system describes the actions that you, if you are the local system manager, will have to take to install the reference system, utilities used by HIRLAM, etc., before you or somebody else can use HIRLAM. If your HIRLAM export file contained a new release of the HIRLAM script `Hirlam', please edit it to make it recognise your version of `Env_system' as well. You'll find the script `Hirlam' in the directory $HL_SCR, where HL_SCR is as defined in Env_system. If your operating/compiling system has not yet been implemented in the HIRLAM reference system, you will have to implement its compiling flags by modifying the Makefile. Also, you may need to modify Makefile_x, if your loading sequences have not been implemented (this would normally only be the case if you want to use extraneous libraries like MPI). The location of Makefile and Makefile_x was reported by Install. If they have to be changed, edit them, store them somewhere, and modify Env_system (section 8.1) to make the environment variables Makefile and Makefile_x point to the full path of your revised copy. If you prefer, you may leave them where you found them, so there is no need to rename the files. If you decide to rerun Install later, either to retry the same installation or to install a later HIRLAM release, an existing Makefile or Makefile_x will not be overwritten. In stead, a new one will be created, with the same file title, but with suffix .new appended to it. The reason for this preservation of older versions is that Makefile and/or Makefile_x may have taken some of your time to find optimum compilation flags and/or loading sequences for your system. It would be a pity to lose the files. Yet, you should verify that your old version is still suitable for the new HIRLAM release. So examine the differences carefully. If your treatment of Makefile and/or Makefile_x requires a change in Env_system, to implement the new full path names of those files, there is no need to rerun Install afterwards, even though Install used a now obsolete version of Env_system. Note that a re-installation of HIRLAM scripts will overwrite existing scripts. So you should keep a copy of scripts that you modified extensively from the reference system versions in a safe place! emulations ---------- Some emulations are provided just to facilitate HIRLAM installation. They are not guaranteed to emulate the originals in all circumstances. Note that they are included in the HIRLAM export files with the execute permission switched off to avoid interference with utilities available on your system. Swith the permission on for those you really want, and copy those from the directory `emulations' to a directory in the HIRLAM system path, e.g. $HL_UTIL/bin. If your system does not have a utility for `whence' or `mkdir -p', you may use the emulations. HIRLAM uses NQS commands to launch jobs. If your system does not have Network Queuing System installed, use the emulations of `qsub', `qdel', and `qstat'. If you want to have diagnostics of the resources usage of the HIRLAM system, you should have a utility `ja' in your path. If you don't have `ja' and do not care about accounting, you may use the emulated `ja'. date/time handling `mandtg' --------------------------- The installation procedure of the program `mandtg', to manipulate date/time groups (yymmddhh) has not been streamlined yet. It is assumed that `Hirlam' is in your path. If it not, replace `Hirlam' by `/Hirlam' in the instructions below. Type: cd ~/ref Hirlam BootLib util port # create the libraries util and port rm -f $HL_DATA/mandtg??.x # only needed if there was a $HL_DATA/mandtg??.x # replace $HL_DATA as defined in Env_system Hirlam sh Boot PGM=mandtg 1999123123 + 1# create and run executable This should print the string 2000010100 on its last line of output, and, if you have write permission in $HL_UTIL/bin (hope this implies you are the local HIRLAM system manager!), this should have created $HL_UTIL/bin/mandtg. You may verify the correct installation of `mandtg' by typing: Hirlam sh mandtg 1999123123 + 1 # should print 2000010100 completion of installation -------------------------- To complete the installation the local system manager should run the experiment `ref'. This also serves to verify the correctness of the installation. In order to be able to run this experiment, the test data set (file testset-export.tar.gz) must have been in the export package. Run the experiment `ref' by typing cd ~/ref # go to working directory of experiment ref Hirlam start # start the experiment If Hirlam is not in your path, replace the last line by start INSTALLATION OF A TEST DATA SET. If you received a packed test data set, (file testset-export.tar.gz) the installation of the system should have produced the test data in $HL_REF_FS/tst. If the test data set was not available at the time you executed `Install', and you want to install the test data afterwards, you may either simply rerun `Install' (note: this may overwrite files that you modified after your previous execution of `Install') or extract section 5 from `Install' and execute this section only. RUNNING HIRLAM. If you have the HTML documentation system, you may find it useful to have a look at it before diving deeply into HIRLAM. You'll enter the documentation system by typing `netscape $html/index.html', assuming `netscape' for browsing and $html to be set as given in `Env_system'. The following is the least you'll have to know. In case of conflicting information, it is good to realise that the HTML documentation usually is less up to date than we would like to have it.... HIRLAM runs are identified by a (preferably 3 character) code, the experiment identifier EXP. To set up an experiment, choose an experiment identifier (NOTE: experiment identifier `ref' is reserved for the system installation runs, and `tst' for the test data set), then create the directory HL_WD (usually $HOME/$EXP). Then extract the scripts and Fortran/C sources that you want to change. You do that by typing: `$HL_SCR/Hirlam cm ' where: HL_SCR is the directory with HIRLAM scripts, as defined in `Env_system' a name you give yourself, at most 11 characters is either `scripts' or the 4-character HIRLAM library name, e.g. `grdy' for grid point dynamics is a list of sources you want to extract Modified sources will only be seen by your experiment if they are in the subdirectory of $HL_WD. To achieve this location of the files, you should execute the above command in working directory `$HL_WD' (probably HL_WD=$HOME/$EXP). Probably you'd at least want to change the four-dimensional domain specification (date/time of your experiment, integration area) from those used in the reference system. You'll achieve this by the sequence: cd $HL_WD $HL_SCR/Hirlam cm scripts Env_domain # get Env_domain and edit scripts/Env_domain. If you want to run forecasts without analyses, you should similarly copy and edit `Env_expdesc' (set `CYCLE=:fo:' to do forecasts only). If you want to modify (Fortran) sources for your experiment, you should store the modified source in a subdirectory of HL_WD; this subdirectory should be named with the 4-character HIRLAM library name. E.g. `$HL_WD/grdy/DYN.f' will override routine DYN.f from library grdy. To have a new DYN.f and SL2TIM.f, type: cd $HL_WD $HL_SCR/Hirlam cm grdy DYN.f SL2TIM.f and edit grdy/DYN.f and grdy/SL2TIM.f. Beware: if you want to modify the script `Hirlam' itself, you might type, as per above: cd $HL_WD $HL_SCR/Hirlam cm scripts Hirlam but note that you would get the reference script `Hirlam' then, which deviates from the script that is valid for your system, if alone in the full path of the file `Env_system'! HIRLAM execution is initiated from the script `Hirlam', which may be looked upon as the `HIRLAM console'. The suggested way to start an experiment is to change working directory to the experiment's working directory (HL_WD, e.g. for experiment `ref' the default as defined in `Env_system' is $HOME/ref; if directory HL_WD does not exist yet, you will have to create it) and then to type `$HL_SCR/Hirlam start' (where HL_SCR is the directory with HIRLAM scripts, as defined in `Env_system'. If your experiment is not `ref', you should set DTG (format yyyymmddhh) either by `$HL_SCR/Hirlam start DTG=1995070300' or by having DTG in your environment. If your experiment is to run more cycles, you should similarly set DTGEND. If an experiment failed, you may re-submit it, after removing the cause of failure (!), with the same experiment identifier. But never re-use an experiment identifier, if the experiment is meteorologically different, and especially not if the integration area has changed. The only exception is that you are allowed to re-use an identifier if the only change is the date/time of the experiment; in fact, you are even encouraged to re-use the experiment code in that case, because many expensive parts of HIRLAM may and will then be skipped, including compilations. HIRLAM produces the following output files: - Log file of the run: in $HL_DATA, file titles start with `HL_', followed by the experiment code ($EXP) and a number of digits derived from the start time of the experiment - analysis and forecast fields: (format: GRIB-coded fields; the file structure is `ASIMOF', which can be read by most GRIB decoders written in C, or by routines as provided in the HIRLAM libraries `port' and `gcod'). files are in $HL_ARC/$EXP; file titles are `an', or `fc', resp., followed by $DTG (yymmddhh) of the initial (analysis) time, and, for forecast fields, by forecast length (ll). These files contain the complete set of fields to describe the state of the model atmosphere (e.g., they contain the fields on the model's eta-coordinate levels). HIRLAM also produces `postprocessed' fields, (e.g., fields on standard pressure levels, pmsl) in files with the same title but with `pp' appended (e.g. `fcpp'; exception: analysis files have `00pp' appended, so: `an00pp'). - Time series files, containing 'meteogram' information, in $HL_ARC/$EXP; file titles `ts00. The format of these files is GRIB-like data, in ASIMOF file format; easiest read by routines from the library `tsfs' as included in the HIRLAM system. - (from the verification package): plain ascii files with summaries of verification of fields against observations files are in $HLARC/$EXP, with title ve$DTG; log files are in $HL_DATA with and called ve$DTG.out PROBLEMS. If you have problems, please first read the file `problems', before contacting the HIRLAM system manager. Your problem may have been encountered before! Especially, be aware that the available documentation is not always up to date. This holds in particular for the recipes for local installation....