The HIRLAM system; an introduction

Gerard Cats, Toon Moene, Ben Wichers Schreur

Available subjects

See also the following documents:

Version Control
Details of the version control system to be implemented soon after today (20 May 1999)
Local installation
Notes on how to copy the reference system from ECMWF to your local system and incorporate local modifications.
GNU utilities
HIRLAM requires some Unix utilities to be compatible with their GNU versions. This note describes which and how to obtain and install the GNU versions.
Libraries, makefiles
A general introduction to the organisation of HIRLAM sources in libraries, and to the way HIRLAM sources are compiled into archives and executables.
Specification of boundary data
Notes on how to set environment variables to specify the location and properties of the files containing the lateral boundary conditions.
Setting up a HIRLAM experiment
How to set up and run an experiment at ECMWF, or locally; including suggestions for operational HIRLAM set up.

Terminology in this document

We assume that the reader has a reasonable knowledge of Unix. On top of that, we use some terminology that is explained here.

(Pseudo) environment variables

The HIRLAM system derives the real environment variables from a file which is constructed during the installation procedure on a certain computer system. There is a `read-me' file (named Doc_system) ,documenting how this file is to be constructed; a sample file is also available (named Env_system). (Another `README' file describes the installation procedure). The following variables are set by Env_system:

CS OS COMPCENTRE
HL_ARC
HL_REF_FS HL_REF_CP HL_UTIL HL_SCR HL_RESOURCES
BUFRTAB_DIR
HL_WD HL_DATA HL_EXP
SETENV
PATH
MAKE CO MKDIR WHENCE
NAMLIS_E
ECLIB SCILIB
HL_COMDAT HL_LIB

the pseudo environment variables are specified in a number of files:

Env_domain
Specifying the (4 dimensional) integration domain
Env_ecmwf
Describing properties of the nestor model (the model to provide lateral boundary conditions)
Env_expdesc
Defining the experiment configuration
Env_input
Defining input files
Env_qsub
Specifying queuing system parameters
Env_send
Specifying how output files should be sent to a different network

Currently, the HIRLAM system collects these files into one, of which the title is given by $SETENV, which then is read in by each HIRLAM script. In future, however, it is intended to let scripts only read those files that are actually needed.

The directories used by the system

The HIRLAM system makes a distinction between a number of directories. There are directories containing the reference system, directories specific to the user and directories specific for an experiment run by the user. The system distinguishes between permanent directories, directories on a volatile medium, and directories in mass storage. PEVs tell the system the names of those directories. See the list of PEVs pointing to directories.

Modified runs

Items in the reference system that the user may want to modify for his/er experiments are:

Modifications may consist of new developments or reversion to an earlier version. No provisions have been made yet for the latter, except that the sources are maintained using RCS. At a later stage, we will try and create facilities to revert to earlier versions, as need arises.

After the modifications have been created the user may submit the experiment as described in the document on `Setting up a HIRLAM experiment'.

Modifying scripts

The user can replace a script from the reference system, or add own scripts, by creating the script in the directory $HL_WD/scripts. If it is to replace a reference script, it should have the same name as the script in $HL_SCR. It should have execute permission. The search path should have (and has, unless the user changed it explicitly) $HL_WD/scripts before $HL_SCR, so that scripts in $HL_WD/scripts take precedence.

Modifying sources

The reference system uses the sources through the script `Make_ref'. This script gives precedence to modules that the user created in subdirectories of $HL_WD. The subdirectory should have the same name as the library that contains the module (these names are 4 characters long see the list of available libraries).

Modifying an existing Fortran or C source (codes or headers)

A modified source file should be placed in the directory $HL_WD/$lib where $lib is the (4 character) HIRLAM library name. If, for example, the user wants to modify the subroutine cloud, in library phys, (s)he should create the source $HL_WD/phys/CLOUD.f (Note that HIRLAM uses capitalised names for the Fortran sources). The system will automatically take the new routine to replace the reference version. If the modification includes a change of dependency on include files, the user should also create an appropriate makefile, $HL_WD/phys/makefile. If the compilation sequence (e.g. a flag)has to be changed, the user should provide a proper alternative makefile, $HL_WD/phys/makefile.qrk.

Adding a new Fortran or C source

If the user wants to add a routine, say routine foo, to a library, say phys, then (s)he should again put the source in the directory $HL_WD/$lib, (in this case $HL_WD/phys/foo.f; note that names of new sources may be in lower case, but o not choose a name that exists already in upper case!); in this case, the user must also modify the makefile, to include foo.o in the list of objects, and to mark the files that foo.o depends on. Also, it may be necessary to specify certain compilation flags for foo.o. See the previous subsection on how to effectuate this.

Modifying the makefile

Each library has a standard makefile, called 'makefile'. A new makefile for a library will automatically be used by the system if it is placed in the directory $HL_WD/$lib, but note that a new makefile will only lead to recompilation for objects that have to be recreated anyhow; so for a new makefile to take effect, you should remove the old objects by (e.g.)
'rm -f $HL_DATA/$lib??/*.o'

Machine specific makefiles

The system will take the following makefiles on top of the standard 'makefile'as described in the previous section:

Makefile.qrk is introduced or modified similarly to the standard makefile., namely by putting the modified version in $HL_WD/$lib.

Modified versions of Makefile and Makefile_x can be stored in $HL_WD/scripts; this directory is experiment-specific. If the user needs modified versions for all experiments, (s)he may store those versions somewhere else. The existence of those modified versions should then be communicated to the HIRLAM experiment by setting the environment variables Makefile and Makefile_x, resp., to the full path. E.g., if there is a modified Makefile in $HOME, the user could submit the experiment by typing from the working directory $HL_WD, and assuming HL_SCR to be set:

Modifying input files

To modify an input file, the user could modify the script(s) that refer to it, in such a way that those scripts now refer to the modified file. The resource file mechanism may work better.

Modifying documentation

With each source file, there is a documentation file with a name obtained by appending '-d.html' to the source file's name. There are several html links connecting documentation to source code sections and vice versa. Please maintain and augment the documentation files!

How to modify the reference system

The HIRLAM reference system at ECMWF is maintained by the HIRLAM system manager. He is the only one allowed to modify it. If you have a suggestion for improvement then, dependent on its nature, you should either communicate it to the system manager (this is the case if it is a correction to a bug) or to the HIRLAM management team (if it is a meteorological/scientific change). In either case, however, the modifications must be in a certain format in order to be acceptable to the system manager. Requirements are:

There are some suggestions for coding to assist you in maintaining a match between the source and the documentation files.

The HIRLAM system on a national computer system should be maintained by the 'local system manager'. Modifications to that system are at his/her discretion.


Last modification of this document:Thu Jan 1 02:00:00 1970