Chapter 1 Getting started with Ad Model Builder

AD Model Builder provides a template-like approach to code generation. Instead of needing to write all the code for the model the user can employ any ASCII file editor to simply fill in the template, describing the particular aspects of the model -- data, model parameters, and the fitting criterion to be used. With this approach the specification of the model is reduced to the absolute minimum number of statements. Reasonable default behaviour for various aspects of modeling such as the input of data and initial parameters and reporting of results are provided. Of course it is possible to override this default behaviour to customize an application when desired. The command line argument -ind NAME followed by the string NAME changes the default data input file to NAME.

The various concepts embodied in AD Model Builder are introduced in a series of examples. You should at least skim through each of the examples in the order they appear so that you will be familiar with the concepts used in the later examples. The examples disk contains the AD Model Builder template code, the C++ code produced by AD Model Builder and the executable programs produced by compiling the C++ code. This process of producing the executable is automated so that the user who doesn't wish to consider the vagaries of C++ programming can go from the AD Model Builder template to the compiled executable in one step. Assuming that the C++ compiler and AD Model Builder and Autodif libraries have been properly installed, then to produce a AD Model Builder executable it is only necessary to type makeadm root where root.tpl is the name of the ASCII file containing the template specification. To simplify model development two modes of operation are provided, a safe mode with bounds checking on all array objects and an optimized mode for fastest execution.

AD Model Builder achieves its high performance levels by employing the Autodif C++ class library. Autodif combines an array language with the reverse mode of Automatic differentiation supplemented with precompiled adjoint code for the derivatives of common array and matrix operations. However, all of this is completely transparent to the AD Model Builder user. It is only necessary to provide a simple description of the statistical model desired and the entire process of fitting the model to data and reporting the results is taken care of automatically.

Although C++ potentially provides good support for mathematical modeling, the language is rather complex -- it cannot be learned in a few days. Moreover many features of the language are not needed for mathematical modeling. A novice user who wishes to build mathematical models may have a difficult time deciding which features of the language to learn and which features can be ignored until later. AD Model Builder is intended to help overcome these difficulties and to speed up model development. When using AD Model Builder most of the aspects of C++ programming are hidden from the user. In fact the beginning user can be almost unaware that C++ underlies the implementation of AD Model Builder. It is only necessary to be familiar with some of the simpler aspects of C or C++ syntax.

To interpret the results of the statistical analysis AD Model Builder provides simple methods for calculating the profile likelihood and Markov chain simulation estimates of the posterior distribution ( optional at extra cost) for parameters of interest (Hastings-Metropolis algorithm).

A short description of each example follows.

1. A very simple example. This is a trivial least squares linear model included simply to introduce the basics of AD Model Builder.

2. A simple nonlinear regression model for estimating the parameters describing a von Bertalanffy growth curve from size-at-age data. AD Model Builder's robust regression routine is introduced and used to illustrate how problems caused by ``outliers'' in the data can be avoided.

3. A chemical kinetics problem. A model defined by a system of ordinary differential equations. The purpose is to estimate the parameters which describe the chemical reaction.

4. A problem in financial modeling. A Generalized Autoregressive Conditional Heteroskedasticity or GARCH model is used to attempt to describe the time series of returns from some market instrument.

5. A problem in natural resource management. The Schaeffer-Pella-Tomlinson Model for investigating the response of an exploited fish population is developed and extended to include a Bayesian times series treatment of time-varying carrying capacity. This example is interesting because the model is rather tempermental and several techniques for producing reliable convergence of the estimation procedure to the correct answer are described. For one of the data sets over 100 parameters are estimated.

6. A Simple Fisheries catch-at-age model. These models are used to try and estimate the exploitation rates etc. in exploited fish populations.

An AD Model Builder template consists of up to nine sections. Six of these sections are optional. Optional sections are enclosed in brackets [ ]. The optional FUNCTION keyword defines a subsection of the PROCEDURE SECTION.

DATA_SECTION

[INITIALIZATION_SECTION]

PARAMETER_SECTION

[PRELIMINARY_CALCS_SECTION]

 \break 

PROCEDURE_SECTION\break
[FUNCTION]

[REPORT_SECTION]

[RUNTIME_SECTION]

[TOP_OF_MAIN_SECTION]

[GLOBALS_SECTION]

[BETWEEN_PHASES_SECTION]

The simplest model contains only the three required sections, a DATA_SECTION, a PARAMETER_SECTION, and a PROCEDURE SECTION.

Section 1 Example 1 -- linear least-squares

To illustrate the method we begin with a simple statistical model which is to estimate the parameters of a linear relationship of the form

where and are vectors, and and are the model parameters which are to be estimated. The parameters are estimated by the method of least-squares that is we find the values of and so that the sum of the squared differences between the observed values and the predicted values is minimized. That is we want to solve the problem


The template for this model is in the file SIMPLE.TPL. To make the model one would type makeadm simple. The resulting executable for the model is in the file SIMPLE.EXE. The contents of SIMPLE.TPL are. (Anything following // is a comment.)

DATA_SECTION
  init_int nobs             // nobs is the number of observations
  init_vector Y(1,nobs)     // the observed Y values
  init_vector x(1,nobs)
PARAMETER_SECTION
  init_number a   
  init_number b   
  vector pred_Y(1,nobs)
  objective_function_value f
PROCEDURE_SECTION
  pred_Y=a*x+b;       // calculate the predicted Y values
  f=regression(Y,pred_Y);  // do the regression -- the vector of 
                           // observations goes first

The main requirement is that all keywords must begin in column 1 while the code itself must be indented.

Section 2 The DATA SECTION

Roughly speaking, the data consist of the stuff in the real world which you observe and want to analyze. The data section describes the structure of the data in your model. Data objects consist of integers ( int) and floating point numbers( number), and these can be grouped into one dimensional ( ivector and vector) and two dimensional ( imatrix and matrix) arrays. The `` i'' in ivector distinguishes a vector of type int from a vector of type number. For arrays of type number there are currently arrays up to dimension 7.

Some of your data must be read in from somewhere, that is, you need to start with something. These data objects are referred to as initial objects and are distinguished by the prefix init, such as init_int or init_number. All objects prefaced with init in the DATA_SECTION are read in from a data file in the order in which they are declared. The default file names for various files are derived from the name of the executable program. If the executable file is named ROOT.EXE then the default input data file name is ROOT.DAT. For this example the executable file is named SIMPLE.EXE so the default data file is SIMPLE.DAT. Notice that once an object has been read in, its value is available to be used to describe other data objects. In this case the value of nobs can be used to define the size of the vectors Y and x. The next line init_vector Y(1,nobs) defines an initial vector object Y whose minimum valid index is 1, and whose maximum valid index is nobs. This vector object will be read in next from the data file. The contents of the file SIMPLE.DAT are shown below.

# number of observations
     10
# observed Y values
    1.4  4.7  5.1  8.3  9.0  14.5  14.0  13.4  19.2  18 
# observed x values
    -1  0 1  2  3  4  5  6  7  8        

It is possible to put comment lines in the data files. Comment lines must have the character # in the first column.

It is often useful to have data objects which are not initial. Such objects have their values calculated from the values of initial data objects. Examples of the use of non initial data objects are given below.

Section 3 The Parameter Section

Roughly speaking it is the parameters of your model which provide the analysis of the data (or perhaps more correctly is the values of these parameters as picked by the fitting criterion for the model which provide the analysis of the data). The PARAMETER_SECTION is used to describe the structure of the parameters in your model. The description of the model parameters is similar to that used for the data in the DATA_SECTION.

All parameters are floating point numbers (or arrays of floating point numbers.) The statement init_number b defines a floating point number (actually a double). The preface init means that this is an initial parameter. Initial parameters have two properties which distinguish them from other model parameters. First, all of the other model parameters are calculated from the initial parameters. This means that in order to calculate the values of the model parameters it is first necessary to have values for the initial parameters. A major difference between initial data objects (which must be read in from a data file) and initial parameters is that since parameters are estimated in the model it is possible to assign initial default values to them.

The default file name for the file which contains initial values for the initial model parameters is ROOT.PIN. If no file named ROOT.PIN is found, default values are supplied for the initial parameters. (Methods for changing the default values for initial parameters are described below.) The statement vector pred_Y(1,nobs) defines a vector of parameters. Since it is not prefaced with init the values for this vector will not be read in from a file or given default values. It is expected that the value of the elements of this vector will be calculated in terms of other parameters.

The statement objective_function_value f defines a floating point number (again actually a double). It will hold the value of the fitting criterion. The parameters of the model are chosen so that this value is minimized. Every AD Model Builder template must include a declaration of an object of type objective_function_value and this object must be set equal to a fitting criterion. (Don't worry, for many models the fitting criterion is provided for you as in the regression and robust_regression fitting criterion functions in the current and next examples.

Section 4 The Procedure Section

The PROCEDURE SECTION contains the actual model calculations. This section contains C++ code and C++ syntax must be obeyed. (Those familiar with C or C++ will notice that the usual methods for defining and ending a function are not necessary and in fact can not be used for the routine in the main part of this section.)

Statements must end with a `` ;'' exactly as with C or C++. The `` ;'' is optional in the DATA_SECTION and the PARAMETER_SECTION. The code uses Autodif's vector operations which enable you to avoid writing a lot of code for loops. In the statement pred_Y=a*x+b; the symbol a*x forms the product of the number a and the components of the vector x while +b adds the value of the number b to this product so that pred_Y has the components . In the line f=regression(Y,pred_Y); the function regression calculates the log-likelihood function for the regression and assigns this value to the object f which is of type objective_function_value. This code generalizes immediately to nonlinear regression models and can be trivially modified (with the addition of one word) to perform the robust nonlinear regression discussed in the second example. For the reader who want to know, the form of the regression function is described in the Appendix.

Note that the vector of observed values goes first. The use of the regression function makes the purpose of the calculations clearer, and it prepares the way for modifying the routine to use AD Model Builder's robust regression function.

Section 5 The Preliminary Calcs Section

NOTE: The use of LOCAL_CALCS and its variants in the DATA_SECTION and the PROCEDURE_SECTION has greatly reduced the need for the PRELIMINARY_CALCS_SECTION.

The PRELIMINARY_CALCS_SECTION as its name implies permits one to do preliminary calculations with the data before getting into the model proper. Often the input data are not in a convenient form for doing the analysis and one wants to carry out some calculations with the input data to put them in a more convenient form. Suppose that the input data for the simple regression model are in the form

# number of observations
     10
# observed Y values  observed x values
    1.4                    -1
    4.7                     0
    5.1                     1
    8.3                     2
    9.0                     3
    14.5                    4
    14.0                    5
    13.4                    6
    19.2                    7
    18                      8

The problem is that the data are in pairs in the form , so that we can't read in either the or first. To read in the data in this format we will define a matrix with nobs rows and 2 columns. The DATA_SECTION becomes

DATA_SECTION
  init_int nobs
  init_matrix Obs(1,nobs,1,2) 
  vector Y(1,nobs)
  vector x(1,nobs)

Notice that since we do not want to read in Y or x these objects are no longer initial objects, that is their declarations are no longer prefaced with int. The observations will be read into the initial matrix object Obs so that Y is in the first column of Obs while x is in the second column. If we don't want to change the rest of the code the next problem is to get the first column of Obs into Y and the second column of Obs into x. The following code in the PRELIMINARY_CALCS_SECTION will accomplish this objective. It uses the function column which extracts a column from a matrix object so that it can be put into a vector object.

PRELIMINARY_CALCS_SECTION
 Y=column(Obs,1); // extract the first column
 x=column(Obs,2); // extract the second column

Section 6 The use of loops and element-wise operations

This section can be skipped on first reading.

To accomplish the column-wise extraction presented above you would have to know that Autodif provides the column operation. What if you didn't know that and don't feel like reading the manual yet? For those who are familiar with C it is generally possible to use lower level ``C-like'' operations to accomplish the same objective as Autodif's array and matrix operations. In this case the columns of the matrix Obs can also be copied to the vectors x and Y by using a standard for loop and the following element-wise operations

PRELIMINARY_CALCS_SECTION
  for (int i=1;i<=nobs;i++)
  {
    Y[i]=Obs[i][1];
    x[i]=Obs[i][2];
  }

Incidentally, the C-like operation [] was used for indexing members of arrays. AD Model Builder also supports the use of () so that the above code could be written as

PRELIMINARY_CALCS_SECTION
  for (int i=1;i<=nobs;i++)
  {
    Y(i)=Obs(i,1);
    x(i)=Obs(i,2);
  }

which may be more readable for some users. Notice that it is also possible to define C objects like the object of type int i used as the index for the for loop ``on the fly" in the PRELIMINARY_CALCS_SECTION or the PROCEDURE_SECTION.

Section 7 The default output from AD Model Builder

By default AD Model Builder produces three or more files ROOT.PAR which contains the parameter estimates in ASCII format, ROOT.BAR which is the parameter estimates in a binary file format, and ROOT.COR which contains the estimated standard deviations and correlations of the parameter estimates. The template code for the simple model is in the file file SIMPLE.TPL. The input data is in the file SIMPLE.DAT. The parameter estimates are in the file SIMPLE.PAR. By default the standard deviations and the correlation matrix for the model parameters are estimated. They are in the file SIMPLE.COR

 index      value      std dev       1       2   
    1   a  1.9091e+00 1.5547e-01       1
    2   b  4.0782e+00 7.0394e-01  -0.773       1

The format of the standard deviations report is to give the name of the parameter followed by its value and standard deviation. After that the correlation matrix for the parameters is given.
NEXT