[Top <../>] [All Lists ]

	 *[Thread ]* next>

    [S] AD Model Builder + mles

from [/Ken Newman/

*To*: 	s-news@wubios.wustl.edu 
*Subject*: 	[S] AD Model Builder + mles
*From*: 	Ken Newman >
*Date*: 	Wed, 12 Jan 2000 11:59:10 PST
*Cc*: 	newman@uidaho.edu 
*Sender*: 	owner-s-news@wubios.wustl.edu

 This is not a strictly S-news posting, but may be of interest
to those who use nlmin() or nlminb() to solve optimization problems.
The program discussed below, AD Model Builder, can, however, be 
used to create DLL's that S-Plus can call.

 I wanted to report success with an optimization problem, finding
74 maximum likelihood estimates for a nonlinear model, using 
AD Model Builder (Otter Research, http://otter-rsch.com), which 
uses automatic differentiation.  My initial attempt to find 
the mle's was with nlmin() and when that failed, the next attempt 
was using NPSOL (Gill, Murray, Saunders, and Wright), which also
didn't work.

 I am definitely not an expert on optimization, so failure to 
get Splus nlmin() and NPSOL to work could be my own ignorance!

 With that caveat in mind, Dave Fournier (Otter Research)
used AD Model Builder to create an executeable program which
calculated the 74 estimates in less than a minute on a Pentium 
300 class PC.  And it output the variance-covariance matrix
for the mles.  Using Splus' nlmin(), it took typically 20 minutes 
to reach ~150 iterations at which point it usually hung, or 
occasionally came back with a false convergence message.  Using 
NPSOL, a Fortran program, results would come back within about 
3 or 4 minutes, reporting failure to converge, numerical 
conditioning problems.  For both nlmin() and NPSOL I tried lots 
of different starting values, initial step sizes, etc but always 
failed to get the mles.  Again I'm sure there are better ways to 
set up the objective function and run nlmin() or NPSOL, and these 
may solve the convergence problems. With AD Model Builder even 
relatively "naive" starting values did not yield convergence 
problems, however.

  Compared to the "default" approach this was not an 
overparametrized problem.  The "default" approach is a "saturated" 
model with 115 parameters (and n=115).  The data consist of 
recoveries of tagged juvenile chinook salmon released at two 
locations, upstream and downstream.  The recoveries of upstream 
releases occur at the downstream release location and in the 
ocean, while the recoveries of the downstream releases occur 
only in the ocean.  

 The "standard" approach for analyzing these 3 recovery types for 
a given pair of releases is to assume a trinomial model for the 2 
recovery categories of upstream releases and a binomial model for 
the 1 recovery category of downstream releases.   For a given pair 
of releases, 3 parameters are estimated: survival to downstream 
point (S), capture rate at the downstream point (pd), and recovery 
rate in the ocean (po).  These parameters are easily estimated 
for a given pair of releases using maximum likelihood. 

 The problem that AD Model Builder solved was to find mle's
for a reparametrized version of the trinomial-binomial model
where survival, S, was modeled as a logit function of 7 
covariates.  There were 48 upstream releases each paired with 
one of 19 downstream releases.  The dimension of the parameter 
space was reduced from 115 to 74 but the estimation problem was 
much harder because the likelihood could no longer be partitioned
by release "pair".

  While my experience with AD Model Builder is limited to
one problem, another plus of the program is its relatively simple 
command structure.  S-Plus-like vectorization can be used, so 
it was easy for Dave Fournier to compactly write the log 
likelihood function.

Ken Newman
Division of Statistics
University of Idaho
This message was distributed by s-news@wubios.wustl.edu.  To unsubscribe
send e-mail to s-news-request@wubios.wustl.edu with the BODY of the
message:  unsubscribe s-news