star.gif (89 bytes) HOME star.gif (89 bytes) ADMB HOME star.gif (89 bytes) ADMB WORKSHOPS

Comparison between AD Model Builder and other statistical modeling packages

There are four comparisons descibed here, S-Plus was unable to complete the test. The simpler forestry model is included to show a comparison between S-Plus and AD Model Builder. Here is a link to the Forestry Example

Another comparison between S-Plus and AD Model Builder. Here is a link to the Salmon tagging example

A comparison between the R routines lmer and glmmPQL and AD Model Builder. Here is a link to the bivariate probit GLMM simulation study

A comparison between SAS NLMixed and AD Model Builder. As far as we know NLMixed never actually could be made to work with this example. Nonlinear mixed model example

A negative binomial loglinear mixed model. There is no comparison because there is no R or Splus routine that can handle this problem except ours which is written in AD Model builder code. negative binomail mixed model example

Description of the fisheries model test

The performance test described here were performed by independent (that is independent of Otter Research Ltd.) researchers Jon Schnute and Norm Olsen at the Pacific Biological Station in Nanaimo, British Columbia.

To compare the performance of AD Model Builder with other statistical modeling packages a "typical" fisheries management model was chosen. This is a catch-at-age model which is described in Schnute and Richards (1995). It is similar to the CATAGE example described in the AD Model Builder documentation. For the runs done here the model had 100 parameters. The model was coded in AD Model Builder, Gauss, Matlab, and S-plus. All versions were given the same initial starting values for the parameters and the various optimization schemes were run until convergence. The S-plus version crashed due to lack of memory (this is apparently due to either a memory leak in S-plus or inefficient garbage collection). For the Gauss runs the Gauss optimization toolkit solver, Optmum, was employed. For MATLAB runs the optimization toolkit solver, Fminu, was employed.

Modeling Package msec/function call number of function calls time to converge
AD Model Builder 131 291 38 seconds
Gauss 167 23,365 1.08 hours
Matlab 639 18,360 3.25 hours
S-plus n/a n/a n/a

These results were presented in a paper presented by Jon Schnute at a meeting of the Resource Modelers Association in Seattle, Washington, June 16-18. Code for these models was written by Norm Olsen.

Gauss and MATLAB perform many more function evaluations than AD Model Builder because they estimate the derivatives by finite differences. In contrast AD Model Builder can compute exact values for the derivatives at the same time as it evaluates the function and this extra computation requires only about 4 times as much time as it takes to calculate the function itself. Even with this extra overhead for derivative calculation, AD Model Builder was still slightly faster per function evaluation than Gauss. This speed reflects the efficiency of compiled C++ code.

Schnute, Jon, T., and Laura J. Richards. The influence of error on population estimates from catch-age models. Can. J. Fish. Aquat. Sci. 52: 2063-2077 (1995).



Otter Research Ltd.
P.O. Box 2040
Sidney, B.C., V8L 3S3
Canada
Phone : 250-655-3364
Fax: 250-655-3364
E-mail: otter@otter-rsch.com

Updated February 2001

Otter Research Ltd.