To: casnet
cc: (bcc: Gregory A. Cuzzi)
From: "Gary Blumsohn" <g.blumsohn.1@alumni.nyu.edu>
Date: 06/15/99 05:27:09 AM
Subject: Software Accuracy
There's an interesting article in the June 1999 edition of the Journal of
Economic Literature on "The Numerical Reliability of Econometric Software,"
by McCullough and Vinod. The authors go through a series of ways in which
various econometric packages can produce wrong answers, with such things as
random number generators that aren't sufficiently random and statistical
distributions that are wrong.
While I would assume that there are few actuaries who use the econometric
packages tested, the authors note (p. 651) that "we strongly caution any
economist who uses a spreadsheet package for econometric estimation to
benchmark the package first." Also, (p. 658) in a section on the inaccuracy
of statistical distributions in some of the packages, they note that "Even
more serious inaccuracies were revealed in the statistical distributions of
Excel 97."
[They refer to articles by Knusel: "On the accuracy of Statistical
Distributions in Microsoft Excel" Computational Statist. Data Analysis
26:3,
p. 375-77 (1998) and McCullough and Wilson "On the accuracy of statistical
procedures in Excel 97" Computational Statist. Data Analysis (1999
forthcoming).]
Many actuaries (including me) use Excel, together with @Risk for
simulation.
We generally trust our results. I'm curious whether anyone has found
errors
or problems with these programs, and, more specifically, what we should
look
out for.
About 5 years ago, I found a problem with the then-current version of @Risk
for Excel on the Macintosh. I was simulating from a distribution of loss
development factors, so all the simulated numbers should have been between
about 1.0 and 1.2. About once or twice in a thousand simulations, though,
the distribution would pop up a weird number, like 14.000 -- always a
whole
number, and certainly it appeared with more frequency than one would expect
from the statistical distribution. I contacted the software maker, and
they
replicated my result, but I never found out whether they got to the bottom
of the matter.
Anyone else have any war stories?
Gary Blumsohn
Visit the CAS Web Site at http://www.casact.org
===============================================
To subscribe or unsubscribe from CASNET:
Send an e-mail to caslists@lists.casact.org
Type in the body join casnet to subscribe
or leave casnet to unsubscribe.
Visit the CAS Web Site at http://www.casact.org
===============================================
To subscribe or unsubscribe from CASNET:
Send an e-mail to caslists@lists.casact.org
Type in the body join casnet to subscribe
or leave casnet to unsubscribe.