Data and Disaster: The Role of Data in the Financial Crisis

Motivation: Since 2007 a global financial crisis has been unfolding. The crisis was initially caused by defaults on subprime loans, aided and abetted by pools of asset-backed securities and credit derivatives, but corporate defaults, such as that of Lehman Brothers, and outright fraud have also contributed to the crisis. Little research has been published investigating the role of data issues in various aspects of the financial crisis. In this paper we illustrate how data that was available to underwriters, credit agencies, the Securities and Exchange Commission (SEC), and fund managers could have been used to detect the problems that led to the financial crisis.

Method: In this paper we show that data quality played a significant role in the mispricing and business intelligence errors that caused the crisis. We utilize a number of relatively simple statistics to illustrate the due diligence that should have, but was not performed. We use the Madoff fraud and the mortgage meltdown as data quality case studies. We apply simple exploratory procedures to illustrate simple techniques that could have been used to detect problems. We also illustrate some modeling methods that could have been used to help underwrite mortgages and find indications of fraud.

Results: In both the Madoff fraud and the mortgage crisis a number of statistical tests could have been applied to uncover fraud and to provide a warning of the deterioration of the quality of mortgages underwritten.

Conclusions: Data quality issues made a significant contribution to the global financial crisis.

Keywords: Data, data quality, financial crisis