Data Criticality in Evolving Markets

As commodity markets evolve in tandem with advents in communications technology, the criticality of data management becomes a lynchpin of corporate success or failure. Advanced analysis on market movement can lead to substantive market gains, while an inability to assess risk in a timely fashion can hinder corporate responsiveness, stagnate growth or result in unmitigated risk. Corporations must be able to collect, validate, harmonize, analyze, report and distribute massive amounts of market data (often in real time) to value and mark trades and to dynamically manage market risk. Intra and end of day data management processes which feed enterprise trade and risk systems are now mission critical components of any organization that actively trades in open markets. Energy trading, with electricity as a prime example, showcases the growing challenges of managing data in evolving markets.

For many years, electricity markets throughout North America have been in a constant state of transition. The advents of wholesale competition and power marketing can be traced back to a Federal Energy Regulatory Commission (FERC) experiment in 1987 that saw the formation of the Western Systems Power Pool (WSPP). WSPP membership facilitated trade between franchised utilities creating inter-regional trade efficiencies; the need for standardized product definition and price indexing materialized soon thereafter. In the early days, pricing was by way of splitting the savings and self-reporting; products traded were essentially a 2 X 2 matrix of peak / off-peak hours for firm / non-firm power. Market transparency was minimal and audit ability almost non-existent; except through the reporting to FERC. It was still very much a handshake and a mark-the-blackboard power trading environment. Furthermore, the telecommunications and information technology industries were still in their infancy; many of the relics of information exchange for commodity trading were, by today’s standards, primitive and almost absurdly inefficient: end-of-day fax bulletins of settlement prices; phone calls to brokers for “real-time” prices; data terminals exclusive only to large-scale corporate trading entities. In essence, the old paradigm of data flow and management was severely limited by 1) the amount of readily available data, 2) the transmission capabilities for data, and 3) the costs of data storage and maintenance.