Friday, February 22, 2013

Catastrophe Modelers' Secret Sauce is in the Science

There is a big world out there filled with scientific information.  Gobs and gobs of information.  Some of it is contradicting because the science advanced or because scientists disagree.  This is normal.  People disagree and discuss before arriving at an agreement.  But when you pick and choose your information carefully, you can come up with your own unique perspective.  Your own, special sauce.  

Public Model
When it comes to catastrophe models submitted to the Florida Commission on Hurricane Loss Projection Methodology the main requirement to establish the scientific soundness of a model is its use of the peer-reviewed literature.  The literature however, represents more a stream of consciousness by the world's scientists then it does any sort of agreement on the state of scientific knowledge.  And while the Commission represents a body of experts intended to review and approve the models, the Commission's approval is a consensus on the model's satisfactory meeting of standards- not the model's reflection of scientific consensus.  So, modelers are able to pick and choose their science from an ongoing discussion in the literature to come up with their own take on the situation.   

Catastrophe modeling is a very big business and a modeling company's product is its unique perspective on risk.  The goal is to develop a perspective on risk that they believe best suits their clients needs and desires.  An analogy is Pepsi and Coca-Cola.  Both sell soda.  Both sell a similar line of products. But their personal take on soda is their unique product and is fiercely guarded.  Consumers have their taste preferences and allegiances.     

Scattered throughout this post are Wordles or "word clouds."  They are constructed from the authors last names listed in the meteorological reference section in the submissions to the Commission under 2009 standard G-1.  Three models were analysed.  AIR and RMS which are the insurance industry's dominant modeling companies and the Public Model which is kinda, sorta, more or less run by the state of Florida.  Perhaps better stated, the Public Model has the state of Florida as its main client.  There is some overlap of authors, although I did not check to see if the the same work by each author is represented.  But really, the word clouds are different and depict the different opinions on the "pertinent" science needed to construct a model.  

Also of note, is that the total number of references varied quite a bit by company.  The Public Model developed with public funds as a means of creating a transparent view into catastrophe modeling (amongst other things) lists 96 references.  The two private companies list far fewer, AIR with 52 references and RMS with 29 .  The science after all, is the secret sauce.    

2009 Standard G-1: Scope of the Computer Model and Its Implementation.
#4 Provide a comprehensive list of complete references pertinent to the submission by Standard grouping, according to professional citation standards.

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.