Uncertainty is ever present in the insurance business, and despite relentless enhancements in data gathering and processing power, it is still a large factor in risk modeling and assessment. This fact, driven home by recent model changes and unexpected natural catastrophes, can be disconcerting - even frightening - to industry participants.
But companies that understand the vagaries of model uncertainty and take a disciplined, holistic approach to managing the catastrophe modeling process are well positioned to adapt and outperform the competition.
Guy Carpenter has undertaken an extensive research initiative into catastrophe models and how they can best be used by primary writers of property insurance. We advocate, among other things, adopting a multiple cat-model approach to better estimate risk and control uncertainty.
Among our findings:
Vendor-reported uncertainty bands vary widely. As the chart below indicates, there is a vast difference in the vendor-reported uncertainty bands from the three major vendor models along an exceedance probability (EP) curve for the same insured portfolio. (We believe the true uncertainty band is actually larger than the largest shown here.)
Figure 1: Vendor Model Uncertainty Bands
Source: Guy Carpenter & Company, LLC
Not all models are created equal. Different vendors use different methodologies in developing their models. They have different estimation, fitting and smoothing techniques, as well as different representations of scientific details. Using multiple models “diversifies” the risk of error from these choices by allowing for independent errors to cancel each other. It also allows clients to use the best tool for a given evaluation, as certain models tend to perform better than others for certain perils, regions or classes of risk.
More data leads to less uncertainty. While various models are founded on a common set of data, they are not founded on identical data. Each has its own interpretation of the historical record, analysis of detailed scientific data (for example, wind fields or earthquake propagation), sources of vulnerability data (damageability data) and data on site conditions. Using multiple models effectively increases the amount of data bearing on the analysis.
More broadly, Guy Carpenter encourages companies to embed awareness of model uncertainty into their overall enterprise risk management (ERM) process, and make catastrophe-risk oriented decisions with a conscious eye towards the possibility of model error.
The issues of model uncertainty and change pose many difficult challenges for the industry. The “black box” should no longer be left to make the decisions; rather, it should be considered a tool to help inform decisions made by (human) professionals. This is an intuitive and straightforward prescription, but making it happen will require the consideration and engagement of virtually every segment in the industry.
Guy Carpenter works with clients to run model validations on a case-by-case basis, the results of which can suggest a preferred model for a company or a blending of multiple models. Our research into the topic is ongoing, and we continue to advise clients and industry participants on how to best utilize and consider these models.