She stressed that models are not absolute truths, but rather tools that offer generalized best estimates. They can contain uncertainties, limitations and even inaccuracies, she warned, insisting they are not designed to replace underwriters or be the final word on which risks are acceptable to an insurer.

Catastrophe modelers that spoke to National Underwriter agreed their products and systems are essentially support tools.

Explaining his view of what models can do for insurers, Jayanta Guin, senior vice president of research and modeling at AIR Worldwide, said that “models provide a robust framework to determine, 'What is risk?'” He said the models help give an insurer a full probabilistic view of its exposure, and act as a tool to help the company make a determination on how to minimize risk.

Models have also “introduced a standard into the industry where everyone is speaking the same language,” Mr. Guin added.

Particularly, he said, “models are great tools for [companies] looking to expand,” since models can help companies identify areas for growth.

The issue, then, is not so much what the models are, but rather how insurers are using them, and then interpreting the results. That, according to Mr. Guin, is up to the insurance carriers. “How each individual company makes decisions is largely up to them,” he said. “Models are tools.”

But to make those decisions, Ms. Clark told NU that insurers need to understand the models and the uncertainties contained within them.

She said companies should ask whether the models can be relied upon to produce credible information. The answer, she said, is a “resounding yes” in many areas, but other areas are more questionable.

For example, insurers sometimes use a model's “location level loss estimates” to optimize their portfolios by cutting off their exposure in areas where the loss estimates are high, Ms. Clark noted.

But at the location level, model estimates are subject to “very wide swings,” she warned, pointing out that a change to a detailed model assumption of just 10 percent could cause a 100 percent change in a location level loss.

Mr. Guin acknowledged potential unknowns in the models. “As we try to quantify risk, there are many sources of uncertainty,” he said.

He reiterated, however, that it is up to the insurer to determine how it applies the model results in its underwriting, given those uncertainties. “When a model produces an estimate with uncertainty–and we can help people understand uncertainty–it's up to the company how [it uses] that information,” Mr. Guin added.

Robert Muir-Wood, chief research officer of another modeling firm–RMS–said modelers are working on the representation of uncertainty and helping clients see “where are we making steps of inference and how do we think about that?”

The uncertainty in models is all about the data, Ms. Clark said–or in many cases, the lack thereof. “Modelers are limited by the lack of scientific data,” she explained.

Modelers are able to obtain their best data about the damage a given catastrophe will do in a certain area when such an event actually occurs, she said.

During the active hurricane seasons of 2004 and 2005, for example, Ms. Clark said modelers were able to “slice and dice” the resulting claims data and fine-tune damage functions. Essentially, major events that produce a lot of claims can allow modelers to better calibrate the models.

But some modeled catastrophes have little historical data associated with them.

For example, Ms. Clark said the New Madrid Seismic Zone produced a series of earthquakes in the central United States in 1811 and 1812, but much is unknown about the intensity and resulting damage from those quakes.

Some newspaper accounts, she said, refer to church bells ringing and cracked sidewalks, but ultimately there is little data, and therefore there will be uncertainty in the model results for a similar event today.

Mr. Guin said data quality is not the same across all perils and regions. “One rule that can be applied,” he said, “is phenomena such as hurricanes, that occur more frequently, are better understood.”

Also using the New Madrid Seismic Zone as an example, he said the lack of loss experience data translates into estimates that are more uncertain than those for earthquakes in California, where there is more data.

However, for a catastrophe or region where there is not a lot of historical data to draw on, modelers can build on catastrophe experience around the globe, according to Mr. Guin. AIR, he noted, builds models for over 60 countries, and that collective experience can provide guidance.

In addition, he said, other scientific processes that are not data-driven can be used. For an earthquake along the New Madrid Seismic Zone, Mr. Guin noted, paleoseismology–examining rocks for signs of previous earthquakes–can help in modeling future occurrences.

The engineering component of the models, he said, are also used to determine how modern buildings would respond to an earthquake. “A lot has changed since 1811,” he noted.

Speaking to hurricanes, Mr. Muir-Wood of RMS said modelers have a good handle on Florida, where there have been many storms from which to draw data. He noted that when Hurricane Wilma struck in 2004 and threatened Miami, modelers were able to gather data on how a major urban center with high-rise, multistory buildings responds to a windstorm.

As for the hurricane risk in other areas, Mr. Muir-Wood said building stock north of Virginia remains largely untested.

While Ms. Clark said modelers still struggle with gathering a lot of wind-speed data from hurricanes, Mr. Muir-Wood said there is a good understanding of what wind speed looks like. He also said modelers have a good handle on storm surge and river flooding consequences. “It's building stock in areas that may be unknown,” he warned.

The question remains whether insurers fully understand these uncertainties when they apply model results to their underwriting.

Mr. Muir-Wood, and other experts, told NU that insurers are making a lot of progress in developing a better understanding of the models and how to interpret the results.

He added that RMS spends a lot of time teaching people “what we think are best practices.” RMS offers a certification program, he noted, that allows customers to train on the skills necessary to demonstrate a reasonable understanding of the science and technology used in the models.

Jayant Khadilkar, a partner at TigerRisk Partners–a privately held reinsurance broker and risk/capital management adviser–said he works with companies so they can make sound decisions on how to apply the information obtained from cat models. (In July 2009, TigerRisk entered into a partnership with Karen Clark & Company to help insurance companies manage catastrophe risks.)

Mr. Khadilkar said there has been progress within the industry with respect to putting the model results into proper context. “Things are moving in the right direction,” he asserted.

A key change is that understanding catastrophe risk is not a reinsurance-biased decision anymore, according to Mr. Khadilkar, who said CEOs and CFOs are now taking a direct interest in determining their catastrophe risks. “The companies we're working with,” he said, “we're working with the CEOs and CFOs and helping them understand.”

He added that insurance companies are “trying to deploy their capital in the best way possible,” and catastrophe risk “is one piece of the puzzle they have to understand.”

Ms. Clark, too, conceded that “insurers are gaining more insight into the uncertainty underlying the models.” She said her company has been working with insurers to help them understand why there is uncertainty, and how there is little modelers can do to limit that factor.

“Once insurers have that insight, they become more sophisticated model users who can put [the model output] in better perspective,” she said.

However, it is still a challenge for insurers to get away from making decisions on narrow point estimates, such as “1-in-100″ and “1-in-250″ year losses, she added, explaining that one reason for this is that rating agencies require such information from carriers, and ask for these point estimates directly from the model “and even suggest that one model should be used.”

Insurers, then, feel compelled to use those point estimates since that is what the rating agencies require. “So that's something we need to change,” Ms. Clark said.

“There is a pretty good understanding that [the models] are just tools and there is uncertainty,” according to Ms. Clark, but she said putting that knowledge into practice–using the model information properly and not just going by point estimates–is where more work needs to be done.

“[Insurance] companies have become more knowledgeable, but what do they do at the end of the day?” Ms. Clark said.

Mr. Muir-Wood and Mr. Guin said their models are only getting better as new data and technology are employed.

Mr. Muir-Wood said hurricane models are now in their fourth generation since Hurricane Andrew in 1992, adding that modelers have learned a lot from recent hurricane activity. In 2004 and 2005, for example, he said modelers learned the effects of “clustering,” where one hurricane follows the track of another.

In 2008, he continued, Hurricane Ike tested Houston and modelers were able to determine that building stock there was weaker than previously anticipated.

There has also been a lot of investment in earthquake engineering, according to Mr. Muir-Wood, including building simulation models and constructing realistic buildings at universities, and using shake tables and computer simulations to see the impact. “That's the only substitute for actual experience on the ground,” he said.

Mr. Guin said AIR models performed “extremely well” in 2004, 2005 and 2008, but those years also served to highlight the importance of data quality provided by insurers. He explained that the data used to assess risk is “equally important as the model itself.”

Mr. Muir-Wood agreed, noting that recent hurricane years revealed some differences between information collected on insured risks and what those risks actually were. He cited an example of structures in Mississippi during Hurricane Katrina that were, according to data provided, concrete hotel structures but were really floating casinos.

Lessons were learned by both insurers and modelers on collecting accurate data for risks. Mr. Muir-Wood said modelers have created tools and capabilities to measure exposure information.

“Insurers can test what they think [a risk] is versus what we know it is,” he added.