A far more searching, rigorous, and useful exercise is to require the vendors to perform a scripted demo. A scripted demo has the following objectives–to create: familiar and predictable scenarios your business partners understand and are comfortable with; a performance benchmark each vendor knows it must meet in order to progress further in the selection process; a situation where the vendors and business must interact in a detailed domain conversation that in and of itself provides the carrier with additional insights into the depth of the vendors' knowledge and the flexibility of the vendors' software.

Slice of Life

So, what is a scripted demo? It is a thin slice of real-life business functionality that is defined, documented, and delivered to the participating vendors that in turn have a finite period of time to configure their software to behave according to the delivered definition. Think, by analogy, of use cases, or a mini proof of concept. A demo addresses the request to "show me what you want me to see"; a scripted demo addresses the specific request "show me what I want to see." In order to illustrate, let's take the example of a carrier engaged in a policy administration system search. The choice of scripted demo usually reflects the relative importance of various business requirements or, equally important, the support for various business drivers (for the distinction between the two, revisit "Hell Is Not a Night Club," February 2007).

Let's assume when our carrier defined its business drivers, "speed to market" was deemed to be very important. In this case, the carrier would be deeply interested in acquiring a system that provides speed and flexibility in defining and modifying insurance products, including both structural components, such as coverages, limits, deductibles, and coverage rules, as well as pricing and underwriting rules, including the ability to add or modify rules that support varying regulatory requirements.

Development and Execution

With this as our working example, let's step through how the scripted demo might be developed and executed. First, the carrier has to define the scripted demo. This consists of two parts: a structural component and a functional component. The structural component requires the carrier to select an existing insurance product capable of some simplification while remaining relevant and substantial. The product may need to be simplified in a couple ways: one, to remove obscure or complex carrier-specific coverages or terms that may be hard to understand and build in a short time frame, and two, to reduce the Rubik's Cube of rate, rule, and forms requirements that comes with a product written in the multiple states. The objective here is to allow the vendor to demonstrate the speed and flexibility of its software, not to test its ability to data-enter thousands of rows in data tables that represent all the rating territories, optional coverages, and state requirements, which constitute the true and complete product.

Once the simplified product (the structural part of the scripted demo) is defined, the carrier then can focus on the functional component. The functional component usually consists of two classes of use case: use cases that validate and verify the structure provided was built correctly by the vendor, and use cases that specify modifications to the original product structure. This latter class can include use cases that modify the product itself, such as adding a coverage or changing the rules between two coverages, and use cases that focus on adding or changing underwriting rules and so modifying the system's behavior.

Additionally, the use cases must be deployed in strict sequence. The product structure initially is sent to each vendor at a specific point in time such that each vendor gets the same period to build the product in its system. This is usually done two or three weeks prior to a vendor visit at which the scripted demo will be exercised. During the intervening period, the carrier needs to provide the vendor with a subject-matter contact to answer questions, clarify issues, and in some instances, negotiate the actual scope of the script. It is not unusual for a vendor to propose minor modifications to the scripted demo to accommodate the carrier's wishes in the required time frame. It is a judgment call by the carrier as to when such an accommodation might compromise the substance or the consistency of the scripted demo across all vendors.

Flexibility Justified

Bear in mind the vendors are being called on to do something of substance in a tight time frame for no pay, so some degree of flexibility is probably justified. These conversations can, in and of themselves, provide the carrier with insights as to the domain expertise of the vendor, based on the sophistication of the questions and issues raised.

At the time either when the vendor visits the carrier or vice versa, the carrier presents the validation use cases to the vendor to verify the structural correctness of the vendor's implementation. It is important to note the vendor has not previously seen the functional scripts and therefore cannot anticipate the results required. The carrier may exercise the structure with several validation use cases to test various parts of the product structure. The carrier should be prepared in advance with expected results to verify the results quickly as they get generated. This class of use case usually is based on a routine life-cycle event, such as quoting, issuing, or endorsing a policy; often several use cases may follow a life cycle of events for one policy.

Once (and assuming) the product structure is verified, the carrier can move on to those use cases that modify the product structure or the associated underwriting rules. This class of use case may build on the prior verification use cases by leading to the modification of the expected results when these use cases are rerun. So, for example, a verification use case that earlier ran successfully and produced a specific premium and associated coverage set may, following the structure change, trigger a new or modified underwriting rule that causes an underwriting referral or an outright rejection. Similarly, if the product structure is changed, the policy subsequently may rate and issue with an additional coverage that now is mandated in the issuing state. The options are considerable.

In fact, one of the key factors to balance is to create a group of use cases that truly can be executed in the scheduled time, allowing, of course, for the likelihood not all use cases will execute exactly as expected. Some vendors offer the carrier ongoing access to the system to perform additional use cases on its own time frame. If the vendor is confident in its software, it obviously would prefer carriers work with its software than to focus on the capabilities of another vendor.

Realistic Outlook

To be realistic, an exercise as limited in scope as a scripted demo will not provide complete reassurance any given vendor is the "right one." However, injecting a performance-based step early in the selection process focuses a vendor's attention, requires it make a significant commitment to winning the carrier's business, and on occasion, leads to vendor withdrawal for one stated reason or another. On this point, let me emphasize there is no good reason for a vendor to withdraw from a scripted demo. I have been involved in software selection projects where a vendor has claimed "lack of available resources" as the reason for being unable to perform (interestingly, no vendor ever has said, "My software is incapable of responding to your requirements in the time available."), and clients have proposed giving the vendor a pass. There are two answers to this: One, if the vendor cannot support a sales cycle, how can it support an implementation, and two, if the scripted demo is a step on your selection process, it either works for all vendors or you should remove it.

Finally, it should be noted a scripted demo will work only with a short list of vendors for a couple of reasons: The preparation and execution time required by the carrier for each vendor is significant; and a vendor will not undertake the work needed to participate in a scripted demo unless it is under serious consideration and on a very short list of remaining vendors. So, a scripted demo is an effective tool in getting from three to two vendors or from two to one. However, as effective as this tool is, it requires the carrier to make a significant analytical and logistical effort, but it is an effort well worth making.

The content of "Shop Talk" is the responsibility of the author. Views and opinions are those of the author and do not necessarily represent those of Tech Decisions.