In the course of a software evaluation project the evaluation team narrows down the list of software solutions that have the potential to make it on the shortlist. Typically, however, this happens late in the process. This article describes an approach which helps you achieve this objective much sooner.

As a starting point, we presume that there are no political constraints that influence the outcome of the evaluation process. Sometimes, the winning product has already been selected before the evaluation actually starts and the only objective of the evaluation is to justify the acquisition of the favored product. Sometimes, only a few products are included in the evaluation in order to save time, which, however, poses the risk that the most suitable product is not on the list.

Depending on the type of software, the number of potentially suitable products may be in the two-digit range. Of course, it would involve too much effort to subject all products to a thorough evaluation.

To minimize effort, the recommendation is to perform the evaluation process twice. In the first instantiation you focus on the most critical requirements. You identify the most critical requirements in the following domains:

• Business partner-related requirements

• Functional (behavioral) requirements

• Non-functional (non-behavioral) requirements

Business partner-related requirements encompass requirements that you expect software vendors to meet (e.g. vendor shall have been in business for at least one year, vendor shall be able to offer 24/7 first level support, etc.).

Functional requirements focus on features, while non-functional requirements primarily address quality aspects (they are often called qualities of a system).

You can import partner-related requirements, functional and non-functional requirements from the requirements repository (a license is required for functional requirements). The requirements repository contains requirements of varying granularity. As far as functional requirements are concerned, you would focus on coarse-grained, lifecycle-oriented requirements.

You may end up with 50-100 absolutely critical requirements. Now, the next step would be to create a Request for Proposal (RFP) and distribute it among vendors. After RFP Responses have been evaluated, there may only be a couple of vendors left that meet all critical requirements.

In the next instantiation of the evaluation process you would add lower-level requirements, which eventually help you identify the products that should be on the shortlist.

On the downside, two instantiations of the evaluation process require more time overall. However, and that is the good news, the workload of the evaluation team decreases. You will most likely find that the total number of working hours used is less than with the conventional evaluation approach.