We end up with the evaluation spreadsheet. Each row lists some feature or sub-feature that someone has determined the technology must have. These features are all weighted according to their importance.

The columns then reflect all the options. Fill each cell with a number assessing how well this option delivers this feature, and you calculate a score for each option. Select the one with the highest weighted score, and you have the best technology for your needs.

Problem is, this rarely works.

For a start, the features and weightings aren’t objective. Someone gathers requirements, filtering them as they go. People argue about weightings. Ultimately, the person with the most power decides. We’ve just shifted the politics into the structure of the spreadsheet.

Research into the way experts make decisions shows that they rarely think through a set of options and assess them against objective “decision criteria”.

In situations where they need to integrate a lot of information, deal with uncertainty, and balance the concerns of diverse perspectives, experts go through the following stages:

They imagine themselves into the situation.

1. They identify a single option that will most likely meet their needs.
2. They test this option, mentally, against the situation.
3. If this option works well enough, they don’t waste time on further analysis. They select it and get into action.
4. If the option doesn't work, they adapt and adjust it in their minds. If they can find a way to make it work, they use it.
5. If they can’t find a way to make it work, they look for other options.
6. They may go through this loop several times, using the selection and testing and adaptation of options to improve their understanding of the situation. They evolve a workable solution.

Experts can do this remarkably quickly. Fire fighters and other emergency service workers go through this loop in life-and-death situations in fractions of a second.

We’re not fire fighters Lives don’t depend on the split-second timing of our decisions. So it makes sense to balance scenario-based decision-making with some thought about features, functionality and weighing off the different options.

We should make it easy to imagine ourselves into the situation we’re trying to address. What would it be like to use this system, work with this agency, etc? Write scenarios rather than feature lists, and ask vendors to explain how their technology fits each scenario.

We need to give ourselves hands-on time with each option, so we can experience how they really work. Vendor-driven presentations aren’t enough. Pilots are ideal, or hands-on workshops where we can try the options for ourselves during the course of the procurement.

Via: http://econsultancy.com/ru/blog/62153-how-we-buy-technology