The organizational innovation experts - at your service!

ARCHIVED PAGE! this page is archived and is no longer being updated. It may or may not be out of date and links may no longer be valid. Click here to return to our home page

Home Jenni Innovation process management About Idea Management Training Services Creativity & Innovation Library Corporate Innovation Machine Report 103: The Innovation eJournal Imagination Club On-Line Imagination Club Brussels Doodles Exercises About us Partners Search Site directory Contact us Suggestions? Also by jpb.com SuperTrade Search Engine

Evaluating Ideas Determining Which Ideas Have the most Potential The problem with a good brainstorming session is that it generates a lot of ideas - sometimes more than a hundred. If you have no means of determining which ideas will best meet your needs, it is all too easy to become overwhelmed by ideas and select none of them, or to choose to implement one of the less creative ideas because you perceive it (sometimes incorrectly) to be less risky. The best approach is to choose the best ideas from all ideas generated in the brainstorming session and then to submit the best ideas to the evaluation matrix. The 5x5 evaluation matrix is very simple. You determine five criteria and compare each idea to each criterion, giving it a score of 0 to 5 points depending on how well the idea meets the criterion. Add up the scores and multiply by four and you have a percentile score for each idea. This facilitates easy comparison of ideas. Normally, you should determine the evaluation criteria and the evaluators before you begin your brainstorming session. The Criteria Your evaluation criteria should meet these five guidelines: 1. Ensure criteria measure the right things Think about your purpose for brainstorming ideas and what the ideas are supposed to accomplish. Make a list of those desired accomplishments. For example, you would most likely want a new product to have profit potential, fit well with your existing product line, not be overly expensive to manufacture, etc. 2.Questions of degree In other words, criteria should ask the evaluators to indicate a level of agreement rather than ask questions which demand an absolute “yes” or “no” answer. For example: “Is this idea profitable” is not a good criterion; “How profitable is this idea?” is a better criterion. 3.Measure positive attributes For example, if you are worried that an idea could too easily be copied by your competitors, do not use the criterion “How easy would it be for our competitors to copy this idea?”. An idea that meets this criterion would get a higher evaluation score when, in fact, you want the opposite. Instead, you should phrase the criterion “How difficult would it be for our competitors to copy this idea?” 4.Avoid double criteria “How quickly can we develop this idea and bring it to market?” is in fact two criteria stuck together. However, an idea might be very fast to develop, but very slow to bring to market. It would be better to divide such a criterion into two separate criteria. 5. Clear and unambiguous It should be immediately clear to the evaluators what you wish to measure with a criterion. Avoid vague criteria such as “Profitability”. Such a vague criterion could suggest you want to know how profitable the idea is, whether or not the idea is profitable or how quickly the idea will become profitable. In addition, it is often useful to provide guidelines for ratings. For example, “How easy would it be to implement this cost cutting measure? 0 = extremely difficult, would probably cause upheaval or significant investment; 5= extremely easy with little disturbance or investment required” The Evaluators There are two approaches to choosing evaluators... Have every participant of the brainstorming session participate in a group evaluation of the best ideas. Select a small group of experts - ideally three or four people - and have them evaluate the best ideas independently. Once they are done, compile their scores in one comprehensive report. If you choose this option, it is not necessary for the evaluators to have participated in the brainstorming session itself. Choose the ideas to evaluate At the end of the idea generation phase of the brainstorming session or ideas campaign, you should choose the most promising ideas. The best way to do this is to quickly reject ideas which you do not want to take further. Ideas should be rejected because they do not solve the problem, are irrelevant or are clearly impossible. Do not reject ideas which are simply difficult or weird. Such ideas could later prove to be your best ideas. You should also reject duplicated ideas and ideas which are components of bigger ideas (as often happens when someone proposes and idea and someone else uses that idea to create a more complex idea). The next step is to select the best remaining ideas. Do try to include some of the more outlandish ideas even if you believe they are too complicated to implement. In particular, look at the ideas you are likely to implement in the short term. Ideas which you do not wish to evaluate for short term implementation should be noted in the evaluation report (see below) and saved for future evaluation and implementation. Once the most promising ideas have been chosen, have the evaluators rate each idea against each criterion as described above. Evaluators should also be able to add remarks to the evaluation. This will allow them to clarify scores and explain how the idea could be modified to improve its score. Evaluation Report Once the evaluations are complete, the facilitator should compile the results into the evaluation report which includes: The issue being brainstormed and any background information The evaluation criteria Names of participants and the names of the evaluators (if different) The most promising ideas. Any ideas recommended for future evaluation. The evaluation scores of the most promising ideas. The evaluators' comments. An appendix with all ideas generated during the brainstorming event. The evaluation report serves several purposes. Demonstrates which ideas are most likely to succeed according to your criteria,. Defends the top ideas to senior management, financial people or clients who might question them (highly creative ideas often seem crazy at first, thus the ability to defend them is important). Provide guidance during the implementation of the idea. Final note The 5x5 criteria based evaluation is a compromise between speed and accuracy. In our experience, if the criteria are well chosen and the evaluators reasonably knowledgeable about the topic, the 5x5 criteria based evaluation can be very accurate. Nevertheless, with ideas that involve substantial financial or other risk, it is advisable to further review ideas - such as through more in-depth analysis, prototypes and limited trials.   By Jeffrey Baumgartner ©2005 Bwiti bvba