Making Decisions: Multi-Attribute Value Theory (MAVT) in Practice

Category: Management

1. Brief Summary 

US military organisations use a plethora of telecommunication devices. In the early 1990’ies a framework to improve interoperation and to ensure the quality of such systems was to be defined. Such a master plan would also ensure that all further development of such systems would evolve in the same direction.

Eight potential frameworks were available; from slightly improved versions of the current architecture to completely new architectures that reflected the state of the art in the telecommunications industry. The case study (Buede & Choisser, 1992) explains how multi-attribute value theory (MAVT, a branch in the tree of multi-attribute utility theories) successfully provided the methodology for the selection of the framework, known as the Worldwide Digital Signal Systems Architecture (WWDSA).

The selection of an optimal architecture was very important, as it directly influenced develop­ment and selection of a wide variety of systems. The initial evaluation phase contained three iterations of building and refining a value tree that was agreed by all involved evaluators.

Values and weights were assigned to the attributes in the value tree; allowing comparison of the options. This process was iterated three times. In the first two iterations, the number of architectures to be considered was reduced and the remaining choices were supple­mented by new options generated. Sensitivity and ‘what-if’ analyses were conducted at several stages to ensure the robustness of the decision. After the third iteration the results produced allowed the decision makers to pick a single architecture as the final choice, the standardized WWDSA.

2. Benefits of the Approach

Clear structure allows comparison of cost/benefit:
The authors show how the methodology helps to find the balance between cost aspects and technical issues; by making implementation (cost, risk, transition) and effectiveness (performance, security, survivability, responsiveness) the top level nodes. This enabled the decision makers to directly plot effectiveness against implementation and to verify the results by sensitivity analyses.

Subdivision into components of manageable size:
The component parts in the value tree were further subdivided. This allowed a stepwise reduction of the complexity of the units under consideration, until entities of manage­able size resulted. The iteration was repeated, resulting in subcomponents with simple enough dimensions that could be assigned quantified values.

Approach enabled successful interaction of engineers and users:
A robust solution needed engineers with a very high level of technical expertise and non technical users with operational experience to cooperate in defining the optimal architecture. It is difficult to ensure that the two very different groups successfully interact. The clearly defined subcomponents of the value tree enabled the communication by clearly setting the frame for the discussion. The method also allowed focusing on the value and importance of the subcomponents, where agreement was easier that on technical or operational details.

Focus on all aspects of the architecture possible:
At the time of the project the cost-effectiveness analysis was the customary selection method, requiring the transformation of all aspects into the monetary dimension. The MAVT approach enabled the analysis and comparison of all aspects without such a transformation. Allowing the integration of all aspects also reduces the risk of overlooking important selection attributes.

Process generates well documented decisions:
An important aspect of all decisions involving money of the tax payers is the documentation of the reason, why a decision has been taken. It is a helpful strength of MAVT, as can be clearly seen in the case study, that the output generated in the selection project has the ‘ability to reconstruct the selection process in explaining the system recommendation to others’ (Buede & Choisser, 1992, p.160, 2nd paragraph).

3. Concerns About the Approach

Growing knowledge used to reduce options, never to reconsider earlier selections:
Additional insight was gained with every iteration, resulting in a stepwise reduction of the options considered. The remaining options were then supplemented by new options, generated by changing the features of the existing options. It would have made sense to quickly check if such a change could drastically improve an option that was dropped. This does not mean to loose momentum by backtracking to an earlier stage; it’s just a very brief verification that the decision to drop some options is still valid under the aspect of the new insight gained.

Exclusive use of relative scoring, never using value functions:
The approach to assign values to the attributes under consideration seems partly too simplistic. 

No attempt was made to assess the uncertainty surrounding the scores:
This statement is actually a quotation (Buede & Choisser, 1992, p.163, last paragraph). An important strength of MAVT is its support to dig deeper in areas of uncertainty. It is a pity if this chance is missed and such problems remain unresolved. This leads to less informed decisions, drastically increasing the risk involved.

Does structuring the problem using MAVT replace decision making?
The description of the steps taken in the project sometimes gives the impression, that the underlying theory was followed in a somehow mechanistic way. This presents some risks that need further consideration. The actual taking of the decision is not part of MAVT. That issue will be further discussed in the following section Questions Raised and Wrap Up.

Figure 5: Iterative analysis – compare diagrams having different underlying weights?
The figure (Buede & Choisser, 1992, p.167, lower half of page) shows three Effective­ness / Implementation charts that give the impression that they can be compared. But the underlying weights have been changed significantly from one diagram to the next. I found the diagrams misleading; looking at them I found the option finally chosen (Option 7) to be almost equivalent to option 3B that has been dropped after the first iteration. They are not equivalent as the scale changes due to a change of the underlying weights.

4. Questions Raised and Wrap Up

Are expensive, specifically fitted rooms prerequisite for the success of MAVT:
The case study describes the very sophisticated and fully IT enabled facilities that were used in the project. This raises the question if this is a prerequisite for the success of the methodology. As I have seen in my project, a simple flipchart, yellow sticky cards and a notebook with supporting software as the only tools can be sufficient.  But yes, in many cases being able to directly show the results to all participants can be beneficial. At least, if not available already; investing in a simple beamer to project the results makes sense.

Is MAVT misleading into a mechanistic decision process?
There is an understandable tendency to search for a method that takes away the pain and uncertainty from making decisions. As there is no such method, all methods are in danger of being misused as a mechanistic way to automatically’ deduct the ‘right’ decision. This seems, to some extent, to have happened in the case discussed here. This is not a flaw of MAVT; it is rather a common management error.

In spite of all the concerns mentioned, the case study shows a successful application of multi-attribute value theory in practice; the architecture that resulted was a success. It’s possible that the authors, simplifying things for the article published, did not reflect the full complexity of the decision making process in use; leading to a wrong impression of a rather mechanistic decision process. But doubts are increased by expressions like ‘The evaluation provided definite results to the decision maker in all three phases’ (Buede & Choisser, 1992, p.160, top of second column).

As a wrap up I would like to stress once again, that decision support methodologies are enablers, are ways to build the foundation for well informed decisions. They are not ‘silver bullets’ to automate decision making; that is simply not possible.

Appendix A List of References

Ackermann, F. and Belton V. (1994) Managing Corporate Knowledge Experiences with SODA and VISA. British Journal of Management, Vol. 5. p.163-176.

Belton, V. and Wright, G. (2001) Making Decisions.
Glasgow: University of Strathclyde Graduate School of Business.

Buede, D. and Choisser, R. (1992) Providing an Analytic Structure for Key System Design Choices. In Belton, V. and Wright, G. (2001) Making Decisions.
Glasgow: University of Strathclyde Graduate School of Business.

Goodwin, P. and Wright, G. (2004) Decision Analysis for Management Judgment. Third edition. Chichester: John Wiley & Sons Ltd.

Hammond, J., Keeney R. and Raiffa, H. (1998) Even Swaps: A Rational Method for Making Trade-Offs. In Belton, V. and Wright, G. (2001) Making Decisions.
Glasgow: University of Strathclyde Graduate School of Business.

Horngren, C. Datar, S. and Foster, G. (2003) Cost Accounting - A Managerial Emphasis. Eleventh edition. Upper Saddle River: Pearson Education - Prentice Hall.

Keeney R.L. (1992) Value-Focused Thinking: A path to creative decision making.
Boston: Harvard University Press.

Keeney R.L., Raiffa H. et al. (2003) Decisions with Multiple Objectives: Preferences and Value Trade-offs. New edition 2003. Cambridge: Cambridge University Press.

Simon, H. (1959) Theories of Decision-Making in Economics and Behavioural Science. American Economic Review, Jun59, Vol. 49 Issue 3, p.253-283