A Process View to Evaluate and Understand Preference Elicitation

This paper focuses on multi-attribute design problems in which several quantitative and/or qualitative attributes are simultaneously involved. Preference elicitation methods dealing with decision-makers’ preferences are often employed to solve such problems. The present paper proposes a conceptual framework that considers preference elicitation as a process shaped by the interaction of different factors including the researcher, methodology, questionnaire and respondent in a given context, at the same time. The aim of the framework is to assist in choosing a decision support tool among various multiattribute decision-making methods, as well as in evaluating and understanding preference elicitation results. To illustrate and exemplify the use of the framework, it is applied to a real-world case of end user preferences of mobile service platforms. We make use of two methods that are suitable for such a setting: analytic hierarchy process and conjoint analysis. Albeit with considerable discrepancies, the results show that both methods provide insightful information on the principle. Nonetheless, to a large degree, these methods perform differently in the case of non-complex tasks and unknowledgeable respondents, which the process view aids in evaluating and understanding. This paper focuses on multi-attribute design problems in which several quantitative and/or qualitative attributes are simultaneously involved. Preference elicitation methods dealing with decision-makers’ preferences are often employed to solve such problems. The present paper proposes a conceptual framework that considers preference elicitation as a process shaped by the interaction of different factors including the researcher, methodology, questionnaire and respondent in a given context, at the same time. The aim of the framework is to assist in choosing a decision support tool among various multiattribute decision-making methods, as well as in evaluating and understanding preference elicitation results. To illustrate and exemplify the use of the framework, it is applied to a real-world case of end user preferences of mobile service platforms. We make use of two methods that are suitable for such a setting: analytic hierarchy process and conjoint analysis. Albeit with considerable discrepancies, the results show that both methods provide insightful information on the principle. Nonetheless, to a large degree, these methods perform differently in the case of non-complex tasks and unknowledgeable respondents, which the process view aids in evaluating and understanding.