Content area
For any organization, designing a new product or planning the features of a new service is a complex decision-making process which depends on understanding its users' preferences and the experiences they have with prototypes of the new product or service. Design of experiments (DOE) offers a framework to aid in the modeling and collection of data to learn user preferences and expe-riences to facilitate this decision-making process. Questions such as "which of our newly proposed products is preferred most by our customers" and "which version of our new service results in the highest monthly revenue" can be readily addressed by carefully designing and analyzing an appropri-ate experiment. This dissertation focuses on the design of experiments for learning user preference and experience, and consists of four projects related to experimental design where product or ser-vice users are involved: (1) Bayesian sequential preference elicitation; (2) Batch sequential designs in Bayesian preference elicitation; (3) Approximate dynamic programming methods in Bayesian pref-erence elicitation and (4) Collaborative design of controlled experiments in the presence of subject covariates.
The first three chapters of this dissertation are research projects which were funded by VIPR-GS, a research group at Clemson University spanning multiple departments which is interested in the virtual prototyping of ground vehicle systems. Chapters 2, 3, and 4 are concerned with the development and investigation of preference elicitation models with application to tradespace exploration for vehicle concept design. In Chapter 2, we introduce the Bayesian sequential preference elicitation problem and propose computational enhancements to the framework proposed by Sauré and Vielma (2019). Namely, we provide simplified versions of their Bayesian updating equations and propose the use of a linear approximation to their query selection criteria, known as D-errorг.which can be easily implemented in commercial optimization solvers. In Chapter 3, we generalize the framework of Sauré and Vielma (2019) to the batch sequential setting, where multiple queries are simultaneously selected and then presented to the participant in a multistage questionnaire. We identify a new summary statistic for describing the relationship between queries in a batch design, called query covariance, and propose surrogate-based mixed-integer programming (MIP) formulations for constructing D-efficient batch designs where the variables in the surrogate model are functions of query covariance and two other summary statisties described in Sauré and Vielma (2019). In Chapter 4, we propose and investigate various approximate dynamic programming methods for query selection, among which are several non-greedy methods which to our knowledge have not been applied in the Bayesian preference elicitation framework. Our findings suggest that there is only a marginal improvement in D-efficiency when using expensive non-greedy methods, suggesting that greedy methods are sufficient for query selection. Lastly, in Chapter 5 we investigate the problem of experimental design in the case where there are multiple, separate controlled experiments, subject covariate information is available, and subjects participate in each of the experiments. Taking into account both subject covariate information and the dependency between responses coming from the same subject, we propose algorithms for solving the corresponding D-optimality problem, finding that our algorithms are able to construct designs which provide more precise estimates of treatment effects.