Quantitative Performance-based Evaluation of a Procedure for Flexible Design Concept Generation

This thesis presents an experimental methodology for objective and quantitative design procedure evaluation based on anticipated lifecycle performance of design concepts, and a procedure for flexible design concept generation. The methodology complements existing evaluation methodologies by measuring anticipated performance via efficient computer-modeling techniques. The procedure, in contrast to others, stimulates flexible design concept generation by packaging a short lecture on flexibility, and a prompting ideation mechanism.

Controlled collaborative experiments had participants suggest alternative solutions to a design problem under different treatment conditions. Experimental conditions used the procedure for flexibility, while control conditions relied on prior training in science and engineering only, and free undirected ideation. Measures included the quantity of flexible design concepts generated, anticipated economic performance improvements compared to a benchmark design, participants' subjective impressions of satisfaction with the process and results, and results quality assessments. Seventy-one designers divided among twenty-six teams performed the experiments involving a simplified real estate infrastructure design problem.

Application of the methodology demonstrated effective and efficient evaluation of the design procedure based on anticipated performance of design concepts. The lecture and prompting mechanism significantly improved anticipated performance compared to the benchmark design, by nearly thirty-six percent. The prompting mechanism significantly improved generation of valuable flexible design concepts. Lecturing improved significantly user satisfaction with the process and results, as well as results quality assessments. Even though prompting demonstrably improved anticipated performance and concept generation, it had no effect on participants' satisfaction with the process and results—unless combined with the lecture. Also, prompting did not lead participants to expect better results quality. This demonstrates the need for thorough and rigorous procedure evaluations based both on subjective user impressions and objective quantitative measurements. A preliminary analysis suggests that the proposed experimental platform can be used to study the influence of uncertainty and flexibility related words on discussion content, although more work is necessary to fully validate the approach. (Copies available exclusively from MIT Libraries, Rm. 14-0551, Cambridge, MA 02139-4307. Ph. 617-253-5668; Fax 617-253-1690.)