Documenting design research by structured multilevel analysis: supporting the diversity of the design research community of practice

Abstract

The diversity of design research studies and their associated methods and reporting style make it difficult for the design research community of practice to leverage its work into further advancing the field. We illustrate how a structured multilevel analysis of diverse studies creates a canonical model that allows for the transfer of insight between studies, enhances their comprehension, and supports improved study designs. The benefits of such an approach will increase if different stakeholders adopt such structured approaches to enrich the design research community of practice.

1. Introduction: The state of design research

Design is a complex all-pervasive endeavour involving individual professionals working alone, in teams, divisions, and other institutional structures, collaborating within and across organisational boundaries. The value of designing emerges from the work of individuals and their interactions. Design is ubiquitous in all professions, as Simon (1969) claimed and subsequently expanded to all human activities by Papanek (1984) and more recently by Subrahmanian, Reich, & Krishnan (2020). Consequently, we encounter design whether we want to study an organisation involved in designing products or services or any other organisation executing its goal-oriented mission.

The range and scale of design studies needed are concomitant to the scope of contexts in which designing takes place to achieve the desired outcomes. The different units of analysis in design studies were identified in an earlier paper on the role of empirical studies in design (Subrahmanian et al. 2004). These units of analysis reflect the units of research in organisational studies (Kozlowski & Klein 2000). However, there is no framework in either design or organisational studies for cataloging and organising these studies as a structured scaffold for the community of practice. The primary goal of this paper is to illustrate the use of the problem, social, and institutional space (PSI) design framework for creating the collective memory of the studies across units of analysis. The framework also provides a template to report results that would enhance the ability of young design researchers to get to know the scope of the field and the potential to replicate the studies. Shared organised collective memory is critical to theory development and practice in the design research community (Konda et al. 1992). Further, there is a need to understand the scope of studies of design and organisations across levels to create and compose multilevel theories of such a complex phenomenon (Mitchell 2009). Given the intertwining of organisational decision-making and the design tasks, the units of analysis paralleling each other from individual behaviour to collective behaviour needs an organising theoretical framework.

The scope of studies also influences the choice of various methods at each unit of analysis. For example, the options could be to use functional magnetic resonance imaging to study a single designer, using various social science methods to study a team, or management science models for studying an organisation. Some other studies deal with the interfaces between the different aspects (e.g., the implication of problem formulation on the design outcome that may apply to multiple disciplines), but they are uncommon (Subrahmanian & Reich 2007; Dorst 2015). This diversity ranges over time scales, product complexity, abstraction level, people involved, lifecycle studied, originality and research methodology approach; it fragments design research (Horvath 2004; Margolin 2010; McMahon 2012). This fragmentation leads to the limited relevance of many studies (McMahon 2012).

Most designers learn design from a disciplinary perspective but often work in multidisciplinary settings and teams. However, in design research, researchers are obliged to cross beyond engineering disciplines and blend psychology, mathematics, sociology, neurosciences and others to study its variety. Design research cannot escape using its multidisciplinary lens to explore its richness to paint a composite tapestry of the field. The inherent complexity of design suggests that performing a significant part of design research in controlled environments is insufficient. There is a necessity for studies through observations of designing through different lenses and contexts. These studies often go beyond a single discipline by a single researcher and consequently, are often hard to replicate. The complexity of design research causes many research studies to miss their stated objective (e.g., confirm a hypothesis) and not provide insight to design analysis or practice.

Creating and sustaining a community of practice requires creating and socialising the language and methods of the discipline that the practitioners share within the community (Lave & Wenger 1991; Bobrow & Whalen 2002). The diversity of disciplines that span the design research community of practice makes it harder to have an overall map of the scope and variety of methods and languages. Currently, the lack of well-structured methods for reporting and analysis of design studies prevents their systematic classification and interrelationships at a more detailed granularity. Such approaches to reporting are common in some disciplines of practice, such as medicine, where they specify certain templates for reporting for specific types of investigations (JAMA 2021).

The scope of this paper is to illustrate how a structured approach provides a means to deconstruct the design at a higher level of abstractions both as means of systematically relating design research in its richness and context and classifying the collective memory of the design research community. Existing literature includes structured approaches such as organisational learning, change, development, sociotechnical systems, action research and the innovative journey model. Instead, this paper will use the PSI framework (Reich & Subrahmanian 2019, 2020) to provide a novel structure to characterise specific design research case studies and inter-relate them. PSI will provide us with a way to unravel the structural inter-relationship between organisational design and product design or design research.

Justification for PSI as a framework and its comparison with other approaches is beyond the scope of this paper. A rationale for using PSI is its evolution bottom-up from case studies to accommodate the range of models in design studies. Furthermore, our collaborative work with other researchers led to the realisation of the need for a framework for a shared memory of design studies similar to the need for a shared memory of theory and practice of artifact design embedded in the context (Konda et al. 1992; Subrahmanian, Reich, & Krishnan 2020). Consequently, the present proposal follows our previous work. Other comparisons are available in the references as mentioned earlier, but moreover, we do not claim that PSI is the only framework applicable or that it is the best. Instead, we advocate for an ecology of methods, theories and frameworks (Hatchuel et al., 2018; Reich 2010). We think that it would benefit the design research community if others would explore other frameworks for structuring design research studies. Also, attempts at classifying and structuring design research have appeared in the literature (Finger & Dixon 1989a,b; Konda et al. 1992; Bayazit 2004; McMahon 2012). However, while beneficial, all of these characterisations do not provide sufficient depth in characterising the studies for our purpose.

We hypothesise that using a structured approach, which, in our demonstration is the PSI framework, to plan, report and analyse design research, we could obtain the following benefits:

• (i) improve the planning and enhancements of studies hence, improving their ability to realise their stated objective;

• (ii) improve the reporting of research leading to improved evaluation and replicability; and

• (iii) extend the concept of replicability bridging disparate design fields, hence improving the relevance and value of design research projects to others.

We contend that even though these hypotheses are relevant to all design research projects, it is more critical for research related to practice because it is more complex, is not controlled, increasing the potential of failures due to a variety of internal and external assumptions of the study. The structuring provided using the PSI framework allows for the classification and clustering of studies to accommodate the numerous theoretical bases used in these studies. The benefits from systematic encoding and classification of studies with PSI lead us to a set of recommendations for diverse groups of research stakeholders, including researchers, journal editors and the community at large.

This paper is structured as follows. Section 2 describes the multilevel instrument we propose – the PSI framework. Section 3 describes the approach we propose using PSI to build a corpus of best practices of design research studies that test the paper’s hypotheses. Section 4 provides an example of using it to analyse a research paper. Section 5 discusses the consequences of our proposal directed at different research stakeholders. Finally, Section 6 concludes the paper.

2. Multilevel modelling with PSI

PSI is a multilevel framework and design theory that has successfully modelled complex design situations (Reich & Subrahmanian 2019, 2020). It was developed bottom-up with colleagues in response to explaining diverse design or practical contexts, often conducted in large organisations. This experience informed the mapping of P to P–S to one layer P–S–I, then to two and three layers, and finally to a network of PSI matrices (Reich & Subrahmanian 2020). We added degrees of freedom to capture the additional scope and variety that we observed in design research in each such evolutionary step. While designing is pervasive, its precise manifestations in particular contexts are different, leading to variations in design studies. The method agnostic and meta-level dimensions of PSI encodes the chosen research perspective in the selected context. In this sense, we can encode positivist, critical realist, social, cultural/organisational and cognitive perspectives within the same framework. As in any community of practice, it is the development of encoding mechanisms and organisation of information that provides the space for multiple common grounds in the world of artifacts. These common grounds include vocabulary and organising principles that operate at different levels of abstraction and require collaborative practice and schematised information serving as a cognitive scaffold and infrastructure (Bowker & Star 2000; Dias, Subrahmanian, & Monarch 2003; Schmidt & Wagner 2004). Here, we demonstrate the use of PSI to plan, evaluate, improve and report research projects towards successfully achieving their stated vision and enhancing their replicability and relevance, and point to how it could strengthen the community of practice by its ability to organise the variety.

We offer PSI as we are not familiar with another multilevel framework for modelling the variety of simple to complex practical design contexts.

PSI primary model is a labelled matrix, shown in Table 1. It models a design situation or a designing entity with three layers (rows), where each addresses four fundamental questions: why and what is the challenge – denoted as the problem or product (P) space; who is involved in dealing with the challenge – social (S) space; and how is the orchestration of the S space to address the challenge – institutional (I) space. P, S and I are the triples corresponding to each layer in the matrix. The lower layer of PSI represents the regular practice, the operation (O) of the designing entity – addressing the day-to-day challenge of designing. We use the term O to designate this layer. Finally, the alignment (A) or reflection layer performs the reflective evaluation of practice modelled at the O layer to detect failures and address them. Fixing these failures constitutes the problem (P_A) in the A layer. An example of failure would be that in the O layer, a new challenge (P_O) cannot use existing development practices (I_O) or recent development tools (I_O) because the designers do not have the necessary skills (S_O). We term these failures as misalignments between the PSI spaces. An example of a simple known alignment between the P_O and I_O spaces is the mirroring hypothesis suggesting that organisational ties within an organisation reflect the dependencies in the work this organisation performs (Colfer & Baldwin 2016).

Table 1. The PSI matrix

Spaces/Questions	P (problem/product)	S (social)	I (institutional)
Layers	Why and what	Who	How
V – vision	PV	SV	IV
A – alignment/ - Reflection	PA	SA	IA
O – operation	PO	SO	IO

We use P, S and I with subscripts later to denote the indexed element in the PSI matrix.

Note that reflection on the PSI is the act of studying it to find missing information flows or relationships between the matrix entries, including problems in the relationships between the information in different cells that we call alignment. Aligning is the act of resolving mismatches in the current situation. Reflection and alignment both happen in the A layer.

For reflection to be successful, the participants need to have reflection skills (S_A) and define acceptable reflection practices (I_A). The vision layer (V) is where the vision and strategy of the organisation are determined (P_V). In this layer also, it is critical that participants, developing the vision, have the necessary skills (S_V) and follow appropriate processes and practices for developing it (I_V). Finally, embedding the vision in the organisation’s operation (O layer) is also the task of the reflection (A) layer. Altogether, the alignment of all the PSI spaces across and within the three layers is necessary for an organisation to be adaptable and sustainable (Reich & Subrahmanian 2020). This reference also provides additional details related to the PSI spaces not discussed here to keep the discussion simple.

To fill the PSI matrix, a researcher or any research stakeholder needs to ask the following questions:

At the vision V-layer:

• (i) P_V – What is the vision of this research? (e.g., to impact practice, to understand a fundamental aspect of design or to influence researchers).

• (ii) S_V – Who determines this vision? (e.g., the researchers, the funding agency or together).

• (iii) I_v – How was the vision determined? (e.g., through genuine dialogue or by the requirement of the funding agency).

• (iv) Are there potential misalignments between the spaces? (e.g., if the vision is to impact practice, it is helpful to include relevant stakeholders in formulating the vision precisely and making sure that executing the formulation process takes entirely into account the issues involved).

At the operation O-layer the questions would be:

• (i) P_O – What is the design topic that is being studied?

• (ii) S_O – Who is involved in the study? (inclusive of all affected stakeholders).

• (iii) I_O – How is the study organised? (e.g., research processes and tools, organisation of the study and its culture).

• (iv) Are there potential misalignments between the spaces? (e.g., when you experiment (P₀), do you know which statistical approach you will use (I₀), consider adding a statistician into your research team (S₀) to be able to design the experiment upfront, given available analysis methods; make sure to read ethics guidelines (I₀) to see whether they impact the kind of experiment intended (P₀).

At the reflection A-layer:

• (i) P_A – What may influence the failure or success of the research project? (e.g., missing researchers skills for the research approach required for the problem, or inappropriate research approach for the research vision).

• (ii) S_A – Who monitors the research project? (e.g., are the people involved proficient in identifying multidisciplinary issues such as misalignment between researchers’ skills and research processes?)

• (iii) I_A – How is the project being monitored? (e.g., are there periodic reviews or mechanisms to detect emerging issues).

• (iv) Are there potential misalignments between the spaces? (e.g., you might be inexperienced and require mentoring (S_A) to be familiar with modes of failure of research (P_A); in multidisciplinary projects, consider mentors with such experience specifically, and consider using project management or other practices (I_A) to guide the reflection process with some milestones (P_A)).

The questions help fill the PSI model that needs checking for misalignments between the O, A and V layers.

Levels arise naturally in two situations. First, when we wish to model an organisation at different units of analysis, for example, teams, departments and the organisation, it is clear that each will have its own PSI model. Further, these models are connected, leading to a more complex model of PSI – the PSI network model (Reich & Subrahmanian 2019). Second, levels also arise during the modelling activity when a particular misalignment issue in the PSI matrix is complex enough. We want to model it with its PSI matrix, or we wish to zoom into one aspect of the PSI matrix even if it is not a misalignment.

We demonstrate such a case in Section 4. We clarify that the term ‘level’ here is different from previous papers on PSI (e.g., Reich & Subrahmanian 2019, 2020). What we referred to before as levels are now called layers. The above is an example of the evolution of PSI concerning not just its structure but also its language to reflect better modelling clarity.

3. Method: Building best practices for design research studies

We form our position with the PSI framework as the instrument following an empirical investigation.

Figure 1 presents the approach of using PSI (or any other appropriate instrument) to create best practices for design research studies. In Step 1.1, we model study 1 with PSI by identifying the elements we interpret the model from the study report or any other available information. Then, in Step 1.2, we analyse the PSI model, PSI1, to find missing information, misalignments or other problems. Depending on the model’s status as captured by the analysis, we can conclude if the stated study objective as captured by the vision P_V has been, or could be, attained. Finally, in Step 1.3, the validation of the prediction from the PSI1 analysis is done against the study’s conclusion. Once we execute this process on sufficiently many cases, we can build a body of knowledge regarding aligned PSI models that reflect cases that realise their vision; we call it the best design research practices.

Figure 1.

An illustration of knowledge transfer from one study to another through the PSI model of the studies.

[Figure omitted. See PDF]

Consider now a new study, Study 2, that is being designed and has some similarities with Study 1. Study 2 can be modelled with PSI in Step 2.1 to yield PSI2. If this modelling is difficult for researchers, they can check the best design research practice repository to use PSI1 as guidance for their model. Subsequently, insight from the best design research practices could help the analysis of the model PSI2 in Step 2.2. For example, suppose Study 2 is similar to Study 1 but does not ensure the participation of stakeholders in the S_A space, compromising the ability to reflect on the O layer. The analysis in Step 2.2 will reveal this, predicting the state of Study 2. In the example case, the outcome would be a failure suggesting corrective measures to align it by assigning people to the role in space S_A. The insights from the analysis can inform study design and the evaluation of the study from its report. For example, suppose our best practices suggest prototyping and testing any alignment activity before any implementation in the organisation (see example in the next section). In that case, if a PSI model of research omits this step, the researchers should modify their research plan to improve its chances of attaining its goals.

We can derive different contextualised best practices from the scope of studies or even from the same study reflecting different perspectives. In using them to improve new studies, the feedback from these activities will help us improve and evolve this collection. In principle, one can use a modelling framework other than PSI if it proves helpful in modelling and analysis of cases that are subsequently validated and can be used to build best modelling practices for transfer to other cases.

4. Example using PSI to model design research studies

We used the PSI framework to analyse research reports in papers published in four major design journals: Journal of Engineering Design, Research in Engineering Design, Design Studies and Journal of Mechanical Design. Reporting all the research studies we modelled is beyond this paper’s scope; here, we provide only one illustrative example; see further examples in (Reich & Subrahmanian 2021). However, we can generate the description by asking the questions from Section 2.

Petersson & Lundberg (2018) discussed developing an ideation method (I_O as it addresses the how question) for a particular context and products (P_O – what question) for design professionals (S_O – who question) through carefully crafted action research (I_A). We can use two PSI levels to structure their study: the overall research project and the action research itself to demonstrate how a particular topic can benefit from its own PSI modelling when it is sufficiently complex. The objective or the vision of the overall project (P_V), shown in Table 2, is to develop an ideation method that would work in a particular practical context. The A layer is responsible for implementing the vision in the operation layer. This process involves identifying ideation methods for use by a multifunctional team of professionals from different organisational units (S_O). Ideation studies are mostly lab experiments (I_O) whose results would not work in a given practical context (P_O). One solution to the challenge is to set up an action research project (P_O) to develop appropriate creativity methods. We could try to model the action research in the P_O cell of this matrix, but it is complex enough to warrant its own PSI matrix model.

Table 2. PSI model of the overall research

	P (problem / product)	S (social)	I (institutional)
	What	Who	How
V	Develop usable ideation methods to work in the context of collaborative teams	Authors	Reflection on existing studies
A	Aligning the operation layer spaces: Move from lab to real context; engage practitioners in designing the methods and provide feedback; diversity improves the chances of success	Authors	Academic study, critical thinking and pragmatism
O	Issue: How to study and improve ideation in its contextPotential solution: Action research with multifunctional professional teams	Authors	Existing methods, typical (lab) research practice

Modelling the action research follows the same questions from Section 2; see Table 3. The vision of the action research could be the same as the vision of the study, but it could also be different if practitioners from the organisation define it so with the authors. The vision follows the preliminary survey of creativity methods but in addition could also use approaches such as a Delphi study or focus groups. The action research is essentially an A-layer activity (P_A) that observed prior design practice (I_O) and ideation methods (I_O) and developed a new process (I_O) with diverse stakeholders coming from different organisations with potentially different cultures and agendas (S_A). The method was prototyped and refined (I_A) to prepare it for use in practice; this is a best practice in using PSI (Reich & Subrahmanian 2020). If we did not use two-level modelling, we would have to document the details of the action research into a single cell in the research PSI model. Flattening it to a single cell would have compromised the ability to understand it.

Table 3. PSI model of the action research part

	P (problem / product)	S (social)	I (institutional)
	What	Who	How
V	Develop usable ideation methods to work in the context of collaborative teams	Authors and potentially practitioners	Result of an initial study and use of additional methods
A	Aligning operation spaces by developing a new ideation method in context, considering design challenge (PO), multifunctional teams (IO, SO), from different organisation units (SO) and existing practices (IO)	Authors, practitioners	Action research: Study, prototyping, iteration and feedback
O	Future work: field testing	Authors, practitioners and organisation units	Existing methods, new ideation method and design principles

We now have two PSI models, one representing the overall research level (Table 2) and one representing the level of the particular choice of research approach – the action research (Table 3). Since action research is one possible way to address the original challenge, we can envision a situation where the action research would fail (e.g., as in Lesca & Caron-Fasan 2008). In this case, the overall research would have had to create another solution approach to develop the creativity methods and make their specific PSI models.

The authors identify, as future work, field testing of the ideation method in real-life projects (P_O). Although, consequently, the action research did not yield a clear O-layer result (P_O in Table 3), the overall study (P_V in Table 2) was not completed as the goal to develop a method for use in practice had not been demonstrated yet. Furthermore, the incompleteness of the O-layer in the PSI model of the action research makes this PSI matrix not aligned, suggesting that the project is ongoing or incomplete, as there is no report on the project’s progress in this paper.

We note that the overall two-level PSI model of Petersson & Lundberg (2018) is almost aligned and could be used to drive forward quality research. Notwithstanding, it could benefit from setting up an explicit reflective process in the research PSI (Table 2) to monitor its progress and make sure that challenges it encounters are addressed potentially by changing the research methods. Such a complete and aligned two-level PSI model is apparent in (Schønheyder & Nordby 2018) as analysed with PSI in (Reich & Subrahmanian 2021) and in many PSI multilevel models of n-dim projects (Reich & Subrahmanian 2019).

Suppose we had the best design research practices repository in place with these subsequent studies modelled; their insight could have helped improve the reflection issue in the research model of Petersson & Lundberg. These latter models could be considered good practice patterns for designing practice-related studies, contributing to better study design and execution, and building a corpus of design research knowledge, as summarised in Figure 1.

5. Discussion

We claim that the PSI framework could provide the language for describing design research. The scope of this PSI provision goes beyond design as a professional activity to encompass design as a general human capability. It also goes beyond positivists’ or critical realists’ views of design to adopt a social constructivist perspective (Konda et al. 1992; Reich et al. 1996; Monarch et al. 1997). Nevertheless, our experience in modelling studies thus far has focused on design situations, and we, therefore, constrain ourselves to these contexts.

PSI is also a classificatory framework for design studies from numerous perspectives and theoretical bases. In contrast to other classificatory schemes in design research, such as prescriptive and descriptive, and individual versus collective, PSI provides a meta-level characterisation of design studies with the potential for accommodating other classifications of the design literature. Our experience with modelling papers with PSI, similar to the example in Section 4, illustrates how such analysis could benefit diverse stakeholders of design research studies interested in the successful design of design research projects. Using PSI as a template for summarising a paper makes it indexable by study types, methods and other dimensions, providing a meaningful and valuable resource for reflection within a project and the community. The community repository created from these classificatory structures becomes a substrate that benefits stakeholders in the design research community as shared memory (Konda et al. 1992). Such an effort can be most successful if the different stakeholders of the design research community participate together. In light of the collective need, we provide interrelated practices and requirements of three major stakeholder groups in the community.

5.1. Practice for design researchers

For design researchers as stakeholders, the issues they are addressing could be:

• (i) Improving the planning and enhancements of studies hence, improving their ability to realise their studies’ stated objectives (reflecting Hypothesis 1).

• (ii) Improving the reporting of their research projects, leading to improved evaluation and reception (reflecting Hypothesis 2).

• (iii) Improving the quality of their publications (making them more understandable, and subsequently, more reproducible; reflecting Hypothesis 2).

• (iv) Improving their research’s relevance and value to others extending its scope and referencing (reflecting Hypothesis 3).

These issues mirror the benefits stated in the hypotheses of this paper. Therefore, in response to these issues, we propose to design researchers the following practice:

• (i) Model their research design with PSI following the questions in Section 2.

• (ii) Look for potential misalignments and use them to improve their research design.

• (iii) Consider best research design practices that emerged from PSI modelling data to improve their research design.

5.2. Practices for journal editors and reviewers

For journal editors and reviewers as stakeholders, the issues they are addressing could be:

• (i) Improving the quality of papers by better reporting (reflecting Hypothesis 2).

• (ii) Improving relevance, reproducibility, scope, value, citations, and consequently, improving the potential for reproducibility and increasing journals’ prestige and impact factor (reflecting Hypotheses 2 and 3).

These issues also mirror the benefits stated in the paper’s hypotheses from another perspective.

Journal editors practice could include:

• (i) Promote researchers to provide a PSI model of their research. The model allows them to assess the research design quality and execution (similar to Study 1 in Figure 1). Further, it improves the ability of readers to understand the precise scope to replicate the study.

• (ii) Review of omissions or misalignments in PSI models to inform the study. Such gaps could appear in the report of the study limitation or future work section. Minimal deviations point to future improvements, whereas significant imperfections suggest the study has substantial limitations.

5.3. Practices for the design research community

For the design research community as stakeholders, the issues they are addressing could be:

• (i) Improve the quality of community work and impact.

• (ii) Increase the prestige of the community among peers.

These benefits extend beyond those stated in the paper’s hypotheses and benefit the design research community. For example, the practice of the research community could include:

• (i) Maintain a community repository of design studies modelled with PSI. Such a library of PSI patterns will ease the design of new studies using insight from previous study outcomes (the best modelling practices in Figure 1).

• (ii) Organise and study the repository. Different researchers may use the repository to organise their taxonomies or classify studies into different types; this corresponds to our observation that there is no single tool, theory, or classification good for everything (Reich 2010; Hatchuel et al. 2018). Subsequently, studies could derive successful research design patterns from different types or taxonomies of design studies and research to avoid common misalignments in PSI models. We briefly discussed such a pattern and its use in section 4. Further studies could include reproducing previous design research and contributing to the study and testing of PSI or other frameworks as a basis for creating best design research practices. Enhancing research practice is a critical aspect of our proposal as further developments of PSI, or other frameworks rely on such testing.

These actions could lead to additional benefits to the community:

• (i) As in theory-driven studies, a well-managed repository can help find potential gaps in PSI patterns that might lead to interesting unexplored research through interrogation.

• (ii) The repository could lead to finding limitations of PSI, hence motivating to look for alternatives to model design research. This reflective step is critical to allow further progress.

5.4. General comments about PSI modelling

We need to acknowledge several important points concerning modelling with PSI. First, every researcher may create a different PSI model of the same research hence compromising the reliability of the approach. This situation often occurs when people use complex tools involving subjective interpretations. For example, one can take two team members working on a design project and ask them to complete a house-of-quality for their project; these two models would likely be different. Rather than discarding these models, they become a departure point for dialogue to understand the project better and arrive at some consensus. The same practice should be exercised with different PSI models. Further, we intend to offer training material for using PSI and continue to improve its usability for the different stakeholders.

Also, some PSI models of research studies may be incomplete because the information is not available in their reports for various reasons. Authors can explicitly state the limitations using PSI models or identifying and explicating them through the review process. This exercise already creates a better context for future studies and their reviews. From our experience modelling studies, even if some aspects are missing, the remaining still provide rich ground for analysis.

5.5. Modelling with other frameworks

Suppose we wanted to put this paper into perspective, using PSI for modelling and in line with the principle of reflexive practice (PRP; Reich 2017). The result would be as shown in Figure 2. All the elements are available, and in fact, the alignment is by construction in the figure if design research stakeholders join us in S_A. It will be better aligned if other researchers interested in developing design research modelling frameworks join us in S_A, but it is not mandatory. The reflective step, carried out at the A layer, challenges the PSI model relentlessly to create additional frameworks or improve PSI or other candidate frameworks, enhancing our ability to model studies. Consequently, while we propose PSI framework as a means, our objective is to trigger the community to move towards creating a rich design research practices repository with whatever means available. There is a good chance that such studies will also lead to a better understanding of design because design research projects are just instances of design; improving our work (design of research projects and their execution) will enhance our understanding of design in general.

Figure 2.

Modelling the proposal in this paper with PSI.

[Figure omitted. See PDF]

6. Concluding remarks

In this paper, we articulated the need to capture the spectrum of design research studies across different levels of units of study and their inter-relationships to support the design research community of practice. We illustrated that the PSI framework provides an organising structure for characterising and inter-relating design research studies spanning the complexity of multilevel units of study. We also showed how we could use the PSI framework for reflection within, for design education and reflection across design studies.

Different stakeholders of design research studies would obtain significant benefits by designing their research and modelling using structures provided by the PSI framework. They will help improve the usefulness and inter-relatedness of studies, understand and evaluate studies better and consequently, lead to better quality, more reproducible studies relevant to a broad range of studies. Of course, these claims about the benefits of using PSI are testable; we anticipate that besides us, some other members of the design research community will study the usefulness and improve those frameworks.

The benefits would increase as more stakeholders use the PSI, and the best design studies practices would be more extensive and accurate. We do not claim that the PSI framework is the only or a perfect way to structure design studies, nor that it is the best. We anticipate that with its use and its visible limitations and benefits, other frameworks or changes to PSI may emerge to advance our practice further. We contend that creating organising frameworks from different perspectives of design research to understand the scope and results of the studies is critical to bringing the community to make collective progress in the field. We contend that it would be valuable to the community to organise and structure design research reports and research plans with diverse models to improve the accessibility and quality of our research and, consequently, replicability and relevance.

Acknowledgements

We wish to thank the five reviewers for their constructive remarks that helped improve the paper’s ideas and exposition.