1. Introduction
Since the 1990s, digital technology has developed rapidly and penetrated into various fields, and the application and service based on computer systems have proliferated. With the continuous expansion of the Internet to ordinary family life, a powerful and highly smart building system is gradually formed, making it possible to achieve a safer, more convenient, energy-saving, and intelligent home life. Smart building is the result of the penetration of information technology and control technology into traditional buildings [1,2,3]. From the first summit in 2000 to the 19th China International Building Intelligence Summit, although the road to intelligent building automation in China has undergone many twists and turns, this process has been unstoppable. The development of science and technology has made people unswervingly pursue a higher quality of life. Smart buildings have attracted more and more attention as representatives of high-quality information life. For example, Fuji Xerox Parker’s main research and development areas are smart workplaces (spaces) and flexible working methods. The main achievements include a wearable remote video communication system and a desktop file management and remote communication system, aiming to make information transmission more efficient, more accurate, more interactive, and social. The Computer and Artificial Intelligence Laboratory of the Massachusetts Institute of Technology is one of the world-renowned computational science and artificial intelligence laboratories. It is also actively studying the application of artificial intelligence technology in urban buildings, looking for innovative ways to make systems and equipment safer, more convenient, and more effective to benefit humankind.
Buildings are for the realization of people’s hope and demand. As an information exchange platform between a smart system and users, the intelligent design of intelligent building systems plays an important role. User-centric building smart design can achieve a sustainable life, and smart technology integration based on smart design can support sustainable development and improve user satisfaction, happiness, and overall quality of life [4]. In intelligent design, researchers focus on the “people-oriented” approach, designed to bring users the ultimate interactive experience. The human–computer interaction design has attracted more and more attention. Interaction design is to create and support human behavior design through physical, virtual, or service products, focusing on defining the interface related to the behavior of human creation [5]. Human–computer interaction is to constantly enrich the user experience of the rich, to achieve the purpose of optimizing the interaction experience, and to build a more practical, easy-to-use, and humanized smart building system. Therefore, the interaction experience plays an important role in the smart design stage of the smart building system. Based on the rapid development of smart buildings, many smart building technologies have emerged one after another. In addition to some physical designs, such as sensors and actuators, there is a growing demand for interactive experience design about emotional perception. For example, Europe, the United States, and Japan have adopted a series of interactive experience-related technologies for population aging, and have been well promoted [6].
In the design of smart building systems, different smart technologies should adopt different interaction modes for different users, and smart designers should pay attention to different user characteristics, that is, the principle of a “people-oriented” approach. According to the survey of the beneficiaries of smart buildings, and according to the age division principle, users can be divided into three stages, mainly including children, adults, and the elderly. In the current social background of China, the demand for human–computer interaction smart design of smart buildings for these three types of users is also increasing. First of all, in terms of children’s interactive experience needs, and in the process of children’s growth, the family is the first stop and plays an important role. Parents’ educational concept and family environment will directly affect children’s development. Therefore, the market demand of human–computer interactive smart building teaching systems related to children’s early education and preschool education is increasing gradually. Secondly, in the aspect of adults’ interactive experience demand, with the continuous improvement of Internet information, there are home office, remote conference, and other smart building systems, and users put forward higher requirements for the life style of interactive experience in smart buildings. Finally, in terms of the interactive experience needs of the elderly, the trend is of the aging population and the social status quo of rapid development of science and technology [7]. The physical and mental health of the elderly is valued, and the demand for intelligent assistance in the daily home life of the elderly continues to expand. On the other hand, it makes a large number of intelligent building auxiliary equipment emerge. To a certain extent, home auxiliary equipment can improve the health and life ability of the elderly, so the interactive smart building equipment for the elderly is an opportunity for the development of smart home systems. User-centered intelligent design can better promote the sustainable development of smart buildings. Sustainability here no longer refers to resource conservation and the green environment, but to bringing users a sustainable life.
The user-centered interactive experience design principles play an important role in the intelligent design of smart buildings. However, most review articles for smart buildings mainly focus on the development of key technologies or intelligent technologies. Few studies start from the perspective of interactive experience and use the “five senses” interaction type to prove the importance of the “people-oriented” approach in smart design. As the carrier of smart buildings and users, the senses will affect the user experience of smart building design. The five senses are interrelated and inseparable, and the mutual influence of each perception element constitutes the entire experience of architectural perception. Starting from the perspective of the “five senses” can better reflect the “people-oriented” principle, and provide more possibilities for smart building design, thereby improving the comfort, satisfaction, and sustainability of smart buildings. Therefore, based on the interactive experience principle of smart design in the smart building system, this article classifies and summarizes intelligent design from the “five senses” interaction, including the visual interaction, the voice interaction, the tactile interaction, the cognitive interaction, and the emotional interaction. We analyze the application of smart design in buildings and discuss how to embody the principles of user-centered interactive experience design in the process of smart design. This article provides a comprehensive and systematic literature review, clarifies the importance of the “people-oriented” principle in the smart design of buildings, and summarizes how to improve the sustainability of intelligent buildings from the perspective of a “people-oriented” approach. This paper proposes future research suggestions and directions and promotes the sustainable development of the smart building. 2. Method
This research used a content analysis method for review analysis. The content analysis method can be used to systematically and effectively make inferences based on the collected data, so as to conduct qualitative and quantitative analysis of the research status, which is beneficial to describe the overall application of the “people-oriented” interactive experience principle in smart design [8]. The qualitative analysis of the content analysis method can reflect the main characteristics of the literature; the quantitative analysis of the content analysis can objectively summarize the research content, and reveal and summarize the potential information of the existing literature in terms of time, region, and so on. Therefore, the content analysis method can more comprehensively analyze the application of the “people-oriented” interactive experience in smart design, and ensure that the research results are reliable and effective. During the review process, the work design of the review study was particularly borrowed from the framework of Kim [9].
2.1. Step 1: Identifying the Research Question
The daily smart building system is mainly composed of people, control systems, and electronic products, as shown in Figure 1. Among them, the control system establishes an intelligent connection between people and electronic products for mutual information transmission and feedback. On the one hand, the performance of the control system determines the intelligence of the building system. On the other hand, people belong to the main service objects of the smart building system, and the performance of the control system also determines the satisfaction degree of the people in the smart building service. The performance of the control system mainly depends on intelligent technology. The “people-oriented” interactive experience design principle based on cognitive psychology, design psychology, and other related theories has a significant impact on the design of smart building technology, fully plays the role of interactive experience, builds a bridge between information transmission and the intelligent environment, and achieves a more complete human–computer interaction mode.
Therefore, this article first consults the existing reviews of the smart design in buildings, clarifies which aspects are currently involved in those reviews, or does not cover too much, confirms the frequency or degree of attention of the interactive experience in those reviews, and ensures the research significance of the interactive experience in this article on the review of smart design in buildings. This article starts with the interactive experience principles of smart design in smart buildings, and follows the “five senses” interactions involved in interactive experiences, including visual interaction, voice interaction, tactile interaction, cognitive interaction, and emotional interaction. It classifies and summarizes smart design, analyzes the application of intelligent design, especially the interaction characteristics of the intelligent environment and the users, and explores how to embody the “people-oriented” interactive experience design principles when designing a smart environment. 2.2. Step 2: Collecting Studies This article used the search terms related to “smart design” and “interactive experience” to conduct a literature search on the five major databases of ISI Web of Science, Science Direct, Ei Village, ACM Library, Scopus, and IEEE Explore. These search terms are universally accepted from journal to journal and from continent to continent, and there is no regional and journal bias. Wearable mobile devices and smart operating platforms are the mainstays in the smart building; for the “five senses” interactive experience, we searched for visual interaction, voice interaction, tactile interaction, cognitive interaction, and emotional interaction. Finally, all the collected documents were reviewed, and preliminary screening was conducted. 2.3. Step 3: Screening Literature In order to ensure that the collected documents were consistent with the definition of the scope set in this article, all the collected documents have undergone strict screening and review, and the inclusion and exclusion criteria have been formulated as follows: The inclusion criteria were (1) literature that clearly introduces smart building equipment or technology; (2) the article aims to demonstrate the application of “people-oriented” interactive experience design principles in smart buildings; (3) the article is published after 2010; The exclusion criteria were (1) survey-type research using a scale or questionnaire; (2) comprehensive research on smart building panel design based on interactive experience; (3) exploring the developmental potential of interactive experience in smart building devices or usability studies. After determining the inclusion and exclusion criteria, the title was first screened, followed by the abstract, and finally the full-text review. 2.4. Step 4: Analyzing Literature We conducted a qualitative and quantitative analysis of the filtered literature. Among them, qualitative analysis was used to identify the year and region of the literature, analyze the interactive characteristics of the intelligent environment formed by intelligent design and users, and explore how to embody the “people-oriented” approach in the process of intelligent design. Quantitative analysis was used to determine the number of years and geographical distribution of the literature. Through the above analysis, we can ensure the development of a trend of application of interactive experience in intelligent design, and propose better future development directions and suggestions.
In addition, the Figure 2 shows more details of the review process of this article.
3. A Survey of Research on Smart Design Based on Interactive Experience 3.1. Trends
3.1.1. Publications in Years
Figure 3 shows the trend of the final selected literature over time. The average number of published studies in 2010–2020 is about 7.2, among which the most published articles in 2018 are 18, far exceeding the average. Figure 3 also shows us clearly that the number of articles published before 2014 is far less than six, but after 2015, the number has increased significantly, which means that people pay more and more attention to the importance of the “people-oriented” interactive experience design principle in the application of smart building systems.
3.1.2. Applications in Geographical Areas
Table 1 shows the literature distribution of “people-oriented” interactive experience design principles applied in smart building systems in various countries and regions. Because many studies involve authors from many countries, the principle of our classification was based on the geographical location of research data or the research background in each research. This method can better reflect the geographical distribution of interactive experience in the application of smart design in building systems. According to the data in the table, there are 13 articles in China, accounting for 19.11%, followed by 8 in the United States, 4 in France, 3 in India, 3 in Australia, 3 in Spain, and 3 in Romania.
3.1.3. Application Fields
Table 2 shows the frequency of five different interaction modes in the literature, among which voice interaction is the highest (30.88%), followed by visual interaction and emotional interaction (20.59%), cognitive interaction (19.12%), and somatosensory interaction (14.70%). It was clearly found that the “people-oriented” interaction experience design principle has different degrees of application in different interaction modes, but these five interaction modes are indispensable in the smart building systems.
3.2. Review Results
3.2.1. The Visual Interaction
In the building design system based on visual interaction, (1) when constructing the intelligent environment, in order to realize the technical development on different platforms and different controls, the hardware of the intelligent building system will adopt different methods. For example, the 3D image synthesis method is introduced to apply the 3D display to the control interface of the intelligent building system, so as to enhance the authenticity, user friendliness, and visual comfort of the interface and enhance user interaction experience [10,11]. (2) In order to ensure the user-centered principle, users participate in building applications that can be configured and managed on the iOS platform, and evaluate their availability [12]. (3) Based on the characteristics of user vision, measures such as “staring” should be taken to attract users’ visual attention, so as to achieve efficient communication between human and machine and improve user satisfaction [13,14]. (4) Based on the feedback mechanism of users, the conscious optimization model of the visual interaction system is established to support better human–computer interaction [15,16]. (5) Give full consideration to the needs of users, especially for users with special needs, such as the disabled, through remote consultation of the visual system; intelligent nursing data can be provided, and the technology that can control the home building system through eye movement is proposed to give full play to the visual interaction of users [17,18,19]. In the design stage of the intelligent building system, let users really participate in the work of intelligent technology design, collect user experience and user data, and establish a user information database, which can not only improve the visual level of the intelligent building system, but also improves users’ comfort and satisfaction [20,21,22,23,24].
3.2.2. The Voice Interaction
Voice interaction enables the user to control the intelligent building system by voice so as to realize the automation and robustness of the intelligent building system [25,26,27]. In the process of intelligent design of buildings, researchers often take suggestions of user participation and use deep learning and other techniques to continuously optimize the sensitivity of speech recognition [28,29]. (1) Researchers pay attention to the voice characteristics of users, develop an inclusive multi-functional intelligent building voice recognition system, improve the accuracy of user voice recognition, and bring more functions and experiences to users [30,31,32]. (2) The elderly or users with speech defects may encounter a series of problems in the home environment. The special speech recognition system developed can help them solve speech disorders and ensure their safety and health in life [33,34,35,36,37]. (3) In order to satisfy the users in the long distance voice interaction under special environment requirements, the designers of the analysis of the intelligent building system in the process of long distance voice interaction in which may exist problems, put forward the speech recognition system for long distances, and broke the voice interaction possible distance limit, to improve the user experience in the intelligence environment [38,39,40,41,42,43,44].
3.2.3. The Tactile Interaction
Body sense interaction is mainly to use smart building system equipment to intelligently identify the user’s actions or make physical contact with the user to realize human–computer interaction [45,46]. When gesture recognition becomes the main interaction mode in smart building systems, designers pay more attention to the optimization of gesture recognition technology. By examining the different needs of users in different stages, gesture recognition technology changes from ordinary static recognition to dynamic recognition, from a single gesture to multiple semantics, constantly enriching the participation and satisfaction of users in the smart building experience [47,48,49]. At the same time, in view of the needs of special users, especially for blind or visually impaired users, from the user’s perspective, the tactile channel is used instead of the perception channel to avoid the frustration brought by visual defects. The user-centered development of such a guidance model can effectively realize human–computer interaction and improve the user’s satisfaction [50,51].
3.2.4. The Cognitive Interaction
Based on the different cognitive characteristics of users, in order to give full play to the advantages of cognitive interaction in smart building systems, a series of systems and devices are developed to meet the needs of users [52,53,54,55]. The interaction interface between the user and the smart building has a high impact on the user’s experience. Analyze the cognitive characteristics of different types of users, and design the interaction interface from the perspective of information cognition, so as to reduce the cognitive friction generated by the user in the process of using a smart home, and then improve the user’s use experience [56,57,58,59]. For the inevitable cognitive obstacles, especially with the coming of the aging age, there are many problems related to the decline of cognitive function. In order to supplement the cognitive function of the elderly users, the smart building user interface simulation model is established by the elderly users. Through the participation and feedback of the elderly users, the auxiliary system is proposed to realize the harmonious communication of the elderly in the intelligent environment and improve the performance of human–computer interaction in the intelligent environment [60,61,62,63,64,65].
3.2.5. Emotional Interaction
With the continuous improvement of users’ needs, higher requirements are put forward for the building environment to obtain psychological and emotional satisfaction. In recent years, more and more emotional management systems based on emotional interaction have emerged in the smart building environment [66,67,68]. By analyzing the different emotional needs of different users, including emotional acquisition, emotional recognition, and all stages of emotional interaction, a highly interactive and experiential personalized and emotional intelligent device has been established [69,70,71,72,73]. The focus on the elderly user group is still the focus of emotional interaction. Design various emotional perception frameworks and devices to trigger emotional communication between the elderly and the smart building environment, and give full play to the importance of a “people-centered” approach [74,75,76].
3.2.6. Interactive Combination
The human–computer interaction in smart building is not only the use of a single interaction, but also the combination of different interactions in recent years. Through various combination modes, it can more comprehensively meet the user’s requirements for interaction [77,78,79,80]. When integrating multiple interaction modes, users play a more important role in the smart design of the building system, especially for users with physical defects, and according to the real needs of users, through the combination mode of voice interaction and touchable sense interaction, the operation of the home system can be realized to ensure that such users complete their daily activities in the smart building system [1,81,82,83]. Many kinds of interactive combinations are more and more popular, which is an opportunity for the future development of smart building; of course, the difficulty of technology is also a challenge.
4. Conclusions
This paper starts with the principle of interactive experience of smart design in the building system according to the “five senses” interaction involved in the interaction experience, including visual interaction, voice interaction, tactile interaction, cognitive interaction, and emotional interaction. This paper classifies and summarizes smart design, and analyzes the application of smart design, especially the interactive characteristics between the intelligent environment and users. It also explores how to embody the principle of “people-oriented” interactive experience design when designing an intelligent environment, and finds the key position of users in the smart design of the building, as shown in Figure 4; that is, the importance of the “people-oriented” principle. From the perspective of a “people-oriented” approach, this paper summarizes how to improve the acceptance of intelligent buildings and promote the development of a sustainable life.
First, the principle of “people-oriented” interactive experience design is to ensure the security, effectiveness, and sustainability of the smart building system, including intelligent communication in the building environment, which fundamentally changes the traditional smart building interaction mode. By reviewing the smart design based on the principle of “people-oriented” interactive experience design, focusing on changing the problems that need attention of the principles of “people-oriented” interactive experience design and the process of “people-oriented” interactive experience application in the process of specific technology use, we can find that: (1) In the research of principles of “people-oriented” interactive experience, with the continuous development of the times, more and more attention has been paid. Especially after 2014, most of the research is in the context of China, and the development trend of the smart building system is to maximize the needs of all users. For example, when we measure a smart building system, the different effects of interaction may make different users feel differently about the same product. Therefore, we should pay attention to the key position of “users” in smart building design. User-centered product design can provide specific uses for specific users in specific use environments, ensure the effectiveness, user satisfaction, and other performance of smart buildings, and can adapt to more use environments and achieve sustainable living. When using the user-centered concept of the “people-oriented” approach to participate in interactive design, we need to pay attention to the following issues. The first are requirements. From functional requirements to emotional needs, methods such as user surveys are adopted to collect user data accurately to clarify the real needs of users [11,34,47]. The second is user participation. In each test process before product technology shaping, users should participate in the experience, express their true feelings, and put forward opinions. Then, special technical designers should retain operational opinions, constantly optimize product technology, and improve the satisfaction of the interactive experience of smart buildings [9,45,64]. (2) In the analysis of the existing research, it was found that not only single interactive applications, but also the combination of two or several kinds of interaction existed, aiming at providing users with a comfortable and satisfactory home experience [73,74,75,76]. In recent years, the frequency of the smart building system model or technology development of this combination model has gradually increased. It is obvious that technology is developing towards a more complex, intelligent, and humanized direction.
Second, in order to ensure the sustainable life brought by smart buildings, the principle of the “people-oriented” interactive experience of smart design in building applications will follow the following principles. The first is to see the demand clearly [24,39]. Before the development of smart home technology, the needs of users, including physiological needs (functional needs) and psychological needs (emotional needs), will be fully considered. In terms of physiological needs, the smart design of the building system has improved people’s quality of life to a certain extent, bringing convenience and comfort to people’s lives. In terms of psychological needs, the smart design of the building system enables information exchange between human and computer, and improves psychological services to users. Therefore, it is extremely important to satisfy users’ needs in the application of interactive experience. The second is rational planning [30,34]. In the application of interactive experience technology, we should follow the principle of the reasonable use of time, space, and family members, so that the whole family can benefit from the smart design of the building system. The third is to build a model [20,72]. From the perspective of cognitive psychology, let users participate in the early design stage of smart buildings, put forward reasonable requirements, and provide a basic model for the interactive experience of the smart building systems.
Thirdly, in the smart building system, the user’s real reaction determines the degree of acceptance of the smart building system. The user’s feelings mainly depend on the following characteristics of product technology—stability performance, interaction mode, emotional expression, and so on [78,83]. Therefore, the smart building system should have a certain stability, and no matter how much user participation, all kinds of smart designs can perform their duties, adapt to more use environments, and ensure the stable performance of the smart environment. It should have a good human–computer interaction mode, and create a smart building environment that can be exchanged, so that the building is no longer a cold space; it should have human emotions, feel the emotional changes of users at all times, and adjust the home environment according to the changes, alleviate the mood of users, and be users’ close friends.
5. Future Research Suggestions and Directions The interaction design principle based on interaction experience is not only on the smart design, but also on the user’s inner needs and motivations, especially how to realize the humanization of smart buildings and the effective communication between people, and to promote the development of a sustainable life; that is, to adhere to the principle of “people-oriented” smart design. A “people-oriented” approach is the development direction of various research content of smart buildings in the future. Only the interaction at the technical level and the spontaneous interaction between human and computer can create a more harmonious and friendly human–computer interaction environment for users, and the smart building system will develop towards a more intelligent, human-oriented, and personalized direction, providing more comfortable, safe, and effective smart services for home needs and promoting sustainable living. In addition, with the emergence of 5G technology, people’s use of the Internet and the Internet of Things will reach a new level—the ultimate experience of high efficiency, rapidity, and economy. The development of 5G smart buildings will receive more and more attention.
Enlarge this image.Figure 4. Application of interactive experience in smart design of the building system.
| Country or Region | Number of Publications | Percentage of Publication (%) |
|---|---|---|
| China | 15 | 19.40 |
| United States | 9 | 10.45 |
| France | 4 | 5.97 |
| India | 3 | 4.48 |
| Australia | 3 | 4.48 |
| Spain | 3 | 4.48 |
| Romania | 3 | 4.48 |
| Italy | 2 | 2.97 |
| Japan | 2 | 2.97 |
| Netherlands | 2 | 2.97 |
| Greece | 2 | 2.97 |
| Portugal | 2 | 2.97 |
| Taiwan | 2 | 2.97 |
| Canada | 2 | 2.97 |
| Sweden, Finland, Singapore, Arabia, Earl, Malaysia, Mexico, Algeria, Israel, Pakistan, South Korea, Norway, Switzerland, Germany, Brazil, Serbia, Sri Lanka | 2 | 1.49 |
| Total | 72 | 100 |
| Interaction Type | Quantity | Percentage (%) |
|---|---|---|
| visual interaction | 15 | 20.59 |
| voice interaction | 22 | 30.88 |
| tactile interaction | 10 | 14.70 |
| cognitive interaction | 14 | 19.12 |
| emotional interaction | 15 | 20.59 |
Author Contributions
All authors contributed equally to this work. All authors wrote, reviewed, and commented on the manuscript. All authors have read and agreed to the published version of the manuscript.
Funding
This work was supported by the National Natural Science Foundation of China (Nos. 71971100,71671078,71501084,71974081,71771127); Social Science Fund of Jiangsu Province (19GLB005,19GLB018); The Key Project of Philosophy and Social Science Research in Colleges and Universities in Jiangsu Province (2018SJZDI052); sponsored by Qing Lan Project of Jiangsu Province; Youth Backbone Teacher Training Project of Jiangsu University.
Acknowledgments
The authors would like to acknowledge the professionals who collaborated on this study. We would also like to thank all the reviewers who provided suggestions to improve this paper.
Conflicts of Interest
The authors declare that there is no conflict of interest regarding the publication of this paper.
1. Wang, Z.; Wei, S.; Shi, L.; Liu, Z. The Analysis and Implementation of Smart Home Control System. In Proceedings of the 2009 International Conference on Information Management and Engineering, Kuala Lumpur, Malaysia, 3-5 April 2009; pp. 546-549.
2. Zhu, J.; Wang, X.; Wang, P.; Wu, Z.; Kim, M.J. Integration of BIM and GIS: Geometry from IFC to shapefile using open-source technology. Autom. Constr. 2019, 102, 105-119.
3. Zhu, J.; Wu, P.; Chen, M.; Kim, M.J.; Wang, X.; Fang, T. Automatically Processing IFC Clipping Representation for BIM and GIS Integration at the Process Level. Appl. Sci. 2020, 10, 2009.
4. Hu, X.; Chong, H.-Y.; Wang, X. Sustainability perceptions of off-site manufacturing stakeholders in Australia. J. Clean. Prod. 2019, 227, 346-354.
5. White, C. Health Care Spending Growth: How Different is the United States from the rest of the OECD? Health Aff. 2007, 26, 154-161.
6. Carnemolla, P. Ageing in place and the internet of things-How smart home technologies, the built environment and caregiving intersect. Vis. Eng. 2018, 6, 7.
7. Shelley, M.; Krippendorff, K. Content Analysis: An Introduction to its Methodology. J. Am. Stat. Assoc. 1984, 79, 240.
8. Zhang, Y.; Ling, Z. The application of autostereoscopic display in smart home system based on mobile devices. In Proceedings of the Sixth International Conference on Graphic and Image Processing (ICGIP 2014), Beijing, China, 18 March 2015; Volume 9443.
9. Kim, B.Y.; Lee, J. Smart Devices for Older Adults Managing Chronic Disease: A Scoping Review. JMIR mHealth uHealth 2017, 5, e69.
10. Fogli, D.; Peroni, M.; Stefini, C. ImAtHome: Making trigger-action programming easy and fun. J. Vis. Lang. Comput. 2017, 42, 60-75.
11. Natephra, W.; Motamedi, A.; Fukuda, T.; Yabuki, N. Integrating building information modeling and virtual reality development engines for building indoor lighting design. Vis. Eng. 2017, 5.
12. Rook, K.; Witt, B.; Bailey, R.; Geigel, J.; Hu, P.; Kothari, A. A Study of User Intent in Immersive Smart Spaces. In Proceedings of the 2019 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops), Kyoto, Japan, 11-15 March 2019; Volume 2019, pp. 227-232.
13. Ai, H.; Li, T. A smart home system based on embedded technology and face recognition technology. Intell. Autom. Soft Comput. 2016, 23, 405-418.
14. Mejía, D.; Kubis, T.; Klimeck, G. NemoViz: A visual interactive system for atomistic simulations design. Vis. Eng. 2018, 6, 6.
15. Wang, K.-J.; Zheng, C.Y.; Mao, Z.-H. Human-Centered, Ergonomic Wearable Device with Computer Vision Augmented Intelligence for VR Multimodal Human-Smart Home Object Interaction. In Proceedings of the 2019 14th ACM/IEEE International Conference on Human-Robot Interaction (HRI), Daegu, Korea, 11-14 March 2019; pp. 767-768.
16. Jang, J.; Bednarz, T. HoloSensor for smart home, health entertainment. In ACM SIGGRAPH 2018 Appy Hour; Association for Computing Machinery: New York, NY, USA, 2018.
17. Caporuscio, M.; Weyns, D.; Andersson, J.; Axelsson, C.; Petersson, G. Iot-enabled physical telerehabilitation platform. In Proceedings of the 2017 IEEE International Conference on Software Architecture Workshops (ICSAW), Gothenburg, Sweden, 5-7 April 2017; pp. 112-119.
18. Zhang, R.; He, S.; Yang, X.; Wang, X.; Li, K.; Huang, Q.; Yu, Z.; Zhang, X.; Tang, D.; Li, Y.; et al. An EOG-Based Human-Machine Interface to Control a Smart Home Environment for Patients with Severe Spinal Cord Injuries. IEEE Trans. Biomed. Eng. 2019, 66, 89-100.
19. Wang, K.-J.; Tung, H.-W.; Huang, Z.; Thakur, P.; Mao, Z.-H.; You, M.-X. EXGbuds: Universal wearable assistive device for disabled people to interact with the environment seamlessly. In Companion of the 2018 ACM, Proceedings of the IEEE International Conference on Human-Robot Interaction, Chicago, IL, USA, 5-8 March 2018; ACM: New York, NY, USA, 2018; pp. 369-370.
20. Song, P.; Zhao, L. Research on Visible Light Communication Control System Based on Steady-State Visual Evoked Potential. In Proceedings of the 2015 7th International Conference on Intelligent Human-Machine Systems and Cybernetics, Hangzhou, China, 26-27 August 2015; Volume 2, pp. 284-287.
21. Majaranta, P.; Laitinen, J.; Kangas, J.; Isokoski, P. Inducing gaze gestures by static illustrations. In Proceedings of the 11th ACM Symposium on Eye Tracking Research and Applications, Denver, CO, USA, 25-28 June 2019.
22. Jacob, R.; Stellmach, S. What you look at is what you get. Interactions 2016, 23, 62-65.
23. Shin, J.-H.; Lee, S.-J. An Analysis on the Degree of 3D Sense Following Distance and Location through 3D Depth Level Change. Int. J. Smart Home 2012, 6, 6.
24. Hussein, D. A user preference modelling method for the assessment of visual complexity in building façade. Smart Sustain. Built Environ. 2020.
25. Liu, S.; Helal, S.; Lee, J.W. Activity Playback Modeling for Smart Home Simulation. In Inclusive Smart Cities and e-Health. ICOST 2015. Lecture Notes in Computer Science; Springer: Cham, Switzerland, 2015; Volume 9102, pp. 92-102.
26. Garcia-Rodriguez, J.; Chamizo, J.M.G. Surveillance and human-computer interaction applications of self-growing models. Appl. Soft Comput. 2011, 11, 4413-4431.
27. Guamán, S.; Calvopiña, A.; Orta, P.; Tapia, F.; Yoo, S.G. Device Control System for a Smart Home using Voice Commands. In Proceedings of the 2018 10th International Conference on Information Management and Engineering-ICIME 2018, Salford, UK, 22-24 September 2018; pp. 86-89.
28. Portet, F.; Vacher, M.; Golanski, C.; Roux, C.; Meillon, B. Design and evaluation of a smart home voice interface for the elderly: Acceptability and objection aspects. Pers. Ubiquitous Comput. 2011, 17, 127-144.
29. Vacher, M.; Lecouteux, B.; Istrate, D.; Joubert, T. Evaluation of a real-time voice order recognition system from multiple audio channels in a home. In Proceedings of the 14th Annual Conference of the International Speech Communication Association, INTERSPEECH, Lyon, France, 25-29 August 2013; pp. 2062-2064.
30. Dumitrescu, S.D. Cassandra smart-home system description. In Proceedings of the 2017 International Conference on Speech Technology and Human-Computer Dialogue (SpeD), Bucharest, Romania, 6-9 July 2017; pp. 1-6.
31. Drosos, K.; Floros, A.; Agavanakis, K.; Tatlas, N.-A.; Kanellopoulos, N.-G. Emergency voice/stress-level combined recognition for intelligent house applications. In Audio Engineering Society Convention 132; Audio Engineering Society: New York, NY, USA, 2012.
32. Song, L.; Yuan, L. Design of IOS smart Home System Based on MQTT Protocol and Speech Recognition. J. Phys. Conf. Ser. 2018, 1069, 012046.
33. Yusri, M.M.; Kasim, S.; Hassan, R.; Abdullah, Z.; Ruslai, H.; Jahidin, K.; Arshad, M.S. Smart mirror for smart life. In Proceedings of the 2017 6th ICT International Student Project Conference (ICT-ISPC), Skudai, Malaysia, 23-24 May 2017; pp. 1-5.
34. Brenon, A.; Portet, F.; Vacher, M. Arcades: A deep model for adaptive decision making in voice controlled smart-home. Pervasive Mob. Comput. 2018, 49, 92-110.
35. Tiwari, V.; Hashmi, M.F.; Keskar, A.; Shivaprakash, N. Speaker identification using multi-modal i-vector approach for varying length speech in voice interactive systems. Cogn. Syst. Res. 2018, 57, 66-77.
36. Mittal, Y.; Toshniwal, P.; Sharma, S.; Singhal, D.; Gupta, R.; Mittal, V.K. A voice-controlled multi-functional Smart Home Automation System. In Proceedings of the 2015 Annual IEEE India Conference (INDICON), New Delhi, India, 17-20 December 2015; pp. 1-6.
37. Pereira, A.; Silva, F.; Ribeiro, J.C.B.; Marcelino, I.; Barroso, J. Smart Remote Control Design for Seniors. In Universal Access in Human-Computer Interaction. Access to Interaction. UAHCI 2015. Lecture Notes in Computer Science; Antona, M., Stephanidis, C., Eds.; Springer: Cham, Switzerland, 2015; Volume 9176, pp. 484-495.
38. Wu, L.; Lu, J.; Zhang, T.; Gong, J. Robot-assisted intelligent emergency system for individual elderly independent living. In Proceedings of the 2016 IEEE Global Humanitarian Technology Conference (GHTC), Seattle, WA, USA, 13-16 October 2016; pp. 628-633.
39. Pérez-Espinosa, H.; Martínez-Miranda, J.; Espinosa-Curiel, I.; Rodríguez-Jacobo, J.; Avila-George, H. Using acoustic paralinguistic information to assess the interaction quality in speech-based systems for elderly users. Int. J. Hum. Comput. Stud. 2017, 98, 1-13.
40. Djaid, N.T.; Saadia, N.; Ramdane-Cherif, A. Multimodal Fusion Engine for an Intelligent Assistance Robot Using Ontology. Procedia Comput. Sci. 2015, 52, 129-136.
41. Dogariu, M.; Cucu, H.; Buzo, A.; Burileanu, D.; Fratu, O. Speech applications in the eWALL project. In Proceedings of the 2015 International Conference on Speech Technology and Human-Computer Dialogue (SpeD), Bucharest, Romania, 14-17 October 2015; pp. 1-7.
42. Potamianos, G.; Huang, J.; Marcheret, E.; Libal, V.; Balchandran, R.; Epstein, M.; Seredi, L.; Labský, M.; Ures, L.; Black, M.; et al. Far-Field Multimodal Speech Processing and Conversational Interaction in Smart Spaces. In Proceedings of the 2008 Hands-Free Speech Communication and Microphone Arrays Conference, Trento, Italy, 6-8 May 2008; pp. 119-123.
43. Xue, S.; Yan, Z.; Yu, T.; Liu, Z. A Study on Improving Acoustic Model for Robust and Far-Field Speech Recognition. In Proceedings of the 2018 IEEE 23rd International Conference on Digital Signal Processing (DSP), Shanghai, China, 19-21 November 2018; pp. 1-5.
44. Caranica, A.; Cucu, H.; Burileanu, C.; Portet, F.; Vacher, M. Speech recognition results for voice-controlled assistive applications. In Proceedings of the 2017 International Conference on Speech Technology and Human-Computer Dialogue (SpeD), Bucharest, Romania, 6-9 July 2017; pp. 1-8.
45. Lin, F.; Song, C.; Xu, X.; Cavuoto, L.; Xu, W. Patient Handling Activity Recognition through Pressure-Map Manifold Learning Using a Footwear Sensor. Smart Health 2017, 1-2, 77-92.
46. Xu, H.; Yuan, C.; Li, P.; Wang, Y. Design and implementation of action recognition system based on RFID sensor. In Proceedings of the 2017 13th International Conference on Natural Computation, Fuzzy Systems and Knowledge Discovery (ICNC-FSKD), Guilin, China, 29-31 July 2017; pp. 3021-3025.
47. Rashid, K.M.; Louis, J.; Fiawoyife, K.K. Wireless electric appliance control for smart buildings using indoor location tracking and BIM-based virtual environments. Autom. Constr. 2019, 101, 48-58.
48. Abid, M.; Petriu, E.M.; Amjadian, E. Dynamic Sign Language Recognition for Smart Home Interactive Application Using Stochastic Linear Formal Grammar. IEEE Trans. Instrum. Meas. 2014, 64, 596-605.
49. Luria, M.; Hoffman, G.; Zuckerman, O. Comparing Social Robot, Screen and Voice Interfaces for Smart-Home Control. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, CO, USA, 6-11 May 2017; pp. 580-628.
50. Feng, Z.; Yang, B.; Xu, T.; Yang, X.; Xie, W.; Ai, C.; Chen, Z. FM: Flexible mapping from one gesture to multiple semantics. Inf. Sci. 2018.
51. Panëels, S.A.; Ritsos, P.D.; Rodgers, P.; Roberts, J.C. Prototyping 3D haptic data visualizations. Comput. Graph. 2013, 37, 179-192.
52. Hussain, M.A.; Ahsan, K.; Iqbal, S.; Nadeem, A. Supporting deafblind in congregational prayer using speech recognition and vibro-tactile stimuli. Int. J. Hum. Comput. Stud. 2019, 123, 70-96.
53. Vega-Barbas, M.; Pau, I.; Augusto, J.C.; Seoane, F. Interaction Patterns for Smart Spaces: A Confident Interaction Design Solution for Pervasive Sensitive IoT Services. IEEE Access 2018, 6, 1126-1136.
54. Yue, P.; Jing, L.; Lei, X. A Study on Intelligent Housekeeper of Smart Home System. In Proceedings of the 2017 9th International Conference on Measuring Technology and Mechatronics Automation (ICMTMA), Changsha, China, 14-15 January 2017; pp. 124-127.
55. Rognini, G.; Blanke, O.; Information, P.E.K.F.C. Cognetics: Robotic Interfaces for the Conscious Mind. Trends Cogn. Sci. 2016, 20, 162-164.
56. Glodek, M.; Honold, F.; Geier, T.; Krell, G.; Nothdurft, F.; Reuter, S.; Schüssel, F.; Hörnle, T.; Dietmayer, K.; Minker, W.; et al. Fusion paradigms in cognitive technical systems for human-computer interaction. Neurocomputing 2015, 161, 17-37.
57. Lee, E.-J.; Park, S.-J. Configuring a Residential Hologram System to Complement the Cognitive Function of the Elderly. J. Archit. Inst. Korea Plan. Des. 2018, 34, 67-74.
58. Civitarese, G.; Belfiore, S.; Bettini, C. Let the objects tell what you are doing. In Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing-UbiComp' 16, Heidelberg, Germany, 12-16 September 2016; pp. 773-782.
59. Fortin-Simard, D.; Bilodeau, J.-S.; Gaboury, S.; Bouchard, B.; Bouzouane, A. Bastien Method of Recognition and Assistance Combining Passive RFID and Electrical Load Analysis That Handles Cognitive Errors. Int. J. Distrib. Sens. Netw. 2015, 2015, 1-18.
60. Källström, M.; Berdal, S.; Joshi, S.G. Designing an Indoor Navigation System for Elderly People's Capabilities. Lect. Notes Comput. Sci. 2015, 9194, 435-445.
61. Zhao, H.; Ma, Y.; Wang, S.; Watson, A.; Zhou, G. MobiGesture: Mobility-aware hand gesture recognition for healthcare. Smart Health 2018, 9-10, 129-143.
62. Yamazaki, T.; Yamazaki, T. Communicative robot interface for the ageing society. In Proceedings of the 2012 12th International Conference on Control Automation Robotics & Vision (ICARCV), Guangzhou, China, 5-7 December 2012; pp. 668-671.
63. Gross, H.-M.; Schroeter, C.; Mueller, S.; Volkhardt, M.; Einhorn, E.; Bley, A.; Langner, T.; Merten, M.; Huijnen, C.; Heuvel, H.V.D.; et al. Further progress towards a home robot companion for people with mild cognitive impairment. In Proceedings of the 2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Seoul, Korea, 14-17 October 2012; pp. 637-644.
64. Mokhtari, M.; Aloulou, H.; Tiberghien, T.; Biswas, J.; Racoceanu, D.; Yap, P. New Trends to Support Independence in Persons with Mild Dementia-A Mini-Review. Gerontology 2012, 58, 554-563.
65. Alrajhi, W.; Alaloola, D.; Albarqawi, A. Smart home: Toward daily use of BCI-based systems. In Proceedings of the International Conference on Informatics, Health & Technology (ICIHT), Riyadh, Saudi Arabia, 21-23 February 2017; pp. 1-5.
66. Fredericks, E.M.; Bowers, K.M.; Price, K.A.; Hariri, R.H. CAL: A Smart Home Environment for Monitoring Cognitive Decline. In Proceedings of the 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS), Vienna, Austria, 2-6 July 2018; pp. 1500-1506.
67. Schwan, J.; Ghaleb, E.; Hortal, E.; Asteriadis, S. High-performance and lightweight real-time deep face emotion recognition. In Proceedings of the 12th International Workshop on Semantic and Social Media Adaptation and Personalization (SMAP), Bratislava, Slovakia, 9-10 July 2017; pp. 76-79.
68. Chen, M.; Ma, Y.; Hao, Y.; Li, Y.; Wu, D.; Zhang, Y.; Song, E. CP-Robot: Cloud-Assisted Pillow Robot for Emotion Sensing and Interaction. In Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering; Wan, J., Humar, I., Zhang, D., Eds.; Springer: Cham, Switzerland, 2016; pp. 81-93.
69. Rodic, A.; Jovanović, M.; Stevanović, I.; Karan, B.; Potkonjak, V. Building technology platform aimed to develop service robot with embedded personality and enhanced communication with social environment. Digit. Commun. Netw. 2015, 1, 112-124.
70. Qian, Y.; Lu, J.; Miao, Y.; Ji, W.; Jin, R.; Song, E. AIEM: AI-enabled affective experience management. Futur. Gener. Comput. Syst. 2018, 89, 438-445.
71. Huang, Y.-C.; Wu, K.-Y.; Liu, Y.-T. Future home design: An emotional communication channel approach to smart space. Pers. Ubiquitous Comput. 2013, 17, 1281-1293.
72. Costa, Â.; Rincon, J.A.; Carrascosa, C.; Julian, V.; Novais, P. Emotions detection on an ambient intelligent system using wearable devices. Future Gener. Comput. Syst. 2019, 92, 479-489.
73. Thakur, N.; Han, C.Y. A complex activity based emotion recognition algorithm for affect aware systems. In Proceedings of the 2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC), Las Vegas, NV, USA, 8-10 January 2018; pp. 748-753.
74. Gamecho, B.; Da Silva, H.P.; Guerreiro, J.; Gardeazabal, L.; Abascal, J. A Context-Aware Application to Increase Elderly Users Compliance with Physical Rehabilitation Exercises at Home via Animatronic Biofeedback. J. Med. Syst. 2015, 39.
75. Curumsing, M.K.; Fernando, N.; Abdelrazek, M.; Vasa, R.; Mouzakis, K.; Grundy, J. Emotion-oriented requirements engineering: A case study in developing a smart home system for the elderly. J. Syst. Softw. 2019, 147, 215-229.
76. Mano, L.Y. Emotional condition in the Health Smart Homes environment: Emotion recognition using ensemble of classifiers. In Proceedings of the 2018 Innovations in Intelligent Systems and Applications (INISTA) Conference, Adana, Turkey, 4-6 October 2018; pp. 1-8.
77. Marti, P.; Iacono, I. Social and empathic behaviours: Novel interfaces and interaction modalities. In Proceedings of the 2015 24th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), Kobe, Japan, 31 August-4 September 2015; pp. 217-222.
78. Wei, J.; Liu, H.; Wang, B.; Sun, F. Lifelong learning for tactile emotion recognition. Interact. Stud. 2019, 20, 25-41.
79. Lanjewar, R.B.; Mathurkar, S.; Patel, N. Implementation and Comparison of Speech Emotion Recognition System Using Gaussian Mixture Model (GMM) and K-Nearest Neighbor (K-NN) Techniques. Procedia Comput. Sci. 2015, 49, 50-57.
80. Yang, S.-H.; Liu, X.-W.; Lo, Y.-C. A design framework for smart TV: Case study of the TaipeiTech smart TV system. In Proceedings of the 2016 IEEE International Conference on Consumer Electronics-Taiwan (ICCE-TW), Nantou, Taiwan, 27-29 May 2016; pp. 1-2.
81. Rathnayake, K.A.S.V.; Wanniarachchi, W.K.I.L.; Nanavakkara, W.H.K.P. Human Computer Interaction System for Impaired People by using Kinect Motion Sensor: Voice and Gesture Integrated Smart Home. In Proceedings of the 2018 Second International Conference on Inventive Communication and Computational Technologies (ICICCT), Coimbatore, India, 20-21 April 2018; pp. 531-536.
82. Almusaylim, Z.A.; Zaman, N. A review on smart home present state and challenges: Linked to context-awareness internet of things (IoT). Wirel. Netw. 2018, 25, 3193-3204.
83. Lee, H.J.; Kim, K.H.; Kim, Y.H. Wireless Sensor Network-Based 3D Home Control System for Smart Home Environment. Int. J. Smart Home 2016, 10, 159-168.
Zhen Li1, Jiao Zhang1, Mengwan Li1, Jizhuo Huang2,* and Xiangyu Wang2,3
1School of Management, Jiangsu University, Zhenjiang 212013, China
2College of Civil Engineering, Fuzhou University, Fuzhou 350108, China
3Australasian Joint Research Centre for Building Information Modelling, School of Built Environment, Curtin University, Perth, WA 6845, Australia
*Author to whom correspondence should be addressed.
© 2020. This work is licensed under http://creativecommons.org/licenses/by/3.0/ (the “License”). Notwithstanding the ProQuest Terms and Conditions, you may use this content in accordance with the terms of the License.