Human-in-Loop: A Review of Smart Manufacturing Deployments

Abstract

The recent increase in computational capability has led to an unprecedented increase in the range of new applications where machine learning can be used in real time. Notwithstanding the range of use cases where automation is now feasible, humans are likely to retain a critical role in the operation and certification of manufacturing systems for the foreseeable future. This paper presents a use case review of how human operators affect the performance of cyber–physical systems within a ’smart’ or ’cognitive’ setting. Such applications are classified using Industry 4.0 (I4.0) or 5.0 (I5.0) terminology. The authors argue that, as there is often no general agreement as to when a specific use case moves from being an I4.0 to an I5.0 example, the use of a hybrid Industry X.0 notation at the intersection between I4.0 and I5.0 is warranted. Through a structured review of the literature, the focus is on how secure human-mediated autonomous production can be performed most effectively to augment and optimise machine operation.

1. Introduction

The seamless integration of the physical and digital world has led to an unprecedented increase in the rate at which automation can be introduced in production. Such smart manufacturing [1] or cyber–physical systems (CPS) [2] will involve the use of artificial intelligence (AI) or machine learning (ML) at some fundamental level. The degree to which the AI/ML interacts with the human user within the production system affects the classification of such a system. This has led to production system examples that have been denoted as either Industry 4.0 (I4.0) or Industry 5.0 (I5.0) in the literature [3,4]. I4.0 introduced the amalgamation of digital technology within the manufacturing processes to create an intelligent manufacturing system. Although I4.0 appears to set aside the human component, the ubiquitous integration of automation technologies in the industrial framework sparked the emergence of I5.0, where humans can work alongside robots rather than being replaced by automated systems. I5.0 focuses on human-centric manufacturing by putting human interests as the core focus of production, and technology which aids industrial workers by developing their knowledge and abilities [2,5,6,7]. I5.0 also focuses heavily on two areas in manufacturing research: (1) sustainability: lower energy use, lower CO${}_2$ emissions, lower waste generation, and higher rates of material reuse and recycling in a circular economy; (2) resilience: creating and using more durable industrial processes and equipment, as well as more resilient supply networks [5].

There tends to be some confusion as to when an I4.0 use case morphs into one that can be described as I5.0. We argue that in many cases a hybrid classification, which we denote here as Industry X.0 (IX), is appropriate [8]. Here, the interaction between the AI and humans takes place but the range of interaction is often quite limited. An operator may have little more responsibility than sign-off or limit setting for the ML algorithm in question and cannot be classed as being fully ’in the loop’. Such systems will however use a broad range of sensor networks to collect and transfer data across a process, to train, where appropriate in real time, and to dynamically measure the performance of the system in real time [9]. The constant monitoring and storage of various manufacturing parameters have led to an improvement in performance, as reported by the broad literature [9,10,11]. Many examples now exist where the availability of real-time data has provided new actionable insight where ML can be used to shape process characteristics including, inter alia, product quality, machine parameters for optimal raw material usage, reduced environmental footprints, and the real-time control of a variety of external factors [9,12]. Intelligent manufacturing systems that exhibit a range of high-level functionality in real time, including effective decision-making [9,13] control of concurrent production lines [10], and optimisation of the machine maintenance and scheduling [14,15].

Antunes and Palma [16] show that incorporating a human-in-the-loop assistance controller improves human–computer interface. Qian et al. [17] propose a framework where actively adding humans to the learning loop requires minimal additional guidance in extracting the semantic attributes in image annotation. Yucelen et al. [18] built a mathematical framework using stability analysis tools, where the stability limit of the human closed-loop system and the range of model parameters are calculated in an adaptive flight control application. Therefore, the authors believe that now is an appropriate time to reassess the literature from a perspective that specifically focuses on human-in-loop (HIL) issues. In particular, we provide an assessment of the recent deployments that necessarily involve human–machine interfaces so that a production system can be certified as fit for purpose. A full HIL implementation will be, by definition, an I5.0 use case. This work, therefore, builds on previous work, for example [6,19], that surveys the different ML techniques used in industrial manufacturing systems. This review updates the literature and outlines the current state of the art with respect to the use of ML in real-time smart manufacturing where HIL is a key feature.

Humans are likely to continue to play an important role in manufacturing operations for the foreseeable future [20,21]. In such a setting, the interaction between humans and machines needs to be described by a socio-technical model that is rich enough to capture the complexity of such an intricate system yet sufficiently tractable to support real-time decision-making. Although many surveys have been carried out on the benefits of ML on manufacturing, the role of human operators in smart factories is beginning to gain attention [7]. A full characterisation of human–machine interaction is a significant challenge as new reference models and architectures for smart manufacturing use cases continue to appear [22]. This review outlines the key principles of I5.0, provides a systematic review outlining the importance of HIL within ML and data engineering context in manufacturing, digests the current research, and presents an extensive list of available references in the field. We curate a range of use cases where humans liaise with fully or semi-autonomous production systems, thereby characterising the advantages and challenges which obtain at present. We also focus on applications where bias associated with ML in the involvement of HIL has been reported.

The paper is organised as follows. The next section, Section 2, reviews the literature on the existing works carried out on the concept of ‘human-in-loop’ and the roles that human operators can play in a smart manufacturing system. Section 3 addresses the research questions and research methodology. The rest of the paper is structured into the following sections and subsections: Section 4: implications of a move towards Industry 5.0 for HIL performance standards; Section 5: cyber–physical systems; Section 5.1: decision-making in CPS; Section 5.2: data visualisation; Section 6: digital twin technology in an HIL setting; Section 6.2: service applications that support HIL for digital twin; Section 6.1: tools to model and manage digital twin applications; Section 6.3: human-in-loop as human–machine interface; Section 7: contextualisation; Section 8: hybrid intelligence; Section 8.1: bottleneck detection integrating human in loop; Section 9: security aspects of human-in-loop, as given in Figure 1.

Section 4 discusses the challenges of describing a particular use case as I5.0 when many companies are still grappling with the implications of deploying I4.0 technologies, the need to move towards I5.0 principles, and why a hybrid IX category is justified at the present time. In Section 5, intelligent systems with combined physical and computational capabilities that can communicate with humans via a variety of new modalities, commonly known as cyber–physical systems (CPS), are discussed along with a discussion of the different roles played by human operators within smart manufacturing systems. Section 6 assesses the new cutting-edge technology, the digital twin, and the importance of HIL within it. Section 7 describes context-aware computing and bias associated with HIL deployments. Section 8 details the importance of hybrid intelligence and the importance of HIL in improving the accuracy of decision-making. Section 8.1 discusses the effect of HIL to detect bottlenecks. Section 9 elaborates how HIL can be beneficial to detect cyber attacks from within a manufacturing context. It is the author’s opinion that insufficient work has been carried out heretofore in relation to a review of the security aspects within the manufacturing industry and, therefore, this section is considered a particularly novel contribution of the work. A standard architecture to detect the potential threats based on the digest of the literature is also proposed. Finally, Section 10 provides a discussion and conclusion of the article and outlines some future research directions.

2. Theoretical Background and Previous Works

‘Human-in-loop’, (HIL) can refer to ‘operators’ and/or ‘agents’ who operate or monitor a system. HIL can also possibly refer to ‘users’ who receive services from and affect the performance of a system. The term ’Human/Operator 4.0’ has been used to describe people in an I4.0 setting equipped with sufficient digital skills so that they can add value to production use cases involving the use of data analytics, machine learning and artificial or augmented intelligence. Humans and technology will be required to interact more intensively and efficiently in use cases that are described as I5.0. Therefore, HIL will continue to play an essential role in any manufacturing setting where I4.0 moves toward I5.0 and certification of process or product quality is necessary. The role that automation and AI play in enhancing the human ability to affect physical, cognitive and sensory performance is becoming more pronounced. Authors such as Cimini et al. [23] and Jones et al. [24] highlighted the need to enhance the role of ‘human-in-loop’ (HIL) in manufacturing systems.

However, the role that AI can play in interpreting the performance of production environments that are complex systems of systems is also of interest. One of the first models to represent systems where each task is performed as a combination of a human operator interacting with a machine is given in Miller and Parasuraman [25]. Here, interaction with a machine is classified in terms of levels of automation denoting the different allocation of physical and cognitive tasks between resources (humans and technology). These steps are described as discrete transitions from 1 (totally manual) to 7 (totally automatic), forming a 7 by 7 ‘levels of automation’ matrix consisting of 49 possible automation solution types [26]. In step (7,7), the machine performs all tasks under a predefined set of normal conditions, and the human operator only intervenes if a problem is encountered that the machine cannot fix. This type of interaction is known as human–machine interface (HMI), discussed later in Section 6.3. Another way to model is using joint-cognitive systems (JCS) where the objective is to shift from a physical conception of HMIs to a cognitive concept. Cognitive automation can be defined as the technical solutions to help the operator, e.g., HOW and WHAT to do and the situation control. This has led to the advent of smart machines with capabilities of reasoning and representing things as humans do [27]. For correct communication between human operators and machines, an effective and mutual understanding of plans and goals is needed. Here, AI and deep learning play a huge role to make machines smarter while interacting and then subsequently learning from human agents, improving their interaction [24]. With the growing automation in manufacturing plants, the importance and role of HIL is progressing rapidly. With the evolving technology, the effectiveness of HIL is increasing as human operators are constantly engaging with a large amount of data and decision-making process chains in manufacturing facilities. This necessitates research in data-driven production facilities integrating the role of HIL for a more successful blending of human intelligence and CPS, including sensor and visual data analytics as an interactive method. HIL engagement is linked to cognitive capabilities and underlining the importance of contextual information management in industries [28]. Such context-driven interaction can be greatly facilitated by a visual analytics environment. Early data analysis had limited options for user interaction [29,30], but current visual analytics are increasingly upgrading the interaction capabilities [31,32]. Visually enhanced data representation enables users to comprehend the data easily compared to reviewing raw data [33]. Situational awareness and context management are vital in resolving context ambiguity to assess industrial alarms, which if unresolved can overwhelm human operators with innumerable alerts [34].

In January 2021, the European Commission published a perspective on Industry 5.0 or the updated Industry 4.0 paradigm which focuses on human-centric manufacturing to achieve holistic digital transformation [35]. Instead of replacing humans, the manufacturing system will focus on improving collaboration between humans and manufacturing appliances.

How This Review Extends Existing Work

This paper presents a conceptual view of the implementation and need for the HIL concept in a range of scenarios within the future of manufacturing instead of focusing on the categorisation and statistics of the existing literature (e.g., based on specific techniques or models), and it investigates the underlying reasons for the difficulties in implementing HIL.

3. Research Questions and Methodology

This work has been produced using a structured procedure that included a review of the research material that has already been published. The initial questions that this research set out to explore and investigate focus on the roles of humans in smart manufacturing in the move towards I5.0. Considering the socio-technical aspects and the role of human-in-loop in smart factories, the research questions addressed in this paper are:

• RQ1: How can the differences between I4.0 and I5.0 use cases be classified and is a new hybrid IX necessary?

• RQ2: How is the role of humans in smart manufacturing impacted as a result of the implementation of AI/ML technologies?

• RQ3: How can the performance of humans within an HIL cyber–physical production process be assessed or benchmarked? In this paper, the different levels of human involvement in a CPS and their relative importance have been investigated.

• RQ4: Where and how does a ‘human-in-loop’ concept add value in a smart manufacturing setting?

Figure 2 illustrates the literature review approach that is used to complete this study and identify the final collection of publications for analysis.

Table 1 displays the search terms used to identify relevant concepts within papers. The search terms are derived from an initial review of different topics within smart manufacturing, and then delineated based on the topics decided to be most suitable by the respective authors. Scopus, a publication database, is used to find pertinent articles that have been indexed mainly between 2000 and 2022. The Web of Science and Scholar databases are also utilised as comparisons to find additional works that Scopus has missed. Table 1 shows the search terms used to identify relevant concepts within papers, the peak year for paper publishing, the number of publications in 2021, and the total number of papers.

This first stage of screening the papers aided in determining which works are most pertinent to the study’s stated research questions. Initially, socio-technical systems are investigated, aiming to analyse the human–machine interaction to understand the performance of a complex system. A second stage involved further filtering with additional attention given to papers that are more likely to contribute to the integration of human operators in the manufacturing industries. At this step, each paper’s abstract, introduction, and conclusions (findings and future research) are quickly examined. The entire total of papers is lowered by the second stage from 2043 to 250. Additionally, more recent publications (those published after 2015) are given a higher weight, which resulted in a majority of these works in the final evaluation. A complete reading of the remaining papers started the final phase of the review, which reduced the total to slightly over 130 relevant works. At this point, a comprehensive review of the remaining publications included evaluations of the publication’s contribution and applicability as well as its impact factor score (as rated by Clarivate). According to the literature assessment undertaken for this study, it is clear that manufacturing systems will still need a ‘human-in-loop’ to offer supervisory level mediation for even the most autonomous implementations in the near to mid future. According to the theory presented in Trist [36], in order to model the next-generation manufacturing systems, research on technology and HIL are needed to be done in parallel. Therefore, I4.0 technologies impacting the human operators directly are explored in order to define the different roles of humans within smart factories.

4. Implications of a Move towards Industry 5.0 for HIL Performance Standards

Although the current literature offers different descriptions of I4.0 [37,38,39], it can be broadly described as the incorporation of digital technologies to support systems integration and automation of production. These technologies might include inter alia the industrial Internet of things (IIoT), ML and AI, cloud, edge and cognitive computing, as well as automatic decision-making. Allied with data collection, AI techniques, real-time analysis, and the capacity for a production line’s components to communicate with one another, production can become extremely efficient and customised. I4.0 aims to maximise productivity and efficiency by restricting humans’ roles to keep up with automation. This automation-only technique produced huge growth in the past, but currently, productivity rates depict stagnation in many economies [40,41].

The European Union has provided a guideline for a move to I5.0 [35], which does not attempt to replace the human factor but rather to empower it through the effective integration of human cognitive capabilities within industrial systems. This concept is further strengthened by reviews presented by Emmanouilidis et al. [28,34]. I5.0 focuses on the cooperation between human intelligence and cognitive computing, leading to automation which is an improvement of human capabilities and not the exclusion of the human component [42]. Recouping humans back into the loop, I5.0 reconstructs the tasks of human operators in the realm of manufacturing with methods which benefit the HIL, i.e., to provide value-added services and to work alongside autonomous workforce, collaborative robots, etc., which are informed about the human’s desires [3,43]. A cursory survey of the literature on I5.0 use cases [4,44,45] shows that there is some confusion about where I4.0 stops and I5.0 starts in relation to the classification of a particular contribution.

Some of the classification discourse might be described as premature at the present time. While it is clear that the move from I4.0 to I5.0 has already started, the risk of misclassification for certain use cases that invites criticism from the research community is very real. We suggest that case studies where I4.0 technologies are applied to examples where HIL is a necessary component should be described as a hybrid Industry X.0 challenge. The term is introduced by Schaeffer [8]; IX examines important facets of the IIoT, analysing and describing those in an entertaining and approachable way. It is based on in-depth research and insights into the essential competencies identified in industries, which include handling the management of smart data, managing the creation of digital products, educating the workforce, mastering innovation, maximising platforms and ecosystems, and many more. Figure 3 highlights the hybrid IX concept, which is the synergy of I4.0 technologies and I5.0 principles.

Although AI and ML techniques are being used, human interfaces are necessary, but the level of AI is not yet sufficiently evolved to self-heal or self-diagnose. Along with I4.0, terms such as operator/worker 4.0, etc., are also being coined to differentiate the significance and level of human skills that are required [42], which again poses difficulties for classification. There are many examples of I4.0 use cases where performance would be hindered without some level of HIL. In Figure 4, we have tried to capture this complexity by summarising the various influences that automation and interaction technologies can have on human cognition.

5. Cyber–Physical Systems

In the prior sections, the research literature on RQ1 and RQ2 has been considered through a comparison of the technologies and concepts of I4.0/5.0/X and how each level impacts the roles of human operators in a smart factory. In the present section, the discussion moves to RQ3, i.e., the role that humans will play within a cyber–physical manufacturing process.

A cyber–physical system (CPS) suggests an intelligent process monitored by computer-based algorithms. CPS consists of intelligent devices, storage systems, and manufacturing facilities that can communicate, act, and be controlled independently and autonomously [12,46]. HIL in CPS makes it easier to include human abilities within a selection of its feedback control loops. The necessity for HIL in CPS in the manufacturing domain is clear as it is likely that there will remain a large number of tasks that are resistant to complete automation for the foreseeable future. Interaction between humans and machines, transfer learning, and support will all continue to be crucial in the development of intelligent production systems. Different researchers have reported on the importance of multi-agent architectures and the need for a distributed decision support system approach to integrate human operators, such as [47] and the references therein. When adopting a transparent architecture based on an interactive multi-objective scheduling strategy, humans in CPS supervise the online management of the decision process to track the system’s overall performance. However, incorporating the role of HIL in the design of CPS is not yet fully matured. The following three challenges can be identified if one wishes to seamlessly integrate the role of HIL within a CPS, denoting such integration as HILCPS.

• What is the spectrum of physical principles, models, and solutions that are available for the task at hand? A systems-based use of HIL within CPS can only be certified when an appropriate HILCPS is defined through a thorough first principles understanding of such models.

• Reliable modelling techniques are required to detect, classify, and possibly predict human behaviour. Current state-of-the-art techniques are now presented where available. A generalised dynamic human behaviour for interaction with a CPS model remains an open research question.

• Human behaviour models must be incorporated with some supervisory architecture, be it feedback, feed-forward, or any form of hierarchical control action.

In Wang and Haghighi [48], it is suggested to use a CPS architecture in which a network layer connects two layers of physical and cyber processes. However, this architecture treats human operators who interact with the physical and cyber layers as separate entities when compared to the rest of the system, which also interacts with the physical and cyber units.

Based on the given architecture in [48], a CPS architecture has been proposed in Cimini et al. [23], with the inclusion of human operators with respect to the physical, control, and cyber layers. The layers are defined as follows:

• Physical layer is where sensor devices mounted on machines are used to communicate with human operators. Here, decisions are made at the edge level with local human intelligence. Smart equipment can be used as an add-on to either implement self-adjusting strategies or to provide additional information to the humans involved for direct intervention. The physical layer supports data collection from the production floor which is then sent to other layers for processing so that decisions can be made in real time.

• Cyber layer is where the data is sent from the physical layer for data processing, visualisation, and virtualisation of the system [49]. In [50] it is shown how HIL can access the sensor data for data analytics to enhance decision-making, operational and financial performance. The human interaction in the cyber layer here is similar to the human-in-mesh approach given in Fantini et al. [51]. Software technology directs the human operator’s interaction with the manufacturing system, allowing the human operator to interpret factory units through a form of cyber-representation and make better-educated decisions that are subsequently communicated to the physical layer.

• Control layer is the layer between physical and cyber layers which reproduces the functioning of every physical object. The holonic and multi-agent control approach presented is compatible with industrial control systems (e.g., SCADA), which, with proper integration, provide control over the manufacturing process [52].

  The control layer’s key attributes that are presented in the above work can be classified as:

• Data are collected from machines and other physical objects via sensors.

• Using a multi-agent system, physical object behaviour is modelled.

• The control layer that links the physical and cyber layers enables the transfer of decisions from the latter to the factory floor’s equipment via actuators.

Alternatively, in [48], humans are considered to be an external entity to the production floor and only communicate via the cyber layer using interfaces. In the architecture defined in Figure 5, human operators are embedded in all three levels, such an architecture is more in accordance with Industry 5.0 principles.

5.1. Decision-Making in CPS

In the HILCPS architecture as given in Figure 5, the decision-making system (DMS) resides in the cyber layer, where the collected data from the factory floor are analysed to enhance the decision-making process; however, the related output is entirely dependant on the quality of the data. In Jugulum [53], the quality of the data can be analysed in four dimensions, completeness, conformity, validity, and accuracy. A measurement to assess the data quality is suggested as confirming the data quality is the most important step to developing data analytics to support decision-making.

The decision-making in CPS is built on digital representations of real-world objects, or digital twins discussed in more detail in Section 6 and continuous synchronisation with its control properties [54]. In Gölzer and Fritzsche [55], the decision-making procedures that are data-driven are discussed to underline the potential of data-driven autonomous feedback control loops. In Cimini et al. [23], HIL is involved in the decision-making process enhancing cognitive capabilities by utilising continuous real-time monitoring, analytics, simulation and optimisation. The decision-making tools in supply chain management are categorised into data analysis, modelling, control, and learning. The role of HIL in decision-making can range from a hierarchical to a flatter organisational structure. In a hierarchical organisation, the human operator at the cyber level carries out a decision-making role, while at the physical level, the human operator carries out a decision-actuator role. The tools augmenting the cognitive ability of the human operator on the factory floor can support on-site decision-making for operational optimisation. This allows for improved convergence between the human operator in the physical layer and the human decision maker in charge of the cyber layer. Overall, the ultimate objective is to enhance or augment the decision-making process, and in particular to improve the way in which human intelligence can help the production system so as to achieve target performance goals and strengthen the HIL objectives.

5.2. Data Visualisation

Visualisation plays an important role in any human-attributed or digital transformation system. In a typical visualisation framework, data are collected, stored, and then retrieved through appropriate management tools. This process enables the cognitive processing of HIL using data categorisation, perception, assessment, monitoring, and prediction. A theoretical framework of HIL in visual analytics is presented in Figure 6.

In Li et al. [56], a categorisation of visual procedures for human–computer interaction (HCI) is provided, outlining a process to select a suitable design method. In Tran and Le [57], the human vision system has been explored to derive better techniques, more fitting to understand complex data streams. Keeping in mind human vision limitations, multidimensional graphs are proposed to increase the ease of perception by the decision actuator.

In smart manufacturing, the importance of visualisation and the requirement for human-understandable communication in a variety of scenarios such as automation, product design and development, and production are highlighted throughout the literature [58]. Content-related visual depiction is provided in Lade et al. [59] through five categories of analytics: reduction of test time and calibration; improvement of quality; reduction of warranty cost; improvement of yield by benchmarking lines; predictive maintenance. Automation of graphical presentation of analytics data and matching of different graph types resolve patterns in data efficiently [60].

Innovations in the display of snapshot examination of real-time sensor data are in process. Ribbon flow diagrams are improved by employing a case study based on data sets related to an industrial pump product, and presented by amalgamating technical parameters and market data within the same graph in Vosough et al. [61]. Automating matching of data with apt illustrative visualisation types and additional development of this technique, i.e., the development of a system pointing visualisation recommendations when presented with any dataset, using an ML technique to select examples, and keywords can be added to sway the choice of visualisation, as given in Luo et al. [62]. The field of graph grammars is utilised for context-based display of industrial data to examine the automated synthesis of graphs from input data. Graph grammars are used to comprehend complex systems with visual programming languages and have been studied in Zou et al. [63], focusing on the establishment of context awareness about the successful fusion of different parameter sets. In other studies, the generation of graphs from data to understand the rationale behind the visualisation is explored in Lensen et al. [64] using genetic programming ‘to evolve interpretable mappings from the data set to high quality visualisations’. Ontologies also play a big role in understanding visual data analytics [65].

6. Digital Twin Technology in a HIL Setting

Digital twin technology is a virtual representation of a system, updated by real-time data, that uses simulation, ML, and reasoning to enhance decision-making. This idea promises to give production systems real-time control [66]. Dynamic visualisation in multi-mode is the key requirement of this concept, which offers the user a close approximation of the application domain in real life, and is used for training and problem solving, as given in Figure 7.

Capturing any differences between a simulation and a digital twin (DT) is critical to the success of any use case. The simulation is an offline, conditional experimentation [67], while the latter is a real-time event and the quality of the model determines how accurate any simulation will be. The DT technology explores how the user interaction is captured by the CPS sensors and actuators, and the loss of information between the real and simulated events is kept vanishingly small. With the aid of augmented reality, discrete events can be overlaid with simulation model layouts in real-time over live manufacturing line scenes via headsets and hand devices. Closed loop decision-making that is facilitated using mixed or augmented reality environments is another way in which this information loss can be minimised [68]. Case studies involving virtual reality (VR) presentations of manufacturing settings that are boosted using motion and depth sensors such as Kinect have yielded promising results [69]. Models are built using industrial floor layouts for operator control of discrete event simulation capturing real-time movements and voice commands [70]. Radio frequency identification (RFID) labels are used to monitor and manage logistics on the industry floor, focusing on the visualisation of logistics trajectories [70]. There is a need for research on DT-based manufacturing system feedback control loops [54,71]. DT has also been investigated as a method to control and visualise information flow for more comprehensive product development, using the finished product’s performance as a feedback loop in designing new products [72]. Both larger and small- and medium-sized enterprises (SMEs) can benefit from digital twin technology, as it provides unified data acquisition. Lower price point solutions are emerging in the literature [67,72,73,74]. The need for additional research in DT and visualisation is highlighted to address questions such as:

• what is the amount of autonomous operation and feedback from the industry floor that will be facilitated through the DT?

• what modifications may be made to better integrate HIL with DT technologies??

To improve urban planning, building, and service, Dassault created a “Digital Twin Singapore” for civil engineering using its 3D Experience Platform [75].

6.1. Tools to Model and Manage Digital Twin Applications

The enterprise tools that are now appearing for DT applications can be broadly divided into the platform, simulation, optimisation, diagnostic, and prognosis categories. Across these categories, data quality is the primary driver behind DT fidelity. As displayed in Figure 8, the various categories, namely collection, transmission, storage, processing, fusion, and visualisation, are the tools for managing data in a DT.

The Thingworx platform [76] has been used to link the DT model to operational products, gather, and display sensor data, and examine results via web applications. Data collection, device administration, big data analysis, industrial protocol translation, and many other services are offered by Thingworx. HIROTEC [77] has also provided case studies in the production of body-in-white closures, exhaust systems, and enclosure manufacturing. Connections between the data from CNC machine operations in real-time and enterprise resource planning system data have been shown to reduce equipment downtime successfully. Siemens has launched the MindSphere platform [78], which provides real-time, secure transmission of data collected from sensors, controllers, and various information systems to the cloud enabling big data analysis and other services. Other data collection tools are also shown in Figure 8. In DT, data transmission needs to be continuous and in real time. Simultaneously, it is also needed to confirm that the data are not missing, the accuracy of the data is maintained to the maximum degree, and that the distance between the real system and DT models remains acceptably small. The container DT model presented by Jedermann et al. [79] provides a state space-based approach to the measurement of this distance. IBM’s Aspera [80] is known for its capacity for fast data transmission of large files over long distances under substandard network speeds. It transmits data using the current WAN infrastructure more quickly than the file transfer protocol (FTP) and hypertext transfer protocol (HTTP). Other tools commonly used for data transmission are shown in Figure 8.

Data storage warrants the safeguarding of data and answers to data access in real time using the read–write mechanism. For instance, Apache HBase is a high-performance, column-oriented, real-time scalable read–write distributed database that is built on the Hadoop software platform. Other data storing systems are shown in Figure 8.

Data processing is the manipulation of data to extract useful information. Apache Spark is an open-source unified analytics engine for real-time data processing and data analytics. Java, Python, and other programming languages are supported by Spark. Data query is also supported using SQL and HiveSQL.

Data fusion combines, correlates, and amalgamates several data sources to provide information that is more reliable, accurate, and practical than information obtained from any one data source alone. In Python, Spyder and Pycharm are used as data fusion software to develop, debug, and as project management, smart prompting, auto-completion, and version control. Other data fusion tools are shown in Figure 8.

Data visualisation provides a graphical representation of data as already discussed in Section 5.2. The open-source software Echarts provides customised data visualisation for large and dynamic data. Similar tools are shown in Figure 8.

6.2. Service Applications That Support HIL for Digital Twin

Service platform tools fuse emerging technologies such as the Internet of things (IoT), big data, and AI. The application tools include monitoring tools, optimisation tools, diagnostic tools, etc. The diagnosis tools provide intelligent predictive maintenance strategy for equipment and reduce downtime, etc., by analysing and processing the twin data. For example, the ANSYS simulation tool helps to design IIoT-connected devices and data analysis of the connected devices, along with design data for troubleshooting and predictive maintenance purposes [81]. Additionally, data-driven methods can be integrated to determine the remaining tool life to inform the human operator when the replacement of machine parts is needed. Such an example can be found in Baker Hughes, which is an oil industry company. They provide products and services to the oil industries and have developed a predictive alarm system on MATLAB [82].

In an HIL setting, the quality of sensor data, energy costs, and required performance factors are all factors that affect the quality of simulations. Control system operation will only be successful through real-time comparison of simulation and real-world states, measurements, inputs, and outputs. The mitigation of risk factors, reducing energy consumption, and increasing system efficiencies have all been considered in the literature. For example, the Plant Simulation software developed by Siemens is able to optimise the scheduling of the manufacturing line and the layout of the factory [83]. Within the digital twin electric grid, Simulink runs several simulated scenarios and analyses the measured grid data to assess whether the energy reserve is adequate and if the grid controllers need to be altered. Comparable tools are shown in Figure 8. Modern simulation technologies carry out diagnostics, decide the optimal maintenance strategies, and gather data to improve the next-generation design. For example, a lack of appropriate FEM simulation analysis during the design of a CNC machine tool may lead to the failure of the machine under vibration. Alternatively, adding more material to boost strength and lessen vibration raises the price. However, simulating the structure analysis on FEA/FEM in ANSYS and taking into account the performance and strength will meet the design requirements of the CNC machine tools [84].

6.3. Human-in-Loop as a Human–Machine Interface

In addition to performing autonomous tasks, CPS also provides HIL with physical and cognitive support. This requires a clear bidirectional information flow in form of interactive human–machine interfaces (HMI). Automatic speech recognition, gesture recognition, and augmented reality are the three key elements of HMI in I4.0 [85]. The latter, commonly referred to as virtual reality, is a vital component of the forthcoming I5.0 and digital twin technology. In the augmented age of I5.0, tools such as VR headsets will be used to augment skills and to make processes simpler and more repeatable. A GUI made up of an enhanced reality visualisation environment, a setup panel, and a gesture detection system are combined to create an HMI [46]. The operator gestures are decoded and transmitted by the gesture recognition system, by specifying the control unit’s input signals. Two frameworks are used to build the recognition system. The former is an optical tracking sensor based on stereo vision and the latter uses a step counter, step detector, accelerometer, gyroscope, linear acceleration sensor, and rotation vector included in an android smartphone. The enhanced reality environment then offers the images captured by the cameras along with a simulation environment to observe any location at random and map gestures in the workspace. This visual input from the HIL enables more effective control of the robot because self-collisions or singular configurations can be clearly anticipated [46]. In other studies, the usage of an evolutionary multi-objective and interactive scheduling framework that incorporates the human’s context-aware preferences in the optimisation process is used to implement HMI at the decisional level [12]. A Flexible Job Shop NSGA2 scheduling scheme that represents a number of Pareto optimal scheduling alternatives is provided to the HMI, and the human operator then selects the preferred choice. This helps in equipping the CPS with decision-making capabilities [12,86].

Zolotová et al. [87] present a series of case studies highlighting the potential applications of intelligent solutions using HIL and describes the changing roles of operators in production systems.

7. Contextualisation

‘Context’ can be described as any information which can be used to characterise an entity, state, or setting. Most current CPSs involving feedback control loops still consider human users as an external factor. However, the incorporation of human operators is affected by a number of factors including the task at hand, the cognitive capabilities required from the operator, and just as importantly the underlining the contextual information that governs performance management [28]. This information will affect input, output and state estimation within a CPS. The efficiency of any HILCPS depends on a good comprehension of the situation for the target problem or environment. In smart factories, IoT-enabled machines or equipment create continuous data streams. Memory can be a limited resource. To efficiently manage data throughput, curation and extraction of relevant contextual information is required. Domain-specific context varies with respect to a target application. For example, in a computer vision use case, relevant objects of interest need to be established, training data collected and suitable validation test data needs to be curated, including relevant false negative and false positive test vectors. Screening procedures for irrelevant information and suitable techniques for contextual reasoning and categorisation need to be applied as a set of training rules in any use case. HIL approaches in this space need to provide sufficient flexibility so that ML with appropriate contextual reasoning and the design of relevant control laws are implemented safely [88]. This has led to complex systems that involve integrated vision, gesture recognition, and object detection as well as (ideally) natural language-based dialogue structures that build up a complete multimodal picture that is interactive and sufficiently rich to guarantee minimum useful levels of performance [89].

The importance of contextual factors in setting appropriate trigger levels within AI models has been identified in [90]. The contextual changes can refer to the dynamics of particular tasks or the collaborative interactions between different sub-systems which can require rigorous HIL metrics to be managed for safe cooperation. In the case of user interface design in autonomous system operation, identification of contextual cues has been identified as significant in [91]. Here, dynamic system adaptation has been demonstrated through the use of predictive sensor measurements. The use of ML to predict these measurements in real time is the next I5.0 research question to be addressed in this space. The use of HILCPS in such a fashion can also help to increase understanding of how design choices are determined. In Cummings and Li [92], the causes of human bias in ML modelling have been taken into account. Consideration has been given to the issue of subjectivity in model and parameter selection as well as sample selection bias resulting from the data. Many open questions remain in relation to the general scoping of the data required for training purposes in such applications. How much and what type of data is necessary in order to reliably deploy ML-based HIL models that are bias-free but also well-tuned to the task at hand?

Instead of taking a comprehensive approach to helping intelligent systems achieve context awareness in the real world, much context-based research has focused on fixing particular challenges [93]. Building on the physical infrastructure already provided by universal computing, which takes the form of smartphones and sensors fitted with intelligent devices, this additional intelligence offers the required functionality to make real-time data processing and knowledge presentation to the user. The next significant advancement toward the implementation of ubiquitous computing paradigms in a CPS is context awareness of the curation/fusion difficulties that emerge when integrating such a vast network of devices [94]. Several authors in the larger IoT literature have thought about the transition to an ‘intelligent age’ of autonomous sensing with the suitable fusion of contextual inputs inside the immediate operational environment of the human user [93,94,95,96]. The use of context-sensitive IoT has seen the introduction of the term ‘Social IoT’ [97], where a suite of IoT devices can be viewed as acting like a combined social network to ensure optimal performance of the task at hand. To establish the necessary context from sensor data, recommender systems have been introduced that adapt in real-time to extract context information using a range of devices including smartphones and environmental sensors. By processing data over time, deriving meaningful content descriptions can become possible [98]. The next step in a move to an I5.0 setting will be an exploration of the automatic detection, flagging, and self-healing properties of HILCPS.

To demonstrate context awareness in smart manufacturing, Alexopoulos et al. [99] present a case study of a white goods producer. A feature-rich context-aware model is used to display operational limitations such as dynamic monitoring of material handling, production planning, real-time status updates, shop floor product assembly support, and shop floor notifications. At the enterprise level, the value of context information in decision support is also taken into account. The need for computational context awareness for HIL is receiving increasing prominence in the literature. An intelligent system has been designed to assist engineers with context awareness on aerospace applications [100]. The employees document their knowledge and observations on specific tasks in a systematic manner that is appropriate for training purposes. Basic data entry tasks, best practices, and interactions with other related materials are carried out in such a system with time stamping so that ML-inspired predictive maintenance will be feasible in the future. The success of any ML model depends on tackling context awareness issues in a knowledge base that is only partially populated as soon as possible. The creation of suitable imputation rules is therefore necessary.

Emmanouilidis et al. [34] affirmed the use of linked data, resolving the context-based linkages between data and knowledge using entity-based semantic description, and defined various types of contextual information relevant to the management of industrial assets. To examine the case for exploring the linked data, an architecture is presented for its collection, using data management and different ML techniques [34]. In order to advance the development of context awareness in computational systems, research on semantics and ontology has been described in [101]. Deep learning-assisted smart process planning is shown to automate the decision-making process through the structured addition of extra computational context awareness using ML, robotic wireless sensor networks, and big data analytics within the CPS [101]. HIL already plays a huge role in embedded context monitoring using control devices. The ongoing need for general rules in relation to the deployment of ML-inspired HIL continues, in order that exceptional constraints in a feedback-based system are dealt with in a controlled fashion so that unsatisfactory decisions in intelligent processes are avoided [102].

8. Hybrid Intelligence

To build effective Industry 5.0 applications, an informed combination of human and artificial intelligence denoted as hybrid intelligence is often required [103]. It is well known that many ML algorithms exhibit bias of some form [104,105], experience difficulty with understanding context [106], offer low precision [105], and have problems in adapting or self-adjustment in different environment [103,107]. This has led to the development of ‘hybrid intelligence’ solutions being reported in the literature, a fusion of machine, sensor and human cognition that augments human intelligence rather than replacing it. During decision-making, the introduction of AI-based or algorithmic models under human control produces a variety of semi-automated or hybrid decision-making processes—where algorithmic and human–agent interactions occur in a complex manner [108]. There are many open research questions here, including how to deploy such systems in a repeatable, rigorous fashion where machines learn from human inputs but more importantly, teach humans about how to effectively train a machine. One such example has been considered in [103]. Hybrid intelligence is only effective when humans and AI can communicate effectively, such as when humans provide feedback to machines through reinforcement learning and when machines provide humans with information to help them make decisions. Examples of such effective communication are often referred to in the literature as explainable AI (XAI) applications [109]. There is still much work to be done to fully explain how the operation of ML applications where HIL is a feature. Informed HIL requires a comprehensive picture of how costs, performance, interaction and uncertainty are dealt with by the control law. Black box optimisation is insufficient if operators are to provide inputs where the signal-to-noise ratio is high. For example, the training process of an ML algorithm can be significantly improved by incorporating training data that have been curated by humans in an informed, structured fashion. Generally, AI techniques require extensive training data to produce a high-accuracy model, but this training cost can be constrictive in many manufacturing use cases. HIL can expedite the training technique so that learning takes place using fewer examples [103,107]. Effective examples of hybrid intelligence through reinforcement learning that improves the quality of decision-making are now appearing. Structured HIL that enhances the accuracy of an algorithm and makes the ML-based outputs more decipherable are considered in [110]. Assimilating human creativity and ingenuity within ML models will be the dominant challenge for HILCPS use cases. The requirement is for highly adaptive, generalised, and faster models that incorporate lower data engineering costs and are generally applicable [103]. A leaner, fairer use of ML within HIL use cases AI requires the assignation of “the right value to the knowledge producers” and weighting implied knowledge gathered from HIL in a structured repeatable fashion [111] (p. 244).

This focus on data engineering for ML within HIL has been reported in a variety of different works. In a supervised learning setting, the authors of [103] reported on how the human input required in labelling training data and associated troubleshooting yielded promising results. To generate large, inexpensive labelled datasets, crowd-sourcing has been reported, i.e., the identification of trees in ReCaptacha’s online security questions and translating websites in language learning applications such as Duolingo [112]. Such an approach allows for distributed and collective human intelligence to feed in an ML model [113].

In unsupervised learning examples, human inputs have been used to understand the clustering of particular events that are useful for increasing training efficiency [103].

In ’semi-supervised’ learning, human inputs have been used to cross-check predictions and for quality assurance [114]. Reinforcement learning is a logical platform to test performance limitations using HIL, to effectively regulate ML behaviour, and to incorporate soft rules (for example ethical values), in [115]. Apart from weighting the contribution of any agent or sensor input, HIL can be used as a hybrid reinforcement/active learning use case where agents poll human experts asynchronously to dynamically select control policies [113]. Figure 9 depicts the collaborative learning process with HIL and XAI in hybrid intelligence.

8.1. Bottleneck Detection Integrating Human-in-Loop

In the production floor workload optimisation literature, disruption events in certain machines that affect the entire system workflow are referred to as ‘throughput bottlenecks’ [116]. To achieve optimal workflow throughput, identification of the throughput bottlenecks relies heavily on domain expert knowledge [117]. The impact of HIL on performance, where domain expertise is a factor is not only time-consuming but also error-prone in the absence of well-defined multiagent models. In the literature, a number of statistical model reference strategies have been suggested [118,119]. It takes time and effort to apply any statistical technique to ensure that the real-world data fits the model. This requires an HIL control law to be fully informed by different statistical techniques so as to validate the data and derive meaningful conclusions. Additionally, as the dynamics of production systems evolve with time, time-varying data from machines may also change patterns. Incorporating domain knowledge or the expertise of HIL would result in generating new hypotheses on real-life production systems over time, something not entirely captured by real-world data. Data scientists can create highly relevant ML-based solutions that identify the current limitations at any sampling interval with the use of operations research results [23]. Unsupervised machine learning with hierarchical clustering is presented in [117] to identify throughput bottlenecks and to design recommender systems by blending action-log data on bottlenecks. In Havur et al. [120], a semi-automated, graph-based model of manufacturing process design is presented, that supports the identification of bottlenecks in an HIL setting.

9. Security Aspects of Human-in-Loop

Many industries are now integrating security aspects into their physical production process models. Reports on the integration of such secure operational technology (OT) within industrial control and information technology (IT) systems are appearing in the literature. Digital technologies and cognitive computing are shifting the traditional boundaries of such OT environments. The penetrable boundary between IT and OT systems suggests new attack windows for malign actors. While IoT devices unlock potential in the physical world, they also expose organisations to an increasing cyber threat. The same vulnerabilities that other networks have with regard to exploitation, malware, denial of service (DoS), device hacking, and other typical attack techniques can be seen in smart factories [121]. The work presented by Maggi et al. [121] illustrates several unusual potential attack vectors that can affect I4.0 systems. High-profile attacks on industrial systems or other critical infrastructure are becoming a more regular occurrence and now have significant practical relevance. Some examples of past industrial cyberattacks are:

2010: The STUXNET Worm targeted the PLC systems at an Iranian nuclear facility. Stuxnet was one of the first famously known examples of advanced persistent threat (APT) malware. It was a worm which searched for the STEP 7 software of Siemens PLCs. These PLCs were used in centrifuges at Iranian nuclear facilities [122,123]. If critical and sensitive systems have a deception architecture, and human operators would analyse the data collected by the deception architecture, then attacks by the likes of Stuxnet can be prevented. Baksi and Upadhyaya [124] proposed a deception architecture which collects data in a system, and the data are analysed by the system administrator who acted as HIL to determine the depth of any penetration by APTs.

2012: The Aramco oil facility in Saudi Arabia saw 30,000 computers infected in a targeted attack on their network [125]. To combat such situations, Baksi and Upadhyaya [126] put forward deception architecture for networked systems. They showed that if one of the computers in a networked system is infected, then the information can be shared with other nodes in the network through a covert channel which remains outside the purview of the attacker. There remains significant utility in HIL applications, whereupon the receipt of new information in relation to attack vector defence actions that minimise the impact on manufacturing operations, while preserving the integrity of the process from a certification perspective can be tailored to a particular type of malware attack.

2019: Well-known industrial companies including Siemens, BASF, and Henkel declared that they had fallen victim to a state-sponsored hacking effort.

2021: An American company called Colonial Pipeline was the target of a ransomware attack, cutting off a significant part of the oil supply to the US east coast.

Colonial Pipeline came under an APT-type ransomware attack in 2021. The attack was carried out by the APT group DarkSide [123]. Such ransomware attacks created systems at Colonial Pipeline to go down for a considerable amount of time. It also created huge fuel shortages in the east coast of the United States, flight delays at the airports, and caused loss of businesses [127]. The company ended up paying USD 4.4 million in ransom payment [128]. Such ransomware attacks leave the victim confused regarding the course of action to be taken. Incorporating HIL in such situations to take a decision to either pay the ransom or not pay the ransom, based on minimising company losses may prove to be very effective. Game theory models may be used to make an informed decision [129].

According to IBM’s “2020 Cost of a Data Breach Report”, the average overall cost of a data breach is estimated to be EUR 3.86 million. The report also states that it takes 280 days on average to discover and contain a data breach [130]. According to Wu et al. (2019) [131], manufacturing is the second-most attacked industry; however, the industry’s security is lacking. Industrial environments are going to have to rethink their approach to cyber defence, particularly as the type of threat becomes ever more sophisticated.

The applications of ML in cybersecurity are vast because it constantly learns by analysing data to uncover patterns to more effectively detect threats in encrypted traffic, reports insider threats, anticipates when ‘bad neighbours’ will be online to keep users safe while browsing, and secures data in the cloud by identifying suspicious node behaviour. Intrusion detection is one of the main application areas for ML in industrial cybersecurity applications, with a significant body of research now emerging [11,132]. Organisations are forced by the cyber threat landscape to continuously track and correlate millions of internal and external data points across their users’ infrastructure. Automating the analysis of cyber attacks using ML and implementing HIL would result in rapid threat detection and analysis. Therefore, it is critical to comprehend the role of HIL while designing any security-conscious CPS, considering the human engagement with the system and its effects on security assurances. There are, however, very few studies in the body of knowledge on human–machine interface design that take into account how human situational awareness affects a system’s performance in terms of cyber–physical security and real-time cyberattack protection. In Elfar et al. [133], an extendable virtual platform is presented that examines how HIL affects CPS security at various levels of autonomy. In Le et al. [134], the authors investigate the visual characterisations of multivariate time series and real-time predictive analytics to highlight potential faults, threats, and malicious attacks.

After analysing the literature for a ‘human-in-loop’ model to detect cyber threats, we propose a standard architecture adopting the standard outlined in Figure 10 to successfully incorporate HIL in the detection and mitigation of cybersecurity attacks on manufacturing operations.

• Cyber attacks are expected to be a recurring problem moving forward and, therefore, the formulation of a mitigation strategy is essential in normal operation. Once a cyber attack is detected, an AI-informed intrusion detection system (IDS) analyses the threat, collects information regarding the attack, and sends the information to the HIL.

• The human operator analyses the information and the output of the AI-based IDS. A decision regarding the plan of action is formulated and a defence strategy is executed by the HIL.

• The defence action taken with HIL optimises the mitigation effort associated with a particular attack.

• Multiple attacks may transpire during any mitigation. The early detection of such attacks and a measurement of the benefits that are derived from incorporating HIL must be recorded in order to help the community of practice engage in what-if thinking in relation to improved resilience to attacks.

Figure 10

Human-in-loop in the detection of cyber attacks.

[Figure omitted. See PDF]

10. Discussion and Conclusions

By establishing a digital network between industrial equipment, smart manufacturing seeks to increase system productivity and product quality while lowering costs. Even though conceptual modelling and engineering principles have received more attention, the majority of work is focused on automation systems, data flow control, systems integration, and optimisation technologies. In order to better align and integrate human skills in the realm of smart manufacturing, this research advances by incorporating ‘human-in-loop’ design elements intelligently and fully, mainly in the following three aspects.

• Intellect aspect: Humans serve as mentors who possess the information that AI models can learn from.

• Interaction aspect: Humans perform collaborative and supervisory responsibilities to guarantee effective engagement.

• Interface aspect: Humans assist in the data gathering process through some interfaces in addition to acting as data requesters.

The majority of the necessary techniques need advanced analytics tools in the backend; therefore, the collaboration between AI and the ‘human-in-loop’ is both an opportunity and a challenge to improve flexibility.

This review article analysed over 130 research papers selected after a thorough exploration of the literature. This paper has surveyed existing research that investigates the present and future need for the importance of the HIL in different aspects of smart manufacturing including its security. It is determined that for the foreseeable future, human decision makers will continue to play a crucial role in many industries for more complicated automation tasks requiring the mass manufacture of highly customised products. As a consequence, the assessment of performance for the ‘human-in-loop’ will remain a crucial element of numerous CPS applications. In this paper, the state of the art with regard to the use of cutting-edge ML approaches is surveyed, especially in contexts where the human operator acts as an active collaborator within a CPS. Given the socio-technical character of the smart factory, it is crucial to accurately describe how humans interact with all of the smart devices. This paper has outlined a list of research questions that remain to be addressed so that Industry 5.0 becomes a reality. In particular, the efficient curation of training data so that the role of human operators can be captured efficiently and the seamless integration of ML within a loop that involves a human operator remain significant questions to be addressed by the research community. The success of I5.0 use cases requires that rigorous metrics must be introduced and, moreover, gain general acceptance so that the performance of HIL can be fairly assessed. This is still an open question in many HMI use cases. It is only through rigorous measurement of performance that the value added by HIL in manufacturing can be properly assessed. Therefore, for future work, creating a unified evaluation framework to specify and quantify important performance indicators would assist in clarifying the benefits and difficulties of HIL.

Author Contributions

Conceptualisation: M.B., E.O. and M.H.; Writing—original draft preparation, M.B.; Writing—review and editing, M.B., M.P., E.O. and M.H.; Funding acquisition, M.S. and M.H. All authors have read and agreed to the published version of the manuscript.

Data Availability Statement

Data sharing is not applicable to this article as no new data was generated or analysed in this work.

Conflicts of Interest

The authors declare no conflict of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Table

Generate image description

Figure 1. Structure of this paper.

Generate image description

Figure 2. Approach followed to complete the literature search and review.

Generate image description

Figure 3. Industry 4.0, 5.0, and X.

Generate image description

Figure 4. Human–machine collaboration in Industry X.

Generate image description

Figure 5. Human-in-loop cyber–physical system (HILCPS) architecture.

Generate image description

Figure 6. Human-in-loop in visual analytics.

Generate image description

Figure 7. Three-dimensional visualisation for operator assistance.

Generate image description

Figure 8. Tools for data management in a digital twin.

Generate image description

Figure 9. Human-in-loop in hybrid intelligence.

Table 1

Structured literature review: search terminologies and papers.

Terminologies	Highest Publication Year	Publications in 2021	Aggregated
Industry 5.0	2021	11,016	9444
Cyber-physical systems	2019	3565	25,219
Data visualisation & human in loop	2021	46	368
Digital Twin	2021	2,119	7519
Human-Machine Interface	2021	1658	17,381
Contextualisation	2021	811	6600
Hybrid Intelligence	2021	3343	27,146
Bottleneck detection in smart manufacturing	2022	3	15
Security aspects of Human in loop	2019	5	41

References

1. Kusiak, A. Smart manufacturing. Int. J. Prod. Res.; 2018; 56, pp. 508-517. [DOI: https://dx.doi.org/10.1080/00207543.2017.1351644]

2. Esmaeilian, B.; Behdad, S.; Wang, B. The evolution and future of manufacturing: A review. J. Manuf. Syst.; 2016; 39, pp. 79-100. [DOI: https://dx.doi.org/10.1016/j.jmsy.2016.03.001]

3. Guerin, C.; Rauffet, P.; Chauvin, C.; Martin, E. Toward production operator 4.0: Modelling human-machine cooperation in industry 4.0 with cognitive work analysis. IFAC-PapersOnLine; 2019; 52, pp. 73-78. [DOI: https://dx.doi.org/10.1016/j.ifacol.2019.12.111]

4. Longo, F.; Padovano, A.; Umbrello, S. Value-oriented and ethical technology engineering in industry 5.0: A human-centric perspective for the design of the factory of the future. Appl. Sci.; 2020; 10, 4182. [DOI: https://dx.doi.org/10.3390/app10124182]

5. Turner, C.; Oyekan, J.; Garn, W.; Duggan, C.; Abdou, K. Industry 5.0 and the Circular Economy: Utilizing LCA with Intelligent Products. Sustainability; 2022; 14, 14847. [DOI: https://dx.doi.org/10.3390/su142214847]

6. Mittal, S.; Khan, M.A.; Romero, D.; Wuest, T. Smart manufacturing: Characteristics, technologies and enabling factors. Proc. Inst. Mech. Eng. Part B J. Eng. Manuf.; 2019; 233, pp. 1342-1361. [DOI: https://dx.doi.org/10.1177/0954405417736547]

7. Nunes, D.S.; Zhang, P.; Silva, J.S. A survey on human-in-the-loop applications towards an internet of all. IEEE Commun. Surv. Tutor.; 2015; 17, pp. 944-965. [DOI: https://dx.doi.org/10.1109/COMST.2015.2398816]

8. Schaeffer, E. Industry X. 0: Realizing Digital Value in Industrial Sectors; Kogan Page Publishers: London, UK, 2017.

9. Cheng, Y.J.; Chen, M.H.; Cheng, F.C.; Cheng, Y.C.; Lin, Y.S.; Yang, C.J. Developing a decision support system (DSS) for a dental manufacturing production line based on data mining. Appl. Syst. Innov.; 2018; 1, 17. [DOI: https://dx.doi.org/10.3390/asi1020017]

10. Priore, P.; Ponte, B.; Puente, J.; Gómez, A. Learning-based scheduling of flexible manufacturing systems using ensemble methods. Comput. Ind. Eng.; 2018; 126, pp. 282-291. [DOI: https://dx.doi.org/10.1016/j.cie.2018.09.034]

11. Xin, Y.; Kong, L.; Liu, Z.; Chen, Y.; Li, Y.; Zhu, H.; Gao, M.; Hou, H.; Wang, C. Machine learning and deep learning methods for cybersecurity. IEEE Access; 2018; 6, pp. 35365-35381. [DOI: https://dx.doi.org/10.1109/ACCESS.2018.2836950]

12. Gaham, M.; Bouzouia, B.; Achour, N. Human-in-the-loop cyber-physical production systems control (hilcp 2 sc): A multi-objective interactive framework proposal. Service Orientation in Holonic and Multi-Agent Manufacturing; Springer: Berlin/Heidelberg, Germany, 2015; pp. 315-325.

13. Kujawińska, A.; Rogalewicz, M.; Muchowski, M.; Stańkowska, M. Application of cluster analysis in making decision about purchase of additional materials for welding process. Smart Technology; Springer: Berlin/Heidelberg, Germany, 2018; pp. 10-20.

14. Gandhi, K.; Schmidt, B.; Ng, A.H. Towards data mining based decision support in manufacturing maintenance. Procedia CIRP; 2018; 72, pp. 261-265. [DOI: https://dx.doi.org/10.1016/j.procir.2018.03.076]

15. Zhang, Y.; Ren, S.; Liu, Y.; Si, S. A big data analytics architecture for cleaner manufacturing and maintenance processes of complex products. J. Clean. Prod.; 2017; 142, pp. 626-641. [DOI: https://dx.doi.org/10.1016/j.jclepro.2016.07.123]

16. Antunes, R.A.; Palma, L.B. Fitts’ evaluation of a developed human-in-the-loop assistive device. Proceedings of the 2020 IEEE International Symposium on Medical Measurements and Applications (MeMeA); Bari, Italy, 1 June–1 July 2020; pp. 1-6.

17. Qian, B.; Wang, X.; Cao, N.; Jiang, Y.G.; Davidson, I. Learning multiple relative attributes with humans in the loop. IEEE Trans. Image Process.; 2014; 23, pp. 5573-5585. [DOI: https://dx.doi.org/10.1109/TIP.2014.2365952]

18. Yucelen, T.; Yildiz, Y.; Sipahi, R.; Yousefi, E.; Nguyen, N. Stability limit of human-in-the-loop model reference adaptive control architectures. Int. J. Control; 2018; 91, pp. 2314-2331. [DOI: https://dx.doi.org/10.1080/00207179.2017.1342274]

19. Dogan, A.; Birant, D. Machine learning and data mining in manufacturing. Expert Syst. Appl.; 2021; 166, 114060. [DOI: https://dx.doi.org/10.1016/j.eswa.2020.114060]

20. Longo, F.; Nicoletti, L.; Padovano, A. Smart operators in industry 4.0: A human-centered approach to enhance operators’ capabilities and competencies within the new smart factory context. Comput. Ind. Eng.; 2017; 113, pp. 144-159. [DOI: https://dx.doi.org/10.1016/j.cie.2017.09.016]

21. Peruzzini, M.; Grandi, F.; Pellicciari, M. Exploring the potential of Operator 4.0 interface and monitoring. Comput. Ind. Eng.; 2020; 139, 105600. [DOI: https://dx.doi.org/10.1016/j.cie.2018.12.047]

22. Moghaddam, M.; Cadavid, M.N.; Kenley, C.R.; Deshmukh, A.V. Reference architectures for smart manufacturing: A critical review. J. Manuf. Syst.; 2018; 49, pp. 215-225. [DOI: https://dx.doi.org/10.1016/j.jmsy.2018.10.006]

23. Cimini, C.; Pirola, F.; Pinto, R.; Cavalieri, S. A human-in-the-loop manufacturing control architecture for the next generation of production systems. J. Manuf. Syst.; 2020; 54, pp. 258-271. [DOI: https://dx.doi.org/10.1016/j.jmsy.2020.01.002]

24. Jones, A.T.; Romero, D.; Wuest, T. Modeling agents as joint cognitive systems in smart manufacturing systems. Manuf. Lett.; 2018; 17, pp. 6-8. [DOI: https://dx.doi.org/10.1016/j.mfglet.2018.06.002]

25. Miller, C.A.; Parasuraman, R. Beyond levels of automation: An architecture for more flexible human-automation collaboration. Proceedings of the Human Factors and Ergonomics Society Annual Meeting; SAGE Publications: Los Angeles, CA, USA, 2003; Volume 47, pp. 182-186.

26. Fasth-Berglund, Å.; Stahre, J. Cognitive automation strategy for reconfigurable and sustainable assembly systems. Assem. Autom.; 2013; 33, pp. 294-303. [DOI: https://dx.doi.org/10.1108/AA-12-2013-036]

27. Hoc, J.M. Towards a cognitive approach to human–machine cooperation in dynamic situations. Int. J. -Hum.-Comput. Stud.; 2001; 54, pp. 509-540. [DOI: https://dx.doi.org/10.1006/ijhc.2000.0454]

28. Emmanouilidis, C.; Pistofidis, P.; Bertoncelj, L.; Katsouros, V.; Fournaris, A.; Koulamas, C.; Ruiz-Carcel, C. Enabling the human in the loop: Linked data and knowledge in industrial cyber-physical systems. Annu. Rev. Control; 2019; 47, pp. 249-265. [DOI: https://dx.doi.org/10.1016/j.arcontrol.2019.03.004]

29. Tukey, J.W. The future of data analysis. Ann. Math. Stat.; 1962; 33, pp. 1-67. [DOI: https://dx.doi.org/10.1214/aoms/1177704711]

30. Tukey, J.W. Exploratory Data Analysis; Sage: Reading, MA, USA, 1977; Volume 2.

31. Idreos, S.; Papaemmanouil, O.; Chaudhuri, S. Overview of data exploration techniques. Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data; Melbourne, Australia, 31 May–4 June 2015; pp. 277-281.

32. Cook, K.A.; Thomas, J.J. Illuminating the Path: The Research and Development Agenda for Visual Analytics; Technical Report Pacific Northwest National Lab. (PNNL): Richland, WA, USA, 2005.

33. Endert, A.; Hossain, M.S.; Ramakrishnan, N.; North, C.; Fiaux, P.; Andrews, C. The human is the loop: New directions for visual analytics. J. Intell. Inf. Syst.; 2014; 43, pp. 411-435. [DOI: https://dx.doi.org/10.1007/s10844-014-0304-9]

34. Emmanouilidis, C.; Pistofidis, P.; Fournaris, A.; Bevilacqua, M.; Durazo-Cardenas, I.; Botsaris, P.N.; Katsouros, V.; Koulamas, C.; Starr, A.G. Context-based and human-centred information fusion in diagnostics. IFAC-PapersOnLine; 2016; 49, pp. 220-225. [DOI: https://dx.doi.org/10.1016/j.ifacol.2016.11.038]

35. Breque, M.; Nul, L.D.; Petridis, A. Industry 5.0, Towards a Sustainable, Human-Centric and Resilient European Industry; Publications Office of European Union: Luxembourg, 2021.

36. Trist, E.L. The Evolution of Socio-Technical Systems; Ontario Quality of Working Life Centre Toronto: Toronto, ON, Canada, 1981; Volume 2.

37. Schmidt, R.; Möhring, M.; Härting, R.C.; Reichstein, C.; Neumaier, P.; Jozinović, P. Industry 4.0-potentials for creating smart products: Empirical research results. Proceedings of the International Conference on Business Information Systems; Poznan, Poland, 24–26 June 2015; Springer: Berlin/Heidelberg, Germany, 2015; pp. 16-27.

38. Glas, A.H.; Kleemann, F.C. The impact of industry 4.0 on procurement and supply management: A conceptual and qualitative analysis. Int. J. Bus. Manag. Invent.; 2016; 5, pp. 55-66.

39. Radziwon, A.; Bilberg, A.; Bogers, M.; Madsen, E.S. The smart factory: Exploring adaptive and flexible manufacturing solutions. Procedia Eng.; 2014; 69, pp. 1184-1190. [DOI: https://dx.doi.org/10.1016/j.proeng.2014.03.108]

40. Llena-Nozal, A.; Martin, N.; Murtin, F. The Economy of Well-Being: Creating Opportunities for People’s Well-Being and Economic Growth; Organization for Economic Cooperation and Development (OECD): Paris, France, 2019.

41. Assembly, G. Sustainable development goals. SDGs Transform. Our World; 2015; 2030, 2015.

42. Romero, D.; Bernus, P.; Noran, O.; Stahre, J.; Fast-Berglund, Å. The operator 4.0: Human cyber-physical systems & adaptive automation towards human-automation symbiosis work systems. Proceedings of the IFIP International Conference on Advances in Production Management Systems; Iguassu Falls, Brazil, 3–6 September 2016; Springer: Berlin/Heidelberg, Germany, 2016; pp. 677-686.

43. Salaken, S.M.; Nahavandi, S.; McGinn, C.; Hossny, M.; Kelly, K.; Abobakr, A.; Nahavandi, D.; Iskander, J. Development of a cloud-based computational framework for an empathetic robot. Proceedings of the 2019 11th International Conference on Computer and Automation Engineering; Perth, Australia, 23–25 February 2019; pp. 102-108.

44. Lu, Y.; Zheng, H.; Chand, S.; Xia, W.; Liu, Z.; Xu, X.; Wang, L.; Qin, Z.; Bao, J. Outlook on human-centric manufacturing towards Industry 5.0. J. Manuf. Syst.; 2022; 62, pp. 612-627. [DOI: https://dx.doi.org/10.1016/j.jmsy.2022.02.001]

45. Javaid, M.; Haleem, A. Critical components of Industry 5.0 towards a successful adoption in the field of manufacturing. J. Ind. Integr. Manag.; 2020; 5, pp. 327-348. [DOI: https://dx.doi.org/10.1142/S2424862220500141]

46. Garcia, M.A.R.; Rojas, R.; Gualtieri, L.; Rauch, E.; Matt, D. A human-in-the-loop cyber-physical system for collaborative assembly in smart manufacturing. Procedia CIRP; 2019; 81, pp. 600-605. [DOI: https://dx.doi.org/10.1016/j.procir.2019.03.162]

47. Trentesaux, D.; Moray, N.; Tahon, C. Integration of the human operator into responsive discrete production management systems. Eur. J. Oper. Res.; 1998; 109, pp. 342-361. [DOI: https://dx.doi.org/10.1016/S0377-2217(98)00062-9]

48. Wang, L.; Haghighi, A. Combined strength of holons, agents and function blocks in cyber-physical systems. J. Manuf. Syst.; 2016; 40, pp. 25-34. [DOI: https://dx.doi.org/10.1016/j.jmsy.2016.05.002]

49. Bhattacharya, M.; Mohandas, R.; Penica, M.; Southern, M.; Van Camp, K.; Hayes, M.J. Analysis of the Message Queueing Telemetry Transport Protocol for Data Labelling: An Orthopedic Manufacturing Process Case Study. Proceedings of the IoTBDS; Online, 23–25 April 2021; pp. 215-222.

50. Wang, S.; Wan, J.; Li, D.; Zhang, C. Implementing Smart Factory of Industrie 4.0: An Outlook. Int. J. Distrib. Sens. Netw.; 2016; 12, 3159805. [DOI: https://dx.doi.org/10.1155/2016/3159805]

51. Fantini, P.; Tavola, G.; Taisch, M.; Barbosa, J.; Leitão, P.; Liu, Y.; Sayed, M.S.; Lohse, N. Exploring the integration of the human as a flexibility factor in CPS enabled manufacturing environments: Methodology and results. Proceedings of the IECON 2016-42nd Annual Conference of the IEEE Industrial Electronics Society; Florence, Italy, 23–26 October 2016; pp. 5711-5716.

52. Qiu, R.; Wysk, R.; Xu, Q. Extended structured adaptive supervisory control model of shop floor controls for an e-Manufacturing system. Int. J. Prod. Res.; 2003; 41, pp. 1605-1620. [DOI: https://dx.doi.org/10.1080/1352816031000074964]

53. Jugulum, R. Importance of data quality for analytics. Quality in the 21st Century; Springer: Berlin/Heidelberg, Germany, 2016; pp. 23-31.

54. Negri, E.; Fumagalli, L.; Macchi, M. A review of the roles of digital twin in CPS-based production systems. Procedia Manuf.; 2017; 11, pp. 939-948. [DOI: https://dx.doi.org/10.1016/j.promfg.2017.07.198]

55. Gölzer, P.; Fritzsche, A. Data-driven operations management: Organisational implications of the digital transformation in industrial practice. Prod. Plan. Control; 2017; 28, pp. 1332-1343. [DOI: https://dx.doi.org/10.1080/09537287.2017.1375148]

56. Li, K.; Tiwari, A.; Alcock, J.; Bermell-Garcia, P. Categorisation of visualisation methods to support the design of Human-Computer Interaction Systems. Appl. Ergon.; 2016; 55, pp. 85-107. [DOI: https://dx.doi.org/10.1016/j.apergo.2016.01.009]

57. Tran, P.V.; Le, T.X. Approaching human vision perception to designing visual graph in data visualization. Concurr. Comput. Pract. Exp.; 2021; 33, e5722. [DOI: https://dx.doi.org/10.1002/cpe.5722]

58. Zhou, F.; Lin, X.; Liu, C.; Zhao, Y.; Xu, P.; Ren, L.; Xue, T.; Ren, L. A survey of visualization for smart manufacturing. J. Vis.; 2019; 22, pp. 419-435. [DOI: https://dx.doi.org/10.1007/s12650-018-0530-2]

59. Lade, P.; Ghosh, R.; Srinivasan, S. Manufacturing analytics and industrial internet of things. IEEE Intell. Syst.; 2017; 32, pp. 74-79. [DOI: https://dx.doi.org/10.1109/MIS.2017.49]

60. Golfarelli, M.; Rizzi, S. A model-driven approach to automate data visualization in big data analytics. Inf. Vis.; 2020; 19, pp. 24-47. [DOI: https://dx.doi.org/10.1177/1473871619858933]

61. Vosough, Z.; Kammer, D.; Keck, M.; Groh, R. Visualization approaches for understanding uncertainty in flow diagrams. J. Comput. Lang.; 2019; 52, pp. 44-54. [DOI: https://dx.doi.org/10.1016/j.cola.2019.03.002]

62. Luo, Y.; Qin, X.; Chai, C.; Tang, N.; Li, G.; Li, W. Steerable self-driving data visualization. IEEE Trans. Knowl. Data Eng.; 2020; 34, pp. 475-490. [DOI: https://dx.doi.org/10.1109/TKDE.2020.2981464]

63. Zou, Y.; Lü, J.; Tao, X. Research on context of implicit context-sensitive graph grammars. J. Comput. Lang.; 2019; 51, pp. 241-260. [DOI: https://dx.doi.org/10.1016/j.cola.2019.01.002]

64. Lensen, A.; Xue, B.; Zhang, M. Genetic programming for evolving a front of interpretable models for data visualization. IEEE Trans. Cybern.; 2020; 51, pp. 5468-5482. [DOI: https://dx.doi.org/10.1109/TCYB.2020.2970198]

65. Silva, I.C.S.; Santucci, G.; Freitas, C.M.D.S. Visualization and analysis of schema and instances of ontologies for improving user tasks and knowledge discovery. J. Comput. Lang.; 2019; 51, pp. 28-47. [DOI: https://dx.doi.org/10.1016/j.cola.2019.01.004]

66. Fuller, A.; Fan, Z.; Day, C. Digital twin: Enabling technology, challenges and open research. arXiv; 2019; arXiv: 1911.01276[DOI: https://dx.doi.org/10.1109/ACCESS.2020.2998358]

67. Lu, Y.; Liu, C.; Kevin, I.; Wang, K.; Huang, H.; Xu, X. Digital Twin-driven smart manufacturing: Connotation, reference model, applications and research issues. Robot. -Comput.-Integr. Manuf.; 2020; 61, 101837. [DOI: https://dx.doi.org/10.1016/j.rcim.2019.101837]

68. Turner, C.J.; Hutabarat, W.; Oyekan, J.; Tiwari, A. Discrete event simulation and virtual reality use in industry: New opportunities and future trends. IEEE Trans. -Hum.-Mach. Syst.; 2016; 46, pp. 882-894. [DOI: https://dx.doi.org/10.1109/THMS.2016.2596099]

69. Hutabarat, W.; Oyekan, J.; Turner, C.; Tiwari, A.; Prajapat, N.; Gan, X.P.; Waller, A. Combining virtual reality enabled simulation with 3D scanning technologies towards smart manufacturing. Proceedings of the 2016 Winter Simulation Conference (WSC); Washington, DC, USA, 11–14 December 2016; pp. 2774-2785.

70. Zhong, R.Y.; Lan, S.; Xu, C.; Dai, Q.; Huang, G.Q. Visualization of RFID-enabled shopfloor logistics Big Data in Cloud Manufacturing. Int. J. Adv. Manuf. Technol.; 2016; 84, pp. 5-16. [DOI: https://dx.doi.org/10.1007/s00170-015-7702-1]

71. Cimino, C.; Negri, E.; Fumagalli, L. Review of digital twin applications in manufacturing. Comput. Ind.; 2019; 113, 103130. [DOI: https://dx.doi.org/10.1016/j.compind.2019.103130]

72. Haag, S.; Anderl, R. Digital twin–Proof of concept. Manuf. Lett.; 2018; 15, pp. 64-66. [DOI: https://dx.doi.org/10.1016/j.mfglet.2018.02.006]

73. Tao, F.; Cheng, J.; Qi, Q.; Zhang, M.; Zhang, H.; Sui, F. Digital twin-driven product design, manufacturing and service with big data. Int. J. Adv. Manuf. Technol.; 2018; 94, pp. 3563-3576. [DOI: https://dx.doi.org/10.1007/s00170-017-0233-1]

74. Uhlemann, T.H.J.; Lehmann, C.; Steinhilper, R. The digital twin: Realizing the cyber-physical production system for industry 4.0. Procedia CIRP; 2017; 61, pp. 335-340. [DOI: https://dx.doi.org/10.1016/j.procir.2016.11.152]

75. Qi, Q.; Tao, F.; Hu, T.; Anwer, N.; Liu, A.; Wei, Y.; Wang, L.; Nee, A. Enabling technologies and tools for digital twin. J. Manuf. Syst.; 2021; 58, pp. 3-21. [DOI: https://dx.doi.org/10.1016/j.jmsy.2019.10.001]

76. Fortino, G.; Guerrieri, A.; Pace, P.; Savaglio, C.; Spezzano, G. IoT Platforms and Security: An Analysis of the Leading Industrial/Commercial Solutions. Sensors; 2022; 22, 2196. [DOI: https://dx.doi.org/10.3390/s22062196]

77. Karmakar, A.; Dey, N.; Baral, T.; Chowdhury, M.; Rehan, M. Industrial internet of things: A review. Proceedings of the 2019 International Conference on Opto-Electronics and Applied Optics (Optronix); Kolkata, India, 18–20 March 2019; pp. 1-6.

78. Petrik, D.; Herzwurm, G. iIoT ecosystem development through boundary resources: A Siemens MindSphere case study. Proceedings of the 2nd ACM SIGSOFT International Workshop on Software-Intensive Business: Start-Ups, Platforms, and Ecosystems; Tallinn, Estonia, 26 August 2019; pp. 1-6.

79. Jedermann, R.; Lang, W.; Geyer, M.; Mahajan, P. Digital Twin Features for the Intelligent Container. Proceedings of the International Conference on Dynamics in Logistics; Bremen, Germany, 23–25 February 2022; Springer: Berlin/Heidelberg, Germany, 2022; pp. 217-228.

80. Labani, M.; Beheshti, A.; Lovell, N.H.; Alinejad-Rokny, H.; Afrasiabi, A. KARAJ: An Efficient Adaptive Multi-Processor Tool to Streamline Genomic and Transcriptomic Sequence Data Acquisition. Int. J. Mol. Sci.; 2022; 23, 14418. [DOI: https://dx.doi.org/10.3390/ijms232214418]

81. Leskovskỳ, R.; Kučera, E.; Haffner, O.; Rosinová, D. Proposal of digital twin platform based on 3D rendering and IIoT principles using virtual/augmented reality. Proceedings of the 2020 Cybernetics & Informatics (K&I); Velke Karlovice, Czech Republic, 29 January–1 February 2020; pp. 1-8.

82. Singh, G. Baker Hughes Develops Predictive Maintenance Software for Gas and Oil Extraction Equipment Using Data Analytics and Machine Learning. 2019; Available online: https://www.mathworks.com/company/user_stories/baker-hughes-develops-predictive-maintenance-software-for-gas-and-oil-extraction-equipment-using-data-analytics-and-machine-learning.html (accessed on 30 August 2022).

83. Zhang, Z.; Wang, X.; Wang, X.; Cui, F.; Cheng, H. A simulation-based approach for plant layout design and production planning. J. Ambient. Intell. Humaniz. Comput.; 2019; 10, pp. 1217-1230. [DOI: https://dx.doi.org/10.1007/s12652-018-0687-5]

84. Gupta, A.; Kundra, T. A review of designing machine tool for leanness. Sadhana; 2012; 37, pp. 241-259. [DOI: https://dx.doi.org/10.1007/s12046-012-0062-8]

85. Gorecky, D.; Schmitt, M.; Loskyll, M.; Zühlke, D. Human-machine-interaction in the industry 4.0 era. Proceedings of the 2014 12th IEEE International Conference on Industrial Informatics (INDIN); Porto Alegre, Brazil, 27–30 July 2014; pp. 289-294.

86. Hadorn, B.; Courant, M.; Hirsbrunner, B. Towards Human-Centered Cyber-Physical Systems; Départment d’informatique Université de Fribourg: Fribourg, Switzerland, 2016.

87. Zolotová, I.; Papcun, P.; Kajáti, E.; Miškuf, M.; Mocnej, J. Smart and cognitive solutions for Operator 4.0: Laboratory H-CPPS case studies. Comput. Ind. Eng.; 2020; 139, 105471. [DOI: https://dx.doi.org/10.1016/j.cie.2018.10.032]

88. Bauckhage, C.; Hanheide, M.; Wrede, S.; Käster, T.; Pfeiffer, M.; Sagerer, G. Vision Systems with the Human in the Loop. Eurasip J. Adv. Signal Process.; 2005; 2005, 302161. [DOI: https://dx.doi.org/10.1155/ASP.2005.2375]

89. Lv, Z.; Poiesi, F.; Dong, Q.; Lloret, J.; Song, H. Deep Learning for Intelligent Human–Computer Interaction. Appl. Sci.; 2022; 12, 11457. [DOI: https://dx.doi.org/10.3390/app122211457]

90. Herrmann, T.; Pfeiffer, S. Keeping the organization in the loop: A socio-technical extension of human-centered artificial intelligence. AI Soc.; 2022; pp. 1-20. Available online: https://link.springer.com/article/10.1007/s00146-022-01391-5 (accessed on 30 August 2022). [DOI: https://dx.doi.org/10.1007/s00146-022-01391-5]

91. Cummings, M.; Stimpson, A. Identifying critical contextual design cues through a machine learning approach. AI Mag.; 2019; 40, pp. 28-39. [DOI: https://dx.doi.org/10.1609/aimag.v40i4.4811]

92. Cummings, M.L.; Li, S. Subjectivity in the creation of machine learning models. ACM J. Data Inf. Qual.; 2021; 13, pp. 1-19. [DOI: https://dx.doi.org/10.1145/3418034]

93. Alegre, U.; Augusto, J.C.; Clark, T. Engineering context-aware systems and applications: A survey. J. Syst. Softw.; 2016; 117, pp. 55-83. [DOI: https://dx.doi.org/10.1016/j.jss.2016.02.010]

94. Gross, T. Towards a new human-centred computing methodology for cooperative ambient intelligence. J. Ambient. Intell. Humaniz. Comput.; 2010; 1, pp. 31-42. [DOI: https://dx.doi.org/10.1007/s12652-009-0004-4]

95. Piccialli, F.; Chianese, A. The internet of things supporting context-aware computing: A cultural heritage case study. Mob. Netw. Appl.; 2017; 22, pp. 332-343. [DOI: https://dx.doi.org/10.1007/s11036-017-0810-4]

96. Sezer, O.B.; Dogdu, E.; Ozbayoglu, A.M. Context-aware computing, learning, and big data in internet of things: A survey. IEEE Internet Things J.; 2017; 5, pp. 1-27. [DOI: https://dx.doi.org/10.1109/JIOT.2017.2773600]

97. Gil, D.; Ferrández, A.; Mora-Mora, H.; Peral, J. Internet of things: A review of surveys based on context aware intelligent services. Sensors; 2016; 16, 1069. [DOI: https://dx.doi.org/10.3390/s16071069]

98. Unger, M.; Bar, A.; Shapira, B.; Rokach, L. Towards latent context-aware recommendation systems. Knowl.-Based Syst.; 2016; 104, pp. 165-178. [DOI: https://dx.doi.org/10.1016/j.knosys.2016.04.020]

99. Alexopoulos, K.; Makris, S.; Xanthakis, V.; Sipsas, K.; Chryssolouris, G. A concept for context-aware computing in manufacturing: The white goods case. Int. J. Comput. Integr. Manuf.; 2016; 29, pp. 839-849. [DOI: https://dx.doi.org/10.1080/0951192X.2015.1130257]

100. Belkadi, F.; Dhuieb, M.A.; Aguado, J.V.; Laroche, F.; Bernard, A.; Chinesta, F. Intelligent assistant system as a context-aware decision-making support for the workers of the future. Comput. Ind. Eng.; 2020; 139, 105732. [DOI: https://dx.doi.org/10.1016/j.cie.2019.02.046]

101. Turner, C.J.; Ma, R.; Chen, J.; Oyekan, J. Human in the Loop: Industry 4.0 technologies and scenarios for worker mediation of automated manufacturing. IEEE Access; 2021; 9, pp. 103950-103966. [DOI: https://dx.doi.org/10.1109/ACCESS.2021.3099311]

102. Hoffmann, H.; Jantsch, A.; Dutt, N.D. Embodied self-aware computing systems. Proc. IEEE; 2020; 108, pp. 1027-1046. [DOI: https://dx.doi.org/10.1109/JPROC.2020.2977054]

103. Dellermann, D.; Calma, A.; Lipusch, N.; Weber, T.; Weigel, S.; Ebel, P. The future of human-AI collaboration: A taxonomy of design knowledge for hybrid intelligence systems. arXiv; 2021; arXiv: 2105.03354

104. Barredo-Arrieta, A.; Laña, I.; Del Ser, J. What lies beneath: A note on the explainability of black-box machine learning models for road traffic forecasting. Proceedings of the 2019 IEEE Intelligent Transportation Systems Conference (ITSC); Auckland, NZ, USA, 27–29 October 2019; pp. 2232-2237.

105. Cush, A. This Program That Judges Use to Predict Future Crimes Seems Racist as Hell. Available online: https://www.gawker.com/this-program-that-judges-use-to-predict-future-crimes-s-1778151070 (accessed on 30 August 2022).

106. Thota, A.; Tilak, P.; Ahluwalia, S.; Lohia, N. Fake news detection: A deep learning approach. SMU Data Sci. Rev.; 2018; 1, 10.

107. Ostheimer, J.; Chowdhury, S.; Iqbal, S. An alliance of humans and machines for machine learning: Hybrid intelligent systems and their design principles. Technol. Soc.; 2021; 66, 101647. [DOI: https://dx.doi.org/10.1016/j.techsoc.2021.101647]

108. Morison, J.; Harkens, A. Re-engineering justice? Robot judges, computerised courts and (semi) automated legal decision-making. Leg. Stud.; 2019; 39, pp. 618-635. [DOI: https://dx.doi.org/10.1017/lst.2019.5]

109. Ahmed, I.; Jeon, G.; Piccialli, F. From artificial intelligence to explainable artificial intelligence in industry 4.0: A survey on what, how, and where. IEEE Trans. Ind. Inform.; 2022; 18, pp. 5031-5042. [DOI: https://dx.doi.org/10.1109/TII.2022.3146552]

110. Correia, A.H.; Lecue, F. Human-in-the-loop feature selection. Proceedings of the AAAI Conference on Artificial Intelligence; Honolulu, HI, USA, 27–28 January 2019; Volume 33, pp. 2438-2445.

111. Zanzotto, F.M. Human-in-the-loop artificial intelligence. J. Artif. Intell. Res.; 2019; 64, pp. 243-252. [DOI: https://dx.doi.org/10.1613/jair.1.11345]

112. Vaughan, J.W. Making Better Use of the Crowd: How Crowdsourcing Can Advance Machine Learning Research. J. Mach. Learn. Res.; 2017; 18, pp. 7026-7071.

113. Holzinger, A. Interactive machine learning for health informatics: When do we need the human-in-the-loop?. Brain Inform.; 2016; 3, pp. 119-131. [DOI: https://dx.doi.org/10.1007/s40708-016-0042-6]

114. Yimam, S.M.; Biemann, C.; Majnaric, L.; Šabanović, Š.; Holzinger, A. An adaptive annotation approach for biomedical entity and relation recognition. Brain Inform.; 2016; 3, pp. 157-168. [DOI: https://dx.doi.org/10.1007/s40708-016-0036-4]

115. Rahwan, I. Society-in-the-loop: Programming the algorithmic social contract. Ethics Inf. Technol.; 2018; 20, pp. 5-14. [DOI: https://dx.doi.org/10.1007/s10676-017-9430-8]

116. Roser, C.; Nakano, M.; Tanaka, M. A practical bottleneck detection method. Proceedings of the 2001 Winter Simulation Conference (Cat. No. 01CH37304); Arlington, VA, USA, 9–12 December 2001; Volume 2, pp. 949-953.

117. Subramaniyan, M.; Skoogh, A.; Muhammad, A.S.; Bokrantz, J.; Johansson, B.; Roser, C. A generic hierarchical clustering approach for detecting bottlenecks in manufacturing. J. Manuf. Syst.; 2020; 55, pp. 143-158. [DOI: https://dx.doi.org/10.1016/j.jmsy.2020.02.011]

118. Subramaniyan, M.; Skoogh, A.; Salomonsson, H.; Bangalore, P.; Gopalakrishnan, M.; Sheikh Muhammad, A. Data-driven algorithm for throughput bottleneck analysis of production systems. Prod. Manuf. Res.; 2018; 6, pp. 225-246. [DOI: https://dx.doi.org/10.1080/21693277.2018.1496491]

119. Yu, C.; Matta, A. Data-driven bottleneck detection in manufacturing systems: A statistical approach. Proceedings of the 2014 IEEE International Conference on Automation Science and Engineering (CASE); New Taipei, Taiwan, 18–22 August 2014; pp. 710-715.

120. Havur, G.; Haselböck, A.; Cabanillas, C. Automated multi-perspective process generation in the manufacturing domain. Proceedings of the International Conference on Business Process Management; Vienna, Austria, 1–6 September 2019; Springer: Berlin/Heidelberg, Germany, 2019; pp. 81-92.

121. Maggi, F.; Balduzzi, M.; Vosseler, R.; Rösler, M.; Quadrini, W.; Tavola, G.; Pogliani, M.; Quarta, D.; Zanero, S. Smart factory security: A case study on a modular smart manufacturing system. Procedia Comput. Sci.; 2021; 180, pp. 666-675. [DOI: https://dx.doi.org/10.1016/j.procs.2021.01.289]

122. Bencsáth, B.; Pék, G.; Buttyán, L.; Felegyhazi, M. The cousins of stuxnet: Duqu, flame, and gauss. Future Internet; 2012; 4, pp. 971-1003. [DOI: https://dx.doi.org/10.3390/fi4040971]

123. Baksi, R.P. Framework and Strategies to Mitigate Advanced. Ph.D. Thesis; State University of New York: Buffalo, NY, USA, 2022.

124. Baksi, R.P.; Upadhyaya, S.J. Kidemonas: The silent guardian. arXiv; 2017; arXiv: 1712.00841

125. Times, N. Saudi Oil Producer’s Computers Restored After Virus Attack. The New York Times; 27 August 2012.

126. Baksi, R.P.; Upadhyaya, S.J. Decepticon: A Theoretical Framework to Counter Advanced Persistent Threats. Inf. Syst. Front.; 2021; 23, pp. 897-913. [DOI: https://dx.doi.org/10.1007/s10796-020-10087-4]

127. Research, T.M. What We Know About the DarkSide Ransomware and the US Pipeline Attack. Trend Micro; 2021; Available online: https://www.trendmicro.com/en_ca/research/21/e/what-we-know-about-darkside-ransomware-and-the-us-pipeline-attac.html (accessed on 30 August 2022).

128. Eaton, C.; Volz, D. Colonial Pipeline CEO Tells Why He Paid Hackers a $4.4 Million Ransom. Wall Str. J.; 2021; Available online: https://www.wsj.com/articles/colonial-pipeline-ceo-tells-why-he-paid-hackers-a-4-4-million-ransom-11621435636 (accessed on 30 August 2022).

129. Baksi, R.P. Pay or Not Pay? A Game-Theoretical Analysis of Ransomware Interactions Considering a Defender’s Deception Architecture. Proceedings of the 2022 52nd Annual IEEE/IFIP International Conference on Dependable Systems and Networks-Supplemental Volume (DSN-S); Baltimore, MD, USA, 27–30 June 2022; pp. 53-54. Available online: https://ieeexplore.ieee.org/document/9833759/authors#authors (accessed on 30 August 2022).

130. Jartelius, M. The 2020 Data Breach Investigations Report–a CSO’s perspective. Netw. Secur.; 2020; 2020, pp. 9-12. [DOI: https://dx.doi.org/10.1016/S1353-4858(20)30079-9]

131. Wu, M.; Song, Z.; Moon, Y.B. Detecting cyber-physical attacks in CyberManufacturing systems with machine learning methods. J. Intell. Manuf.; 2019; 30, pp. 1111-1123. [DOI: https://dx.doi.org/10.1007/s10845-017-1315-5]

132. Salloum, S.A.; Alshurideh, M.; Elnagar, A.; Shaalan, K. Machine learning and deep learning techniques for cybersecurity: A review. Proceedings of the International Conference on Artificial Intelligence and Computer Vision; Cairo, Egypt, 8–10 April 2020; Springer: Berlin/Heidelberg, Germany, 2020; pp. 50-57.

133. Elfar, M.; Zhu, H.; Raghunathan, A.; Tay, Y.Y.; Wubbenhorst, J.; Cummings, M.L.; Pajic, M. Platform for security-aware design of human-on-the-loop cyber-physical systems. Proceedings of the 8th International Conference on Cyber-Physical Systems; Pittsburgh, PA, USA, 18–21 April 2017; 93.

134. Le, D.D.; Pham, V.; Nguyen, H.N.; Dang, T. Visualization and explainable machine learning for efficient manufacturing and system operations. Smart Sustain. Manuf. Syst.; 2019; 2, 20190029. [DOI: https://dx.doi.org/10.1520/SSMS20190029]