Digital Twins in Plant Factory: A Five-Dimensional Modeling Method for Plant Factory Transplanter Digital Twins

Abstract

To address challenges such as the complex correlations among multiple parameters during the modeling process of plant factory transplanters, the large differences between simulations and actual models, and the difficulties in data acquisition and processing, this paper proposes the concept of a Plant Factory Transplanter (PFT) digital twin five-dimensional model based on research of plant factory transplanters. The PFT digital twin five-dimensional model builds on traditional 3D modeling and includes physical entities, virtual models, services, twin data, and connecting interactions. This study delves deeply into the connotations and construction methods of the PFT five-dimensional model from the five aspects of PFT physical entity, virtual entity, services, twin data, and connections, and illustrates the implementation steps and effects of each link. Finally, practical examples of the application of the PFT digital twin five-dimensional model are presented in actual scenarios. The five-dimensional modeling approach for plant factory transplanters based on digital twins can monitor the working status of transplanters online and evaluate the effectiveness of transplantation. This method overcomes problems such as poor adaptability and difficulty in updating physical models, thus improving the efficiency of monitoring and optimizing configuration parameters. Moreover, the generated virtual entities are more intuitively reflected in the control interface, significantly reducing the reliance of equipment operators on relevant professional skills. In the future, the proposed digital twin five-dimensional model is expected to be further refined and optimized, with creation tools and application scenarios studied. Application research will also be conducted to meet different application requirements.

1. Introduction

In modern agriculture, plant factories have gained increasing attention as a new type of agricultural production method. With the continuous improvement in automation in plant factories, transplanting machines have become one of the key factors in improving efficiency and quality [1,2,3]. In the past, the manufacturing and use of transplanting machines inevitably required manual labor and experience. Improper use could cause damage and death to plants, while also increasing labor costs and production risks [4,5]. Plant transplantation is a vital link in the production process of plant factories [6,7,8], thus it is crucial to establish a transplanting machine model suitable for the operating environment of plant factories in order to optimize costs and increase efficiency in the planting process.

There is a large amount of sensors and artificial intelligence technology used for the modeling, simulation, and optimization of transplanting equipment in plant factory agriculture [9,10,11,12,13]. These outstanding works have greatly promoted the development of transplanting equipment technology [14,15,16,17]. Recently, driven by the rapid development of ICT such as cloud computing, IoT, big data, machine learning, augmented reality, and robotics, equipment modeling methods are becoming increasingly intelligent and data-driven [18]. The intelligent detection technology that integrates virtual reality and physical sensor networks has provided new ideas for modeling plant factory transplanting machines [19,20]. Digital twin technology is a powerful driving force behind this development [8].

In recent years, digital twin technology, as a technology that converts a physical entity into digital forms, has been widely used in the manufacturing, medical, transportation, and other fields [21]. Digital twin technology originated from the engineering practice of the National Aeronautics and Space Administration (NASA) of the United States, aiming to convert physical entities in the real world into digital forms and predict and optimize their behavior and performance through computer simulation. With the continuous development of information technology, artificial intelligence, and big data, digital twin technology is constantly being promoted and applied. In the field of manufacturing, digital twin technology can provide more accurate modeling and simulation methods for enterprises, reduce product development and production costs, and improve product quality and market competitiveness [22]. With the continuous maturity and application of digital twin technology, its application in transplanting scenarios in plant factories may demonstrate excellent production efficiency and economic value.

The modeling of transplanting machines in plant factories is difficult due to the complex correlations among multiple parameters, errors between environmental factors and actual operations, hardware implementation limitations, and data acquisition and processing [23,24,25]. However, traditional modeling methods rely solely on experience and trial and error, often failing to meet the requirements of refined and intelligent modeling. As a technology that creates a digital model of physical entity in the real world and performs simulation analysis, digital twin technology may save the cost of model optimization in the application of plant transplanting machines. Through digital twin technology, each part of the transplanting robot (including the mechanical arm, sensor, control system, etc.) can be modeled into a digital model, and the motion trajectory and action planning of the robot can be optimized through simulation and analysis to reduce plant damage and improve transplantation efficiency [26,27]. In addition, digital twin technology can also integrate IoT, cloud computing, and other technologies in order to achieve the automation and intelligence of transplanting robots, thereby improving plant factory productivity and quality while reducing costs and risks. Therefore, the research and application of digital twin technology in the field of plant factory transplanting machines are of great significance.

This paper proposes a five-dimensional modeling method for plant factory transplanting machines based on digital twin technology, in order to provide effective support for the design and optimization of transplanting machines and promote the application and popularization of digital twin technology in plant factories. The rest of this paper is organized as follows. Section 2 introduces the composition, interconnection, and construction method of the Plant Factory Transplanter (PFT) digital twin five-dimensional model. Section 3 describes the modeling process and application scenarios of the PFT digital twin model. Section 4 summarizes this study and points out future work.

2. Materials and Methods

In order to further promote the application of digital twin technology in more fields and scenarios, Professor Tao Fei of Beihang University [21,22,28], and others, have extended the existing 3D models and creatively proposed the concept of a five-dimensional digital twin model. Inspired by this concept, this paper proposes a digital twin applicable to a plant factory transplanting machine, which includes physical entities, virtual models, services, twin data, and their interconnected interaction. This can be expressed as shown in Equation (1):

M_DT = (PE, VE, Ss, DD, CN)(1)

where PE stands for physical entity, VE stands for virtual entity, Ss represents services, DD represents digital twin data, and CN represents the connections between the components. The five-dimensional digital twin model structure is shown in Figure 1 based on Equation (1). This chapter elaborates on the basic connotation of PFT and its multidimensional and multiscale intelligent space model, and achieves a multidimensional and high-fidelity modeling of PFT from five dimensions: physical space, data space, virtual space, knowledge space, and connection.

The five-dimensional digital twin model of PFT can meet the new demands of digital twin applications. Its five-dimensional structure can be integrated and fused with technologies such as IoT, big data, and artificial intelligence to meet the needs for integrated information-physical systems, fusion of information-physical data, bidirectional connection and interaction between virtual and physical spaces, and more. Additionally, digital twin data (DD) integrates and fuses information and physical data, ensuring consistency and synchronization between information space and physical space, and providing accurate and comprehensive all-element/all-process/all-business data support, thereby reducing redundancy. Services (Ss) encapsulate various data, models, algorithms, simulations, and results that are required in different fields, levels, and businesses in digital twin applications, and provide them to users in the form of application software or mobile apps, achieving convenient and on-demand use of the services. Connections (CN) achieve universal industrial interconnection between physical entity, virtual entity, services, and data, supporting real-time connection and integration between the virtual and physical worlds. Virtual entity (VE) characterizes and describes physical entity from multidimensional, multi-spatial scale, and multi-time scale perspectives. Physical entity (PE) describes the physical attributes of the plant factory transplanting equipment from the perspective of the physical space.

2.1. PFT Physical Entity

In digital twin, physical entity refers to the artificial or natural objects that exist in the real world and that can be simulated and digitized. These physical entities can be buildings, factory equipment, vehicles, robots, plants, animals, etc. Through sensors and other data collection devices, their operating status and environmental information are digitized, matched and compared with corresponding data in digital twin to establish the digital twin. In PTF digital twin, physical entity usually require three-dimensional modeling and mapping to better reproduce their appearance and internal structure. For example, in the plant factory production line, digital twin needs to accurately reproduce the geometry, material, decoration, and so on of mechanical equipment. In the digital twin of electrical equipment, attention should be paid to the structure, material, power, circuit, power supply, and so on of the equipment. After determining the physical entity in digital twin, it is necessary to use various sensors and instruments to collect their operating and environmental data, such as temperature, humidity, pressure, speed, power, vibration, etc. At the same time, it is necessary to transmit this data to the digital twin platform in various ways, so that the simulated physical entities are highly consistent with actual operation behavior. Physical entity models in digital twin can be continuously updated and improved to more accurately predict operational results and provide more accurate decisions for practical applications.

2.2. PFT Virtual Entity

In a PFT digital twin model, the virtual entity (VE) refers to a virtual object built on the digital twin platform for a specific physical entity. It is used to simulate, predict, and optimize the operating conditions and efficiency of the physical entity. The VE is composed of various types of data, such as geometric information, process parameters, operation records, historical data, etc. These data need to be collected and processed through sensors, monitoring devices, and other means to create the corresponding VE model on the digital twin platform. The PFT VE plays an important role in the digital twin model, providing users with a visual simulation environment that supports simulation experiments, performance evaluations, and maintenance optimization for physical entities under different scenarios. The VE can also analyze and predict historical data through machine learning algorithms to help users make more accurate decisions and predictions.

The following are common types of VE in digital twin models. Machine equipment virtual entity: this type of VE model usually contains information about the geometric structure, control system, process parameters, etc., of machine equipment. It is used to predict the operating status and maintenance needs of machine equipment and provide decision support for equipment optimization. Through the VE in the digital twin model, users can more accurately and effectively simulate the behavior and performance of physical entities under different scenarios and optimize and improve them accordingly. This is of great significance for improving operational efficiency, reducing losses, and risks.

To ensure that the VE can accurately represent and map the PE, it is important to verify and validate the consistency, accuracy, sensitivity, and other aspects of the VE through model checking and validation. This will help reduce redundancy and ensure that the VE is an accurate representation of the PE. Additionally, VR and AR technology can be used to achieve the virtual–real overlay and fusion display of the VE and PE, enhancing the VE’s immersion, realism, and interactivity. By utilizing these technologies, the VE can more accurately reflect the characteristics of the PE, further reducing the redundancy.

2.3. PFT Twin Data

DD is the driving force behind the PFT digital twin. The following are the types of twin data in the PFT digital twin model. ① Geometric data: geometric data are the basic data in the digital twin model used to describe the 3D geometric shape of physical entity, including parameters such as length, width, height, volume, and curvature. These data are usually generated by CAD models or laser scanning technologies. ② Operating data: operating data are key data describing the operational state of physical entity, such as rotation speed, temperature, pressure, liquid level, current, and other operating parameters, which can be obtained in real-time through monitoring devices such as sensors. ③ Operational data: operational data describes the operation records of physical entity, such as equipment on/off, adjustment operations, and maintenance records. ④ Equipment parameters: equipment parameters are important input data in the digital twin model used to describe and calculate the performance and operational status of physical entity, such as motor power, transmission ratio, cooling capacity, heat dissipation area, fan speed, and so on. ⑤ Historical data: historical data are important reference data in the digital twin model and can be used to compare and analyze the state and performance of physical entities at different stages, thereby predicting their possible future state changes, such as equipment age, operating duration, maintenance records, and so on. Through the twin data in the digital twin model, users can efficiently simulate the operational state and performance of physical entity in a virtual environment, achieving accurate monitoring and management of physical entity.

2.4. PFT Connection

Connections in the PFT digital twin model refer to the method of merging different models or datasets to generate a complete digital twin system. A digital twin system is a complex system composed of multiple models and data, which may be located in different locations, platforms, or applications. By connecting these models and data, the digital twin system can work efficiently together to achieve more accurate simulation and predictive analysis. The following are the types of connections in the PFT digital twin model.

Data connection: data connection is the most common type of connection in the digital twin model, which consolidates data from different sources for processing and analysis. For example, a digital twin system may need to collect data from multiple sensors and monitoring devices. These data can be connected to the digital twin platform through an interface to generate twin data with complete information.

Model connection: model connection is the type of connection that integrates multiple models into one digital twin system. For example, a digital twin system may include CAD models, fluid simulation models, mechanical models, and other models, which can be connected through an interface to achieve comprehensive modeling of physical entity.

System connection: system connection is the method of integrating a digital twin system with other systems and applications. For example, a digital twin system may need to connect with ERP systems, MES systems, SCADA systems, and others to achieve functions such as production planning, operation monitoring, and data collection.

Connections in the digital twin model are an important method of building a digital twin system, which can help users achieve efficient collaboration of information flow, logistics, and value stream. By using connection technology, different models and data can be organically combined to form a digital twin system with complete information, better supporting business processes such as production, operation, and maintenance.

2.5. PFT Services

The services in the PFT digital twin model refer to a series of customized services based on the digital twin model, which are provided through technologies such as cloud computing, big data analytics, and artificial intelligence. These services aim to provide users with efficient, intelligent, and sustainable solutions to meet different needs. The following are common types of services in the digital twin model. ① Simulation and emulation: the digital twin model can accurately simulate the performance of physical entity under different environments through data modeling and simulation technologies, providing users with a visual simulation environment and a platform for simulation experiments. ② Operational monitoring: The digital twin model can access sensors and other monitoring devices to monitor and record the status and operation of physical entity in real-time and alert and handle abnormal situations. ③ Comprehensive management and optimization: The digital twin model can analyze and predict historical data and real-time data through big data analytics and machine learning technologies, providing reliable data support and decision-making basis to help users with operational management and equipment optimization. ④ Efficiency improvement and energy conservation: the digital twin model can improve the efficiency and energy efficiency level of facilities and production lines by optimizing equipment layout, adjusting process parameters, improving the supply chain, and achieving the goal of energy conservation and emission reduction. ⑤ Intelligent maintenance and service: the digital twin model can provide users with fast and efficient maintenance support and service solutions through an intelligent maintenance management system and an online service platform. The services in the digital twin model can provide intelligent, efficient, and sustainable solutions for users in different industries and fields, thereby improving operational efficiency, reducing costs, strengthening security, and promoting the process of digital transformation and sustainable development.

This paper mainly introduces the composition and modeling method of the five-dimensional digital twin structure of the plant factory transplantation machine, and only discusses one implementation case in the digital twin service system. This case monitors the quality of vegetable bowl seedlings in the PFT digital twin model and completes the virtual mapping in the digital twin model. Due to the relatively closed environment in the plant factory, the quality of unmanned equipment operation needs to be evaluated by an intelligent system. The operation effect of the plant factory transplanting machine can be evaluated by evaluating the quality of the bowl seedlings in the seedling tray. This process can be displayed online through digital twin technology to show the quantity and quality of transplanted bowl seedlings dynamically in the digital twin service system. As shown in Figure 2, this paper proposes a three-dimensional block-matching filtering algorithm, which can perform noise reduction and segmentation on the collected image signal, and then use the pixel block evaluation method to classify and count the vegetable bowl seedlings. Different types of bowl seedlings are intuitively displayed dynamically on the system interface.

In reality, most of the acquired images of vegetable bowl seedlings are affected by noise interference, so it is very important to perform denoising processing on the acquired images. Image denoising refers to reducing or removing the interference noise in the acquired image. According to the distribution of the noise, it can be divided into Gaussian noise, Poisson noise, and granular noise. Gaussian noise is usually the main noise encountered in image processing. The block-matching 3D (BM3D) algorithm is a denoising algorithm based on three-dimensional transform domain, and is currently one of the best algorithms for video and image denoising. The algorithm is mainly divided into two steps: in the first step, a basic estimate is obtained from the three-dimensional matrix obtained by block matching, and in the second step, the noise image is filtered by the obtained basic estimate. Then, the overlapping blocks are re-estimated and weighted averages using the aggregation method are used to obtain a final image.

The basic principle of matching blocks in the 3D block-matching and filtering algorithm is to divide the image into several non-overlapping parts of a specific size, where the displacement of each pixel in each part is the same. A specific search area is selected and a matching criterion is established to search for similar blocks within the search area as the matching block.

In this article, I represents the noisy image, P represents any pre-divided matching block, with a block size of K × K, and Q represents the sliding window block during the search process. When the block size is known, the upper left pixel is used to represent the block, so P ∈ I and Q ∈ I. In the process of block matching, the appropriate step length h is determined first, and then the blocks are divided and the blocks are searched in the order of from top to bottom and from left to right. The current block P is selected as the reference block, and an area with a diameter of d and centered at P is used as its search area. Therefore, we have:

(2)$S\left(P\right)=\left\{Q\in I\left|d\left(X_P,X_Q\right)\right|<{\tau }_d\right\}$

where $S\left(P\right)$ is the set of similar blocks aggregated into a three-dimensional matrix and ${\tau }_d$ is the distance threshold in the search process. The distance d between the matching blocks in the search process has the following formula:

(3)$d=h^{-1}\Vert X_P-X_Q\Vert$

where X is the matrix value of the matching block.

Finally, the set of matrix blocks in the matrix is arranged in the order of size, and, finally, a three-dimensional matrix of size K × K × S(P) is obtained.

(4)$F\left(P\right)=N_{3D}^{-1}\left(\gamma \left(N_{3D}\left(T_{S\left(P\right)}\right)\right)\right)$

where $N_{3D}$ denotes the three-dimensional you-transformation of the three-dimensional matrix $T_{S\left(P\right)}$ and the operator is $N_{3D}$. The equation $\gamma$ of the function is as follows:

(5)$\gamma \left(X\right)=\{\begin{array}{c}0,\left(\left|X\right|\le {\lambda }_{3D}\mathsf{\sigma }\right)\\ X,\left(\left|X\right|>{\lambda }_{3D}\mathsf{\sigma }\right)\end{array}$

where ${\lambda }_{3D}$ is the threshold parameter of the hard threshold filtering, $\mathsf{\sigma }$ is the Gaussian white noise parameter.

$N_P$ is used to denote the non-zero values in the filtered matrix coefficients, and $W_P$ is used to denote the estimated values of the underlying weights of the current block as:

(6)$W_P^{basic}\{\begin{array}{c}\frac{1}{N_P}\left(N_P\ge 1\right)\\ 1\left(N_P<1\right)\end{array}$

After calculating the basic estimated value of three-dimensional transform domain filtering using Equation (6), the final estimated weights can be obtained as follows:

(7)$W_P^{final}=\frac{{\left|{\tau }_{3D}\left(T_{S\left(P\right)}\right)\right|}^2}{{\left|{\tau }_{3D}\left(T_{S\left(P\right)}\right)\right|}^2+{\mathsf{\sigma }}^2}$

From this equation, it can be seen that the larger the estimated weight obtained, the smaller the noise in the real image. The final real image is obtained by calculating the average estimated value of each overlapping block. The vegetable bowl seedling image has been preprocessed to separate its seedling image from the uninteresting background. The leaf area of the vegetable bowl seedling is an intuitive expression of its growth status. Therefore, the leaf area feature of the vegetable bowl seedling is selected as one of the classification criteria for identifying the health of the seedlings. Traditional methods for extracting leaf area features of seedlings include leaf area meter measurement and grid method, but these methods cannot provide accurate and efficient feedback on leaf area information. Therefore, it is necessary to conduct relevant research to accurately and efficiently extract leaf area feature information of seedlings. A digital image is composed of several square pixel blocks. By calibrating the camera size, the actual physical size corresponding to the image pixels can be obtained. Then, by counting the number of pixels occupied by the seedling leaves in the image and performing proportional conversion, the actual area of the seedling leaves can be obtained. This method is simple to operate and has a small area calculation error. The extracted leaf area value of the vegetable bowl seedling is compared with the ratio of the hole area of the vegetable bowl seedling tray to obtain a threshold. Finally, the intelligent recognition algorithm for the health of the vegetable bowl seedling detects the vegetable bowl seedling by comparing the threshold.

3. Case Study

3.1. PFT Physical Entity Modeling

The PFT physical entity modeling process should first complete the main structure design of the mechanical equipment. According to the design requirements, the main structure of the entire machine was designed. In the model design stage, SolidWorks software was used to design the structural part, and, subsequently, mechanical processing was completed. The overall structure of the plant factory transplanting machine is shown in Figure 3. Its main body consists of a tray transport positioning mechanism, a cultivation trough transport positioning mechanism, an overall framework, a horizontal translation mechanism, a vertical translation mechanism, and a seedling transplantation end effector. The overall height of the mechanism is 1.70 m, the width is 1.40 m, and the length is 1.80 m.

The tray seedling low-loss transplantation robot consists of six parts. The first part is the positioning and conveying mechanism of the tray, which is mainly responsible for the positioning and transportation of the tray and completing the acquisition of the side image of the seedling. The second part is the positioning and conveying system of the cultivation groove. The third part is the framework. The fourth part is the seedling-picking truss mechanism, which adopts a dual-guide rail design to reduce the interference space of the end effector of the picking mechanism and improve the number of end effector installations. The fifth part is the dual-axis drive system, which consists of two sets of linear screw rods and is mainly responsible for driving the X and Z axes of the end effector. The sixth part is the end effector of the seedling-picking mechanism, which is mainly responsible for the low-loss harvesting operation of the seedlings. It is driven by a servo motor, which can ensure its working accuracy and precisely control the working stroke. The robot system can autonomously complete the transplanting operation. Firstly, the tray seedlings to be transplanted are transported to the transplanting area by a conveyor belt. An infrared position sensor determines the stop position of the tray and the transplanting manipulator is slid over the tray seedlings, clamping them in parallel and moving them to the top of the seedling rack. The transfer manipulator slowly places the tray seedlings into the cell of the seedling tray, and returns to the initial position to complete the transplanting action.

Due to the various differences in the structure and design of the plant factory transplanting robot, it is necessary to model the digital twin physical model according to the robot and the application scenario. As shown in Figure 4, the PFT digital twin physical entity modeling mainly includes the overall structure of the transplanting robot, control system modeling, sensor system modeling, and modeling of the plants to be transplanted.

The overall structure and construction of the transplanting robot include mechanical arms, sensors, control systems, mobile components, etc. To reduce redundancy in the control system modeling for plant factory transplanters, it is important to integrate the various subsystems into a comprehensive control system model. This includes the Programmable Logic Controller (PLC) electrical control system of the transfer arm, the mechanical hand control system of the transplanting robot, and the human–machine interaction touch system. By integrating these systems into a single model, redundancy can be reduced and the overall control system can be more efficient and effective. Additionally, this integrated model can be used to monitor and optimize the performance of the entire system, further reducing redundancy and improving efficiency. The sensor system modeling includes three-axis acceleration sensors, high-definition industrial cameras, laser infrared position sensors, and LED auxiliary light sources. The modeling of plants to be transplanted includes plants during the transplanting period and standardized tray seedlings.

3.2. PFT Virtual Entity Modeling

Plant factory transplanting machine digital twin virtual entity modeling is usually performed using computer programs. By inputting the parameters and features of the physical entity into the model, a virtual entity with similar characteristics can be generated. As shown in Figure 5, PFT virtual entity modeling mainly includes virtual environment modeling, virtual modeling of the transplanting robot, control system modeling, virtual modeling of plants, and reinforcement learning algorithm modeling.

Virtual environment modeling: CAD software or other 3D modeling software can be used to construct a virtual environment on the digital twin platform, including the space where the transplanting robot is located, the environment around the robot, and the different light sources.

Virtual modeling of the transplanting robot: the modeling of the various components of the transplanting robot (such as the mechanical arm) and drivers. These models can be based on the robot’s CAD model or by analyzing data collected from the actual robot to generate a virtual model of the robot.

Control system modeling: the digital twin model needs to consider the control system of the robot, including its sensors, drivers, actuators, communication protocols, control algorithms, etc. Establishing control system models can help optimize the robot’s control strategy.

Virtual plant modeling: by inputting the characteristics and morphology of plants into the model, virtual plants with similar characteristics, such as roots, stems, leaves, etc., can be generated.

Reinforcement learning algorithm modeling: the robot transplanting task can be controlled through reinforcement learning algorithms. Based on this, it is necessary to establish an intelligent algorithm model that interacts with the environment to train the robot to complete transplanting tasks under various conditions.

In summary, PFT virtual entity modeling technology provides an effective means of virtual simulation and optimization for plant factory transplanting machines. It can help design teams test and verify the performance and accuracy of robots, and make improvements and optimizations.

3.3. PFT Twin Data Processing and Applications

Plant factory transplanting machine digital twin data processing and application refers to processing and applying data related to plant transplanting through digital twin technology. The system can use various sensors and algorithms to real-time record the process of the robot executing transplanting tasks, establish a digital twin model, and perform data processing, analysis, and application.

The PFT twin data processing and application process includes data collection, data preprocessing, data analysis, and data application. Data collection: the system is equipped with various sensors, such as visual sensors, acceleration sensors, and position sensors, to capture data on the robot’s execution of transplanting tasks and plant growth status. The data obtained from these sensors can be used to establish digital twin models and provide decision support.

Data preprocessing: to improve data quality and reliability, data preprocessing is an essential step. The system can filter, enhance, and correct raw data to remove noise and errors and make data easier to understand and analyze.

Data analysis: the system uses data statistical analysis tools and algorithms, such as machine learning and artificial intelligence algorithms, for data mining and analysis. By statistically analyzing the data, the system can evaluate the effectiveness of the transplanting operation and the operating status of the transplanting machine, providing a reliable basis and guidance for decision-making in plant factories.

Data application: the system can use data to establish and optimize digital twin models, as well as improve robot control and plant cultivation strategies. Through simulation experiments and prediction in the digital twin environment, the system can provide optimized solutions for plant cultivation and help robots complete more efficient and accurate transplanting tasks.

In summary, the data collection and processing of the digital twin of the plant factory transplanting machine is the core link of the entire digital twin system. It can help us obtain key data and analyze and apply them to improve the transplanting strategy and enhance the quality of transplanting.

3.4. PFT Connection

CN realizes the interconnection of various components of PFT-DT. According to the theory proposed by Professor Tao Fei from Beihang University, PFT-DT’s connections can be divided into six types: the connection between PE and VE (CN_PV), the connection between PE and DD (CN_PD), the connection between PE and Ss (CN_PS), the connection between VE and Ss (CN_VS), the connection between VE and DD (CN_VD), and the connection between Ss and DD (CN_SD). Among them: CN_PV is used to upload data from the real plant environment to the digital twin environment to establish a digital twin model for simulation; CN_PS is used to upload data from each subsystem of the PFT system to the digital twin environment, providing more information for the digital twin model; CN_VS is used to feed back the simulation results from the digital twin environment to the PFT system to optimize the control strategy; CN_PD is used to feed the decision-making plan from the digital twin environment back to the PFT system to guide the robot to perform transplantation tasks; CN_VD is used to feed the optimization plan from the digital twin environment back to the PFT system to optimize the robot control and cultivation strategy; and CN_SD is used to exchange data between the subsystems in the PFT system to collaborate and complete transplantation tasks. Through these connection methods, digital twin technology can interact closely with the PFT-DT system and improve the efficiency and quality of transplantation.

(8)$CN=(CN\_PD,CN\_PV,CN\_PS,CN\_VD,CN\_VS,CN\_SD)$

where CN_PD can use sensors such as accelerometers, position sensors, and vision sensors to collect real-time data from PE and transmit the data to DD through standardized protocols such as MQTT, in order to achieve the interaction between PE and DD. Additionally, processed data or instructions can also be sent back to PE through these protocols to optimize its operation.

Similarly, the method for realizing the interaction between PE and VE in CN_PV is similar to that in CN_PD. Real-time data from PE are collected using sensors and transmitted to VE for updating and correcting the digital model. The simulated analysis data of the VE can be converted into control instructions and sent to the PE actuators to achieve real-time control of the PE. By utilizing these methods of interaction, redundancy is reduced, ultimately leading to improved accuracy and efficiency in both CN_PD and CN_PV.

The method used to achieve the interaction between the PE and the Ss in CN_PS is similar to that in CN_PD. Real-time data from the PE are collected and transmitted to the Ss for updating and optimization. The operational guidance, professional analysis, and decision optimization results generated by the Ss can be provided to users through application software or mobile apps. Additionally, users can manually control the PE through artificial operation.

In CN_VD, the ODBC database interface is utilized to store the simulation and related data generated by the VE in the DD in real-time. These data are then used to drive the dynamic simulation by reading fusion data, associated data, and life cycle data from DD in real-time. Similarly, two-way communication can be achieved through software interfaces such as Socket, RPC, and MQSeries in CN_VS to achieve the interaction between the VE and Ss. This includes direct instruction transmission, data transmission/reception, and message synchronization operations.

Lastly, the method for realizing the interaction between Ss and DD in CN_SD is similar to that of CN_VD. The database interface is used to store Ss data in DD in real-time. Historical data, rule data, common algorithms, and models can be read from DD in real-time to support Ss operation and optimization. By utilizing these methods of interaction, redundancy can be reduced and accuracy and efficiency improved across all systems.

3.5. PFT Service System

The digital twin transplanting quality monitoring service system for plant factory transplanting machines refers to providing users with monitoring and analysis of transplanting quality based on a virtual model through digital twin technology and services. The system interface includes virtual entities, physical entities, signal monitoring, seedling trays, and transplanting evaluation. The virtual entity window interface reflects the operation process of the transplanting equipment in real-time through animation. The seedling tray window interface analyzes the situation of the seedling tray after transplanting through machine vision algorithms. The transplanting evaluation window interface evaluates the transplanting effect in real-time and counts the quality of the transplanted seedlings. Below are the results analysis of threshold segmentation, denoising, and classification recognition using different methods of potting seedling images in the PFT service system.

3.5.1. Comparison of Traditional Maximum Interclass Variance Method Threshold Segmentation and Optimal Threshold Method Segmentation

To verify the rationality of the genetic algorithm-based optimal threshold method used in this article, threshold segmentation, using both the genetic algorithm-based optimal threshold method and the traditional maximum interclass variance threshold method, was performed on two sets of original images containing 4200 vegetable seedlings in total, as shown in Figure 6a,b. Among them, there were 3150 high-quality seedlings in group (b) and 2750 seedlings in group (a), with the seedlings in group (b) generally having better growth than those in group (a). Group a is the normal potted seedling group and group b is the high quality potted seedling group. The threshold segmentation results are shown in Figure 7 and Figure 8.

From the above Figure 7 and Figure 8, it can be observed that the image obtained by threshold segmentation using the genetic algorithm-based optimal threshold method can more clearly separate the vegetable seedling leaves from the background. On the other hand, the image obtained by threshold segmentation using the traditional maximum interclass variance threshold method failed to separate the vegetable seedling leaves from the background and the overall image is blurred and clearly does not meet the requirements. Therefore, this comparison can verify the rationality of using the genetic algorithm-based optimal threshold method for threshold segmentation in this article. In addition, as Figure 7 clearly does not meet the requirements and cannot meet the conditions for further experimentation, the following comparison of denoising algorithms is based solely on Figure 8.

3.5.2. Comparison of Traditional Wiener Algorithm and 3D Block-Matching Filtering Algorithm Image Denoising

To verify the rationality of the 3D block-matching filtering algorithm used in this article, further denoising of the two images in Figure 8a,b was performed using the 3D block-matching filtering algorithm and the traditional Wiener algorithm, respectively. The denoising effects are shown in Figure 9 and Figure 10.

From the various results obtained, it can be observed that the image obtained by denoising using the 3D block-matching filtering algorithm in Figure 10 was able to remove almost all of the noise generated from the background except for the vegetable seedling leaves, and the denoising effect was good. However, the images obtained by denoising using the traditional Wiener algorithm on Figure 9 were unable to remove the noise that was closer to the leaves and was larger in size, and there were large errors overall. Therefore, this comparison can verify the rationality of using the 3D block-matching filtering algorithm for image denoising in this article.

The above Figure 11 is the final recognition image obtained by using the optimal thresholding method for threshold segmentation and using the traditional maximum interclass variance method and the 3D block-matching filtering algorithm for image denoising. The health condition of the seedlings is analyzed comprehensively through the threshold F and unit leaf area, and the seedlings are classified as healthy, sub-healthy, poor quality, or empty based on this analysis. The red number “1” represents healthy seedlings, the green number “1” represents sub-healthy seedlings, the yellow number “1” represents poor-quality seedlings, and the blue number “0” represents empty areas. The coordinates of different types of seedlings are output after calibrating the coordinates of the seedling tray, as shown in Figure 11. The output result is generated in a virtual mapping in the digital twin system and presented dynamically in the service system.

The results of different types of seedlings output in Figure 12 are transmitted in real-time to the digital twin service system for counting. As shown in Table 1, The recognition results using different algorithms show significant differences. Overall, the accuracy of identifying healthy seedlings after denoising based on the 3D block-matching filtering algorithm is higher than that based on the traditional Wiener algorithm. The highest accuracy rate for identifying healthy seedlings can be as high as 98.25%. However, in group A, the highest accuracy rate for identifying healthy seedlings is 96.62%, which is lower than the data in group B. The possible reason for this is that the seedlings in group B have better growth, and there are more healthy seedlings overall, so the algorithm performs well in identifying healthy seedlings. From the data in the table, we can see that the accuracy rate for identifying sub-healthy seedlings and poor-quality seedlings is lower than that for identifying healthy seedlings and empty areas. This may be due to the obscuration and confusion between sub-healthy seedlings and poor-quality seedlings during leaf segmentation and detection. In terms of identifying empty areas, the accuracy of identifying healthy seedlings after denoising based on the 3D block-matching filtering algorithm is higher than that based on the traditional Wiener algorithm. This may be due to the effective removal of image noise by the 3D block-matching filtering algorithm, which improves the detection accuracy.

This study provides a five-dimensional digital twin modeling method for a plant factory transplantation machine. This method allows the post-transplantation information of bowl seedlings to be reflected in the digital twin service system through virtual mapping. Currently, in terms of image processing, the success rate of bowl seedling recognition based on the traditional Wiener algorithm is 95.64%. After applying the proposed three-dimensional block-matching filtering algorithm on the experimental machine, the success rate of bowl seedling recognition increases to 96.62%. The use of the three-dimensional block-matching filtering algorithm effectively reduces the impact of interference signals on bowl seedling recognition. The main focus of this research is on monitoring the quality of transplantation operations and evaluating the transplantation outcomes. It aims to classify the transplanted bowl seedlings, without excessive reliance on operator experience, and to evaluate the transplantation effectiveness. Based on the feedback from the evaluation of operational effectiveness, the transplantation equipment can be optimized and designed. Moreover, the optimal working parameters and conditions for the plant factory transplantation machine can be explored. This research shares the same application scenarios as other studies by fellow researchers, and, together, they can mutually promote and advance transplantation technology.

Figure 13 shows the PFT digital twin transplantation quality monitoring service system. This system can provide a visual and interactive environment to help users evaluate the quality of the transplantation task completed. During this process, problems with recognition may be identified and the transplantation strategy optimized.

The transplantation effect evaluation method, based on the digital twin model and a data-driven approach, can monitor the working status of the plant factory transplantation machine online, and also evaluate the transplantation effect. This method overcomes the problem of difficult updates based on physical models and greatly improves the efficiency of monitoring and optimizing configuration parameters. At the same time, the generated virtual entity mapping is more intuitively reflected in the control interface, which can greatly reduce the operator’s dependence on relevant professional skills. In the future, if VR/AR technology can be combined, it will bring a better plant farming experience, and it may become possible for operators to plant vegetables as they would play electronic games.

4. Conclusions

This paper takes the plant factory transplantation machine as its research object. Based on traditional 3D model research, the concept of the PFT digital twin five-dimensional model is proposed, which includes physical entities, virtual models, services, twin data, and their connection interactions. The connotation and construction methods of the PFT five-dimensional model were studied deeply and explained in detail. From the aspects of PFT physical entities, virtual entities, services, twin data, and connections, the construction strategy and method of the five-dimensional model of the plant factory transplantation machine are described. Then, the implementation steps for each link are introduced through examples. Finally, combined with practical application scenarios, a case study of on-site bowl seedling detection using the PFT digital twin five-dimensional model is presented. The digital-twin-based modeling method of the five-dimensional model of the plant factory transplantation machine can monitor the working status of the transplantation machine online and also evaluate the transplantation effect. The highest accuracy rate for recognizing healthy bowl seedlings can reach 98.25%. This method overcomes the problem of the difficult monitoring of bowl seedlings in a plant factory environment and greatly improves the efficiency of monitoring and optimizing configuration parameters. At the same time, the generated virtual entity mapping is more intuitively reflected in the control interface, which can greatly reduce the operator’s dependence on relevant professional skills. It is hoped that this work can provide ideas and references for the landing application of digital twins in other different fields. In the future, the proposed digital twin five-dimensional model will be further refined and optimized, and its creation tools and application scenarios will be studied, and application research will be conducted in conjunction with different application needs.

Author Contributions

Conceptualization, K.C.; formal analysis, B.Z. and H.Z.; funding acquisition, L.Z., Y.Y. and Y.Z.; investigation, B.Z., H.Z. and L.Z.; methodology, K.C.; project administration, Y.Y. and X.J.; resources, Y.Z.; software, K.C.; supervision, Y.Z.; visualization, R.L.; writing—original draft, K.C., X.J. and R.L.; writing—review and editing, K.N. and R.L. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Not applicable.

Data Availability Statement

Not applicable.

Conflicts of Interest

The authors declare no conflict of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Table

Figure 1. PFT digital twin five-dimensional conceptual model.

Figure 2. The 3D block-matching filtering algorithm.

Figure 3. The schematic diagram of the structure of the plant factory transplanting machine. 1. Tray transport positioning mechanism 2. Cultivation trough transport positioning mechanism 3. Equipment framework 4. Seedling-picking truss mechanism 5. Dual-axis drive system 6. Seedling transplantation end effector.

Figure 4. PFT digital twin physical entity modeling diagram.

Figure 5. PFT digital twin virtual entity modeling diagram.

Figure 6. Original picture of vegetable seedlings.(a) the normal potted seedling group (b) the high quality potted seedling group.

Figure 7. Threshold segmentation of traditional maximum interclass variance method. (a) the normal potted seedling group (b) the high quality potted seedling group.

Figure 8. Optimal threshold method threshold segmentation. (a) the normal potted seedling group (b) the high quality potted seedling group.

Figure 9. Traditional Wiener algorithm image denoising. (a) the normal potted seedling group (b) the high quality potted seedling group.

Figure 10. Three-dimensional block-matching filtering algorithm image denoising. (a) the normal potted seedling group (b) the high quality potted seedling group.

Figure 11. Final recognition based on 3D block-matching filtering algorithm. (a) the normal potted seedling group (b) the high quality potted seedling group.

Figure 12. Digital twin system generates virtual mappings. (a) the normal potted seedling group (b) the high quality potted seedling group.

Figure 13. Plant factory transplanting machine digital twin transplant quality monitoring service system.

Table 1

Comparative analysis of the accuracy of healthy pot seedling identification by different kinds of algorithms.

		Healthy Seedlings	Sub-Healthy Seedlings	Poor-Quality Seedlings	Empty	Health Seedling Recognition Accuracy
Group a. Potted seedlings		2750	1030	294	126	-
Group a. Based on 3D block-matching filtering algorithm		2657	1080	315	148	96.62%
Group a. Based on the traditional Wiener algorithm		2630	1036	367	167	95.64%
Group b. Potted seedlings		3150	630	252	168	-
Group b. Based on 3D block-matching filtering algorithm		3095	670	264	171	98.25%
Group b. Based on the traditional Wiener algorithm		3011	656	286	247	95.59%

References

1. Van Henten, E.J. Automation and robotics in greenhouses. Achieving Sustainable Greenhouse Cultivation; Burleigh Dodds Science Publishing Limited: Cambridge, UK, 2019; Volume 63, pp. 359-378.

2. Yin, W.; Liu, H.; Hu, F.; Yan, H.; Guo, D.; Wu, Y. Optmization Design and Experiment on Eight-linkage Planting Mechanism of Dryland Transplanter. Nongye Jixie Xuebao/Trans. Chin. Soc. Agric. Mach.; 2020; 51, pp. 51-60.

3. Jiang, L.; Wu, C.; Tang, Q.; Zhang, M.; Wang, G. Kinematics model and parameter optimization of planting process of rape carpet seedling transplanter. Trans. Chin. Soc. Agric. Eng.; 2018; 34, pp. 37-46.

4. Dai, L.; Sun, L.; Zhao, X.; Zhao, Y. Parameters optimization of separating-planting mechanism in transplanter based on kinematics objective function. Trans. Chin. Soc. Agric. Eng.; 2014; 30, pp. 35-42.

5. Jo, J.S.; Okyere, F.G.; Jo, J.M.; Kim, H.T. A Study on Improving the Performance of the Planting Device of a Vegetable Transplanter. J. Biosyst. Eng.; 2018; 43, pp. 201-210.

6. Li, M.; Xiao, L.; Ma, X.; Yang, F.; Jin, X.; Ji, J. Vision-Based a Seedling Selective Planting Control System for Vegetable Transplanter. Agriculture; 2022; 12, 2064. [DOI: https://dx.doi.org/10.3390/agriculture12122064]

7. Chen, K.; Yuan, Y.; Zhao, B.; Zhou, L.; Li, R.; Niu, K.; Xu, M.; Wang, C.; Han, N.; Jin, X. et al. Research and Development for the Hurdle Avoidance During Transplanting Manipulation Based on Kinect Vision Processing. Am. J. Biochem. Biotechnol.; 2022; 18, pp. 370-377. [DOI: https://dx.doi.org/10.3844/ajbbsp.2022.370.377]

8. Pylianidis, C.; Osinga, S.; Athanasiadis, I.N. Introducing digital twins to agriculture. Comput. Electron. Agric.; 2021; 184, 105942. [DOI: https://dx.doi.org/10.1016/j.compag.2020.105942]

9. Zhu, D.; Yao, Y.; Yang, S.; Song, Y.; Zhang, J.; Wang, Y. Kinematics analysis and optimization design on separating-planting mechanism of narrow row spacing transplanter. J. Mach. Des.; 2016; 33, pp. 73-77.

10. Patil, S.B.; Shahare, P.U.; Aware, V.V. Study on planting performance of power operated paddy transplanter suitable for root washed seedlings. J. Indian Soc. Coast. Agric. Res.; 2017; 35, pp. 70-74.

11. Chen, K.; Fu, Y.; Zheng, Y.; Zhao, B.; Yuan, Y.; Zhou, L.; Jin, X. Multi-Feature Recognition of Healthy Vegetable Seedlings Based on Machine Vision Technology. Am. J. Biochem. Biotechnol.; 2022; 18, pp. 141-154. [DOI: https://dx.doi.org/10.3844/ajbbsp.2022.141.154]

12. Jiangtao, J.; Kaikang, C.; Xin, J.; Zhaoyang, W.; Baoqiong, D.; Jingyuan, F.; Xiaojun, L. High-efficiency modal analysis and deformation prediction of rice transplanter based on effective independent method. Comput. Electron. Agric.; 2020; 168, 105126. [DOI: https://dx.doi.org/10.1016/j.compag.2019.105126]

13. Jin, X.; Chen, K.; Ji, J.; Zhao, K.; Du, X.; Ma, H. Intelligent vibration detection and control system of agricultural machinery engine. Measurement; 2019; 145, pp. 503-510. [DOI: https://dx.doi.org/10.1016/j.measurement.2019.05.059]

14. Doi, K. Development of planting height control system for rice transplanters—Automatic planting height adjusting system. J. Jpn. Soc. Agric. Mach. Food Eng.; 2018; 80, pp. 440-446.

15. Cui, J.; Li, X.; Zeng, F.; Bai, H.; Zhang, Y. Parameter Calibration and Optimization of a Discrete Element Model of Plug Seedling Pots Based on a Collision Impact Force. Appl. Sci.; 2023; 13, 6278. [DOI: https://dx.doi.org/10.3390/app13106278]

16. Jorg, O.J.; Sportelli, M.; Fontanelli, M.; Frasconi, C.; Raffaelli, M.; Fantoni, G. Design, Development, and Testing Feeding Grippers for Vegetable Plug Transplanters. AgriEngineering; 2021; 3, pp. 669-680. [DOI: https://dx.doi.org/10.3390/agriengineering3030043]

17. Shuangyan, H.; Minjuan, H.; Wenyi, Z. Design and experiment of flexible clamping device for pepper plug seedlings. Adv. Mech. Eng.; 2022; 14, 16878132221107254. [DOI: https://dx.doi.org/10.1177/16878132221107254]

18. Wang, X.; Vladislav, Z.; Viktor, O.; Wu, Z.; Zhao, M. Online recognition and yield estimation of tomato in plant factory based on YOLOv3. Sci. Rep.; 2022; 12, 8686. [DOI: https://dx.doi.org/10.1038/s41598-022-12732-1]

19. Ariesen-Verschuur, N.; Verdouw, C.; Tekinerdogan, B. Digital Twins in greenhouse horticulture: A review. Comput. Electron. Agric.; 2022; 199, 107183. [DOI: https://dx.doi.org/10.1016/j.compag.2022.107183]

20. Verdouw, C.; Tekinerdogan, B.; Beulens, A.; Wolfert, S. Digital twins in smart farming. Agric. Syst.; 2021; 189, 103046. [DOI: https://dx.doi.org/10.1016/j.agsy.2020.103046]

21. Tao, F.; Sui, F.; Liu, A.; Qi, Q.; Zhang, M.; Song, B.; Guo, Z.; Lu, S.C.Y.; Nee, A.Y. Digital twin-driven product design framework. Int. J. Prod. Res.; 2019; 57, pp. 3935-3953. [DOI: https://dx.doi.org/10.1080/00207543.2018.1443229]

22. Tao, F.; Cheng, J.; Qi, Q.; Zhang, M.; Zhang, H.; Sui, F. Digital twin-driven product design, manufacturing and service with big data. Int. J. Adv. Manuf. Technol.; 2018; 94, pp. 3563-3576. [DOI: https://dx.doi.org/10.1007/s00170-017-0233-1]

23. Tang, Y.; Wang, J.; Cheng, S. Parameter Optimization for Dibble-type Planting Apparatus of Vegetable Pot Seedling Transplanter in High-speed Condition. Nongye Jixie Xuebao/Trans. Chin. Soc. Agric. Mach.; 2016; 47, pp. 91-100.

24. Zhang, M.; Zhang, W.; Zhou, C.; Qi, B.; Ji, Y. Parameter Analysis and Experiment of Three Planting Bars of High-performance Rice Transplanter with Non-uniform Rotation. Nongye Jixie Xuebao/Trans. Chin. Soc. Agric. Mach.; 2015; 46, pp. 32-38.

25. Hu, F.; Guo, D.; Chen, C.; Yan, H.; Yin, W.; Yu, H. Design and Experiment on Compound Crank Rocker Double-row Planting Device of Vegetable Plug Seedling Up-film Transplanter. Nongye Jixie Xuebao/Trans. Chin. Soc. Agric. Mach.; 2021; 52, pp. 62-69.

26. Chen, K.; Yuan, Y.; Zhao, B.; Jin, X.; Lin, Y.; Zheng, Y. Finite element modal analysis and experiment of rice transplanter chassis. Int. J. Agric. Biol. Eng.; 2022; 15, pp. 91-100. [DOI: https://dx.doi.org/10.25165/j.ijabe.20221505.6230]

27. Chen, K.; Zhao, B.; Zhou, L.; Wang, L.; Wang, Y.; Yuan, Y.; Zheng, Y. Real-time missed seeding monitoring planter based on ring-type capacitance detection sensor. INMATEH Agric. Eng.; 2021; 64, pp. 279-288. [DOI: https://dx.doi.org/10.35633/inmateh-64-27]

28. Tao, F.; Zhang, M.; Cheng, J.; Qi, Q. Digital twin workshop:a new paradigm for future workshop. Comput. Integr. Manuf. Syst.; 2017; 23, pp. 1-9.