A Bio-Inspired Visual Network That Fuses Motion and Contrast Features for Detecting Small Moving Objects in Dynamic Complex Environments

Abstract

In complex and dynamic environments, traditional motion detection techniques that rely on visual feature extraction face significant challenges when detecting and tracking small-sized moving objects. These difficulties primarily stem from the limited feature information inherent in small objects and the substantial interference caused by irrelevant information in complex backgrounds. Inspired by the intricate mechanisms for detecting small moving objects in insect brains, some bio-inspired systems have been designed to identify small moving objects in dynamic natural backgrounds. While these insect-inspired systems can effectively utilize motion information for object detection, they still suffer from limitations in suppressing complex background interference and accurately segmenting small objects, leading to a high rate of false positives from the complex background in their detection results. To overcome this limitation, inspired by insect visual neural structures, we propose a novel dual-channel visual network. The network first utilizes a motion detection channel to extract the target’s motion position information and track its trajectory. Simultaneously, a contrast detection channel extracts the target’s local contrast information. Then, based on the target’s motion trajectory, we determine the temporal variation trajectory of the target’s contrast. Finally, by comparing the temporal fluctuation characteristics of the contrast between the target and background false positives, the network can effectively distinguish between the target and background, thereby suppressing false positives. The experimental results show that the visual network performs excellently in terms of detection rate and precision, with an average detection rate of 0.81 and an average precision as high as 0.0968, which are significantly better than those of other comparative methods. This indicates that it has a significant advantage in suppressing false alarms and identifying small targets in complex dynamic environments.

1. Introduction

Tracking object movement technology is one of the key technologies in visual computing and has been extensively and successfully employed in many fields, such as security surveillance [1], autonomous driving [2], and early warning systems. Although it can effectively detect object motion in various complex scenes, there remains a great challenge in accurately distinguishing small target motion in complex dynamic environments due to low-resolution and limited visual features. For example, when an object is located at a great distance from the observing camera, it occupies only a few pixels and lacks distinct physical characteristics in the captured image, making it challenging to extract the limited features of such small objects (see Figure 1a). Moreover, in the real visual environment, the small object is usually concealed in complex backgrounds, which further complicates the discrimination of the small target from small-target-like objects (such as leaves, branches, and so on) in backgrounds. Therefore, developing a robust visual system that can recognize the movement of small objects in complex dynamic environments is an urgent problem for future computer vision systems.

Over the past few decades, researchers have proposed a variety of algorithms to address the problem of object motion detection. Traditional computer vision methods (such as background subtraction [3,4], temporal differencing [5], and optical flow [6]) perform well in static backgrounds, but they have limitations in detecting targets, especially small targets, in complex dynamic scenes. Deep learning methods have made significant progress in object classification [7,8,9] and salient object detection [10,11], but due to the inconspicuous visual features of small targets, their detection performance is not satisfactory.

Biological studies have found that some insects, such as dragonflies, can perfectly track flying small objects in complex natural environments by using limited visual information and the successful rates of pursuit are greater than 97% [12]. This superior ability is due to a special type of neurons in the visual nervous system of insects called small target motion detectors (STMDs), which present a strong sensitivity to small moving objects [13,14]. The perceptual neurons respond powerfully to small moving objects covering 1 degree to 3 degree of the perceptual field, but show weak or no response to larger moving objects exceeding 10 degrees, and to large stimuli [15]. Further research indicates that these unique neurons are equally sensitive to small moving objects in complicated dynamic environments. Therefore, the special visual system of tracking small targets with precision in an insect’s brain may provide a reliable template for designing a robust small target detection visual neural network.

In the past decades, based on the revelation of the electrophysiology of STMD neurons, several visual recognition networks have been developed to recognize the movement of small objects in complex natural environments. For example, Wiederman et al. first designed ESTMD [16] to mimic the size selectivity of STMD neurons. This model detects small target motion by correlating successive luminance-change signals at each pixel. DSTMD [17] and two cascaded models [18,19] detect motion direction by correlating information from two spatial locations. But they exhibit limitations in detecting small moving targets in complex dynamic backgrounds. To address these limitations, Wang et al. and Ling et al. proposed Feedback STMD [20] and $\mathcal{F}$STMD [21], which improve the detection accuracy by suppressing background small-target-like features responses. Moreover, Xu et al. developed fractional-order visual networks [22] by using fractional derivative models to detect small target motion in low-sampling-frequency videos. To enhance the detection performance of low-contrast small moving targets, Wang integrated attention and prediction modules into the traditional STMD-based model to construct an attention model (apg-STMD) [23]. The model dynamically modulates the output based on the detection results of the STMD model from the previous frame, thereby enhancing the detection of potential small target motion in the current frame. In addition, Chen et al. theoretically demonstrated that the STMD computational model can rigidly propagate the dynamics of translational visual motion, meaning that the dynamics of the model output remain consistent with the dynamics of the input visual motion [24]. M. A. Billah et al. applied the small target motion detection system to the control field and developed relevant control theories based on it [25]. Uzair M et al. applied the STMD neural vision system to visible spectrum image analysis to achieve accurate detection of distant small targets [26,27,28]. The relevant STMD models discussed above can recognize small moving objects and match the electrophysiology of STMD neurons by using motion features detected by large monopolar cells [29,30]. However, these relevant STMD models face difficulties in separating small objects from background noise features, and as a result, the detection outcomes inevitably include significant interference from background noise fake features. This is because the fake features are always hidden in complex backgrounds and move with the dynamic backgrounds, which makes it difficult for existing STMD-based visual networks to filter out background false targets via only relying on location information. Recently, Ling et al. designed an $\mathcal{R}$STMD model based on the STMD neural system, leveraging the differential responses of STMD neurons to moving targets versus background noise, in order to filter out background noise and enhance the detection of small moving objects [31]. The research results confirm that by combining other visual feature information, more precise detection of small target motion can be achieved. Inspired by the aforementioned research, in order to filter out small background false features, it is necessary to combine other visual information, such as the contrast between the object and the background, to identify small objects from false traits. Contrast serves as a metric that quantifies the disparity in brightness attributes between the object and its surrounding environment. Many research results show that using contrast information can help detect object motion and distinguish small objects from false features in the background.

Within the insect nervous system, there are many visual neurons that are able to extract and process a variety of external natural movement information to ensure optimal visual performance. For example, biological studies confirm that the ommatidia act as visual brightness receptors and can capture external brightness signals and denoise external brightness signals. As the posterior neurons of the ommatidia, LMC cells [32,33] have been confirmed to receive information from the ommatidia outputs and extract the motion information cue from ommatidia outputs. However, the mechanism of another type of amacrine cells (AMCs) [34,35], which are connected to the ommatidia by slender fibers, in the visual system for detecting movements of small objects is currently unclear. Cumulative research confirms that integrating contrast information into the visual system may be effective in discriminating moving objects from distracting background features. Therefore, a contrast information processing module may be built by the AMCs and their downstream neurons to distinguish small objects from false background features.

In this paper, we develop a new visual neural network called $\mathcal{CI}$STMD to detect and distinguish small moving objects in complex dynamic backgrounds. The visual neural network mainly consists of two channels; one is a motion information detection channel, and the other one is a local contrast information detection channel. The motion information detection channel is capable of extracting the motion information of small objects. When a small target passes over a pixel, the output of the ommatidia will show a trend of first decreasing and then increasing, and finally return to the initial level. These changes are caused by the entry and exit of the small target, respectively. LMC cells can sense the changes in brightness and pass the signal to Tm3 and Tm1 neurons. Among them, the response of Tm1 neurons has a certain delay. Finally, STMD produces a strong response by multiplying the outputs of Tm3 and Tm1 neurons at the same time position (see Figure 2a). Meanwhile, the contrast information detection channel uses AMC neurons to calculate the sum of the brightness values of the local area centered on each pixel and its surrounding area. Subsequently, T1 neurons calculate the difference in average brightness values between these two areas, thereby extracting the contrast information of each central local area (see Figure 2b). In the information fusion stage, the system integrates the information extracted by the motion detection channel and the contrast feature recognition channel. By obtaining the contrast trajectory of the target over time through its motion trajectory, the system finally calculates the variance of contrast and compares it with a threshold. This enables the system to effectively suppress background noise and achieve the precise localization of small targets (see Figure 3). The experimental findings support the notion that the proposed network can accurately distinguish small objects from background interference, and the $\mathcal{CI}$STMD system demonstrates significant performance in detecting small objects in complex dynamic scenes. The contributions of this paper are as follows:

(1) This paper points out that moving small targets and background false positives exhibit different temporal contrast variation trends. Specifically, the contrast of background false positives is relatively stable, while that of small targets fluctuates significantly.

(2) A novel visual neural network based on contrast features has been proposed for detecting small moving targets and suppressing background false positives.

(3) The experimental results indicate that the proposed $\mathcal{CI}$STMD visual network achieves better performance than comparable state-of-the-art algorithms.

The presentation of this article is structured as follows. A review of the pertinent literature is presented in Section 2. Our visual system is elaborated upon in Section 3. Section 4 and Section 5 detail a wide range of experimental outcomes to examine the efficiency and behavior of our visual system. Lastly, Section 6 provides a summary of the findings.

2. Related Work

In this section, we mainly review previous works about neural models based on motion-sensitive neurons and some methods for infrared small object detection. Finally, we briefly review research works about using contrast information for object detection.

2.1. Neural Models Based on Motion-Sensitive Neurons

Physiologists have found that insects in nature can accurately perceive various motion stimuli from their visual field, including collision visual stimuli, wide region motion stimuli, and small target motion stimuli. These excellent talents come from the special three classes of motion-detecting neurons in the visual processing system of insects, including the lobula giant movement detector (LGMD) [36,37], lobula plate tangential cell (LPTC) [38,39], and STMD [12]. Relevant studies present that LGMD neurons respond strongly to the object approaching while showing minor attention to the receding object. Some neural visual networks [40,41] have been designed to detect collisions based on the function of LGMD neurons. In the study of insect visual systems, scholars discovered that the LPTC can perceive wide-field stimuli when objects take up a significant portion of the visual field. Based on the functions of the LPTC, some models were designed to detect wide-field motion, including the EMD [42], two-quadrant-detector (TQD) [43], and weighted-quadrant-detector [44]. These visual systems are highly effective at detecting both collision events and motion across a large visual area. However, they are all ineffective in detecting and distinguishing small objects since they cannot discriminate the size of the object. STMD is also a special class of neuron and can accurately perceive small object motion stimuli. Based on the responsiveness of cells to small moving objects, some STMD-related visual systems, such as ESTMD [16], DSTMD [17], feedback STMD [20], $\mathcal{F}$STMD [21], and fractional-order STMD [22], have been proposed for identifying small moving objects in various complex scenarios. These models exhibit promising performance in detecting small targets within complex dynamic scenes. However, they struggle to differentiate between small objects and background clutter.

2.2. Motion Detection Based on Contrast Feature

The real visual world is extremely complex, and there are many motion cues that may be used to help detect the motion detection, such as the contrast information. Contrast denotes the level of luminance disparity between the object and the backdrop within an image and is utilized to ascertain object movement in the domain of computer vision. For example, Han et al. proposed a detection algorithm that operates at multiple levels, utilizing local contrast to identify infrared small targets in complex backgrounds [45]. Xia et al. put forward a multiscale local contrast assessment approach derived from the brightness variation between small targets and the surrounding backgrounds to delineate small objects from the backgrounds in infrared images [46]. Guan et al. advanced the local contrast analysis to enhance little objects and suppress complex backgrounds through the gray-scale aspects of local images, thereby increasing the computational throughput and acceleration of detection [47]. In addition, the cumulative research shows that using contrast information can improve the performance of the systems, including salient object detection [48], contrast enhancement [49], and visual segmentation [50]. However, little work has been carried out on small target motion detection by combining motion and contrast information in the STMD vision system.

2.3. Infrared Small Object Detection

The main objective of infrared small object detection is to pinpoint small, heat-emitting objects like missiles and bombs. For the past 10 years, numerous procedures have been presented and developed for the study of infrared small object detection. For example, Zhang et al. [51] developed a new infrared patch tensor model for detecting infrared small targets. This model employs a novel constraint that combines the tensor kernel norm with a weighted $L_1$ norm, effectively suppressing the background and enhancing the detection performance. Nie et al. [52] introduced a multiscale local uniformity measure to distinguish small targets from the infrared background, and the results demonstrated that the model exhibited outstanding performance. Zhang et al. [53] proposed a method to improve the detection effect of small targets in the infrared image via the target intensity and gradient. In addition, some scholars have proposed other methods, such as asymmetric contextual modulation [54], dense nested attention network [55], and nonconvex tensor fibered rank approximation methods [56], etc., to achieve the identification of infrared minor objectives. While these techniques successfully differentiate the movement of minor objectives, their detection outcomes are heavily dependent on the thermal contrast between the background and the object. Furthermore, identifying minor objects in infrared images always requires a distinct background, a situation that is uncommon in natural environments.

3. Model Description

The proposed visual system contains two modules, where one is a motion detection module, and the other is a contrast detection module. The external visual brightness signal is firstly received and smoothed by the ommatidia, and then further transmitted into the neurons with different functions in the motion information detection channel and contrast information detection channel to detect the position information and contrast information of small features, respectively. Finally, the STMD neuron integrates position information and contrast information to separate small objects from background small-target-like objects (see Figure 4b). The information processing of the $\mathcal{CI}$STMD network is illustrated in Figure 4a. When entering a video frame, the visual network ascertains the positions of small targets and false features by calculating the brightness change of each pixel over time t in the motion information detection channel while obtaining contrast information by calculating the brightness information of each pixel and the brightness information of its surrounding image points. The position information and contrast information are further processed in STMD neurons to record motion trajectory and contrast trajectory, and to calculate contrast variance in each contrast trajectory at each time step. Finally, the network uses contrast variance as the criterion for discrimination, combined with threshold setting, to achieve robust detection of small objects. Our idea stems from the following observation: When a small object moves against the background, its contrast information fluctuates significantly over time. In contrast, the contrast information of fake features, which is static relative to the background, changes very little over time. Based on the above findings, we propose a new discrimination method: By calculating the standard deviation of contrast information over time t, small moving objects can be effectively distinguished.

3.1. Motion Information Detection Channel

In this subsection, we will introduce the mathematical representation of the motion information detection channel in detail.

3.1.1. Ommatidia

As illustrated in Figure 1b, ommatidia are crucial components of the pathway for detecting motion information [57,58] and function as sensory receptors to detect the brightness value within the actual visual environment. In addition, they are designed with Gaussian filters for processing the information with Gaussian blurring. Mathematically, we define a function $I(x,y,t)$ on the real number domain to describe the light stimuli received by the ommatidia neurons. The definition of ommatidia responses outputs $P(x,y,t)$ as shown below:

(1)$P(x,y,t)=\int \int I(u,v,t)G_{{\sigma }_1}(x-u,y-v)dudv,$

where

(2)$G_{{\sigma }_1}(x,y)={\displaystyle \frac{1}{2\pi {\sigma }_1^2}}{e}^{{\displaystyle \frac{-(x^2+y^2)}{2{\sigma }_1^2}}}$

is a Gaussian kernel, and ${\sigma }_1$ denotes the standard variance.

3.1.2. Large Monopolar Cell (LMC)

As presented in Figure 1b, the large monopolar cell (LMC) is a specialized type of neuron that receives information from the ommatidia cells and is highly sensitive to variations in light intensity [16,17]. The dynamic movement of an object through a pixel in the image results in a progressive change in the pixel’s brightness across time t. In the proposed model, LMCs serve as information processors, performing time–frequency analysis on the output of the ommatidia cells to extract the brightness variation information that is crucial for the visual system (see Figure 4a). That is,

(3)$L(x,y,t)=\int P(x,y,s)H(t-s)ds.$

$L(x,y,t)$ represents the LMC response function and $H\left(t\right)$ is expressed as

(4)$H\left(t\right)={\Gamma }_{n_1,{\tau }_1}\left(t\right)-{\Gamma }_{n_2,{\tau }_2}\left(t\right)$

(5)$\begin{array}{c}\hfill {\Gamma }_{n,\tau }\left(t\right)=\left\{\begin{array}{cc}{\left(nt\right)}^n{\displaystyle \frac{e^{\frac{-nt}{\tau }}}{(n-1)!·{\tau }^{n+1}}}\hfill & \hfill \mathrm{for}t\ge 0\\ 0,\hfill & \hfill \mathrm{for}t<0,\end{array}\right.\end{array}$

where n and $\tau$ denote two parameters of Gamma function [59].

3.1.3. Tm3 and Tm1 Cells

As illustrated in Figure 1b, Tm3 and Tm1 neurons are downstream neurons of the LMC, with each one particularly reacting to increments and decrements in brightness, respectively [60]. The Tm3 neuron is particularly sensitive to increases in brightness, while the Tm1 neuron responds significantly to decreases in brightness. In the motion information detection channel, Tm3 and Tm1 neurons act like biological half-wave rectifiers, rectifying the neural signals transmitted by the LMC, thereby enabling the parallel processing of different types of visual information. Let $S^{+}(x,y,t)$ signify Tm3 neuron output, which is calculated by selecting the positive component of the LMC response $L(x,y,t)$. The $S^{+}(x,y,t)$ is expressed as

(6)$S^{+}(x,y,t)={\left[L(x,y,t)\right]}^{+}.$

In addition, findings suggest that the Tm1 neuron’s output response lags behind that of the Tm3 neuron at the same spatial locations $(x,y)$ (see Figure 4a). Therefore, the Tm1 neuron response function $S^{-}(x,y,t)$ is described as

(7)$S^{-}(x,y,t)=\int {\left[L(x,y,s)\right]}^{-}{\Gamma }_{n_3,{\tau }_3}(t-s)ds,$

where ${\left[x\right]}^{+}={\displaystyle \frac{\left|x\right|+x}{2}}$, ${\left[x\right]}^{-}={\displaystyle \frac{\left|x\right|-x}{2}}$.

3.1.4. STMDs

STMD neurons play a key role in our visual neural network by integrating the visual information provided by Tm3 and Tm1 cells, enabling accurate localization of moving objects [61,62]. We calculate STMD output by multiplying $S^{+}(x,y,t)$ with $S^{-}(x,y,t)$ to generate a big response at position $(x,y)$ (see Figure 4a). That is,

(8)$S_B(x,y,t)=S^{+}(x,y,t)\times S^{-}(x,y,t).$

To prevent sizeable objects from interfering with the system, the STMD neuron output $S_B(x,y,t)$ requires a lateral inhibition mechanism to suppress the response of large objects. The response output function after lateral inhibition is defined as follows:

(9)$\begin{array}{c}\hfill D(x,y,t)=\int \int S_B(u,v,t)\widehat{\Theta }(x-u,y-v)dudv,\end{array}$

where function $\widehat{\Theta }(x,y)$ indicates the suppression kernel, and its mathematical expression is as follows:

(10)$\begin{array}{c}\hfill \widehat{\Theta }(x,y)=\phi {\left[\tilde{G}(x,y)\right]}^{+}+\psi {\left[\tilde{G}(x,y)\right]}^{-}\end{array}$

(11)$\begin{array}{c}\hfill \tilde{G}(x,y)=G_{{\sigma }_2}(x,y)-e{G}_{{\sigma }_3}(x,y)-\rho ,\end{array}$

where $\phi ,\psi ,e,\rho$ are constants.

3.2. Contrast Information Detection Channel

The contrast information detection channel contains two special types of neurons, including AMCs [63] and T1 neurons [64,65] (see Figure 1b). Cumulative research has shown that AMCs are connected to ommitidia neurons through elongated fibers, and also with downstream T1 neurons. However, the working mechanism of AMCs in the visual system of insect small object movement identification is still unclear. Therefore, in this paper, we apply AMCs and T1 neurons to form a feature extraction channel by extracting the contrast information of the features to pinpoint objects from the background false traits. As shown in Figure 1b, in the contrast information detection channel, AMCs first receive the ommatidia outputs and then further flow into the T1 neuron to calculate the target contrast. Next, we will introduce the mathematical expression of the contrast information detection module.

3.2.1. AMCs

In the contrast information detection path, the AMC can directly obtain information from the ommatidia output and integrate local pixel brightness information. In our system, AMCs are categorized into two types: One extracts the total brightness within a local neighborhood centered at each spatial location, and the other extracts the total brightness within the surrounding local neighborhoods excluding the central one. Let $A_T(x,y,t)$ represent the output information of the AMC cell receiving the brightness of the central local neighborhood, and $A_B(x,y,t)$ represent the output information of the neuron receiving the brightness of surrounding local neighborhoods excluding the central one. They are defined as follows:

(12)$A_T(x,y,t)={\Sigma }_{(x,y)\in \Phi }P(x,y,t),$

(13)$A_B(x,y,t)={\Sigma }_{(x,y)\in \Pi }P(x,y,t)-{\Sigma }_{(x,y)\in \Phi }P(x,y,t),$

where $\Phi ,\Pi$ represent the object area and the background area, respectively, and $\Phi =\left\{\right(x,y\left)\right|x\in [x-k,x+k],y\in [y-k,y+k]\}$, $\Pi =\left\{\right(x,y\left)\right|x\in [x-z,x+z],y\in [y-z,y+z]\}$, $k,z$ are constants, and $k<z$.

3.2.2. T1 Neuron

As shown in Figure 1b, the T1 neuron accepts AMC cell output and calculates the local contrast information of each position in time t. Here, we define the local contrast output $C(x,y,t)$ at position $(x,y)$ in time t by

(14)$C(x,y,t)=|{\displaystyle \frac{1}{N_{\Phi }}}{A}_T(x,y,t)-{\displaystyle \frac{1}{N_{\Pi }-N_{\Phi }}}{A}_B(x,y,t)|/255,$

where $N_{\Phi }$ represents the total number of pixels in the selected central region, and $N_{\Pi }-N_{\Phi }$ represents the total number of pixels in the selected surrounding region.

3.3. Information Intergation

3.3.1. Record the Motion Trajectory

In the $\mathcal{CI}$STMD model, we apply movement information channel outputs to track the locations of small traits and record the movement trajectories of small traits. The process of recording the motion trajectory is described in Figure 5. We give an identification criterion $\vartheta$, and the small feature position is determined by comparing the threshold $\vartheta$ with the motion detection channel outputs. For the threshold $\vartheta$ and fixed time point $t_0$, when the motion detection channel outputs $D(x_0,y_0,t_0)$ at location $(x_0,y_0)$ in time $t_0$ satisfy $D(x_0,y_0,t_0)>\vartheta$, then, we conclude that a little feature is tracked on site $(x_0,y_0)$. Analogously, at the next time point $t_1$, the another location $(x_1,y_1)$ can be tracked and recorded. In particular, when position $(x_1,y_1)$ at time point $t_1$ is in a small area of position $(x_0,y_0)$ at time point $t_0$, we judge that position $(x_1,y_1)$ and position $(x_0,y_0)$ belong to the same trajectory. Iterating the aforementioned steps, a movement trajectory can be obtained throughout a duration. The movement traces $TRC$ are depicted by

(15)$\begin{array}{c}\hfill TRC=(x\left(t\right),y\left(t\right)),t\in [t_0,t_i].\end{array}$

Here, $\left(x\right(t),y(t\left)\right)$ represents the coordinates of the pixel as it moves over time t. $t_0$ and $t_i$ denote the initial and present times of the motion, respectively.

3.3.2. Contrast Trajectory Acquisition

Applying the detected motion trajectories, we are able to search the contrast traces based on the position of each motion trajectory and the contrast output of the T1 neuron. The contrast motion trajectories $CRC$ are provided by

(16)$\begin{array}{c}\hfill CRC=(C\left(x\left(t\right)\right),C\left(y\left(t\right)\right)),t\in [t_0,t_i].\end{array}$

After searching contrast motion trajectories $CRC$, we evaluate the variance of contrast values $C_V$ of each trajectory at every time t. The $C_V$ is depicted by

(17)$\begin{array}{c}\hfill C_V=(x\left(t\right),y\left(t\right),t,CRC),t\in [t_0,t_i].\end{array}$

We identify small objects by evaluating the contrast variance $C_V(x,y,t)$ at each instant t against a predetermined threshold $\upsilon$. When the contrast variance at a pixel $(x_0,y_0)$ at time $t_0$ exceeds the threshold, we consider a small object to be detected.

4. Experiment Result and Analysis

To comprehensively evaluate the effectiveness of our proposed method, we carried out comprehensive experiments on both simulated [66] and real image sequence datasets [67]. The simulation image data sequences include some synthesis image sequences data via applying complex background images and a small target generated by the computer, which presents the scene of the small target moving in complex backgrounds. All simulation image data sequences are sampled at a frequency of 1000 Hz, that is, 1000 frames of video images are generated per second. The video sample in our experiment is 800 frames. The RIST dataset contains some real videos obtained by capturing the various moving targets and environments with a high-definition camera. The RIST dataset covers several types of scenarios, including significant clutter, camera motion, and variations in overall brightness. The real video sample in our experiment is 750 frames. The experiments are conducted using MATLAB R2017a on a machine with Windows 10, featuring an i7 CPU at 3.10 GHz and 16 GB of memory. As shown in Table 1, the parameters of the $\mathcal{CI}$STMD network are presented, and the motion detection Chennel parameters are determined via sensitivity analysis in [17]. These variables match the speed and size requirements of STMD neurons when responding to moving stimuli.

4.1. STMD Neuron Features

To demonstrate the STMD neuron features, we present the output for objects with varying motion velocities, horizontal widths, and vertical heights, as well as Weber contrast values. As seen in Figure 6a, for a small object sized $w\times h$, the background area is set to $(w+2d)\times (h+2d)$. The initial conditions for Weber contrast, speed, horizontal extension, and vertical extension are defined as 1, 250 pixel/s, 5 pixels, and 5 pixels, respectively. Figure 6b illustrates the STMD responses under different Weber contrasts. We can obtain that the STMD output is increase with increasing the Weber contrast of the target, and the illustration shows that the STMD output becomes maximum when the Weber contrast of the target reaches 1. This indicates that when the moving object’s Weber contrast is higher, the moving object is tracked more easily. The STMD response result to the different motion velocities of the moving object is presented in Figure 6c. Notably, the preferred velocity range of STMD neurons is 200 pixel/s–300 pixel/s and the optimal velocity is 250 pixel/s. The STMD responses under different target widths and heights are displayed in Figure 6d,e. The STMD neuron appears to preferentially respond to moving objects with equal widths and heights of $5\times 5$ pixels. The above four images demonstrate the features of the STMD neuron revealed by biological research, including its sensitivity to Weber contrast and selectivity for velocity, width, and height.

4.2. The Working Mechanism of Motion Information Channel

To demonstrate the working mechanism of the $\mathcal{CI}$STMD model precisely, we use a simulation image data sequence generated by vision agg to help us visualize the information processing flow of the system. Figure 7a provides a visual representation of a data image from the simulation dataset, showing the small object and complex background moving in reverse orientations. The object and background exhibit speeds of 250 pixels per second and 150 pixels per second, respectively, while the indicators $V_T$ and $V_B$ represent their respective movement orientations. Moreover, the tree and the artificial characteristic are regarded as a large object and a false positive in the background, both moving in concert with the backdrop. Figure 7b visually displays the target’s motion trajectory. We take a frame of the test video at time $t_0=500$ ms, and display each neuron response output of x concerning $y_0=$ 12,668 pixels. The incoming brightness data $I(x,y_0,t_0)$ and ommatidia response $P(x,y_0,t_0)$ are displayed in Figure 8a–d, where Figure 8a,c signifies the input brightness and ommatidia response that includes a column with a true small target, Figure 8b,d denotes the input brightness and ommatidia response that includes a column with a fake target. It is easily identifiable that the input stimuli has a lot of backdrop noise and the true small object is hidden in the detailed backdrop interference. It is powerless to discriminate small target motion from backdrop noise. From Figure 8c,d, we can see that the ommatidia output signal is smoother than the input signal. This is because the ommatidia output signal is obtained by Gaussian denoising in the ommatidia neuron. As demonstrated in Figure 8e,f, the LMCs’ output signals are extremely complex and they are generated by computing the change in ommatidia output signal over time t. A positive value signal from the LMC reflects an increase in brightness, while a negative value signal indicates a decrease in brightness at time $t_0$. Figure 8e,f only shows the change of the signal at time $t_0$ and does not point out the traits of the small object, so it cannot be used to identify the movement of the small target or suppress the false target.

The Tm3 and Tm1 neuron outputs are given in Figure 9a–d. The Tm3 and Tm1 neuron’s output signals are acquired by separating the positive and negative signals of LMCs’ output signals. We observe that the Tm3 neuron output is determined by the upward component of the LMC response result, whereas the Tm1 neuron output reflects the negative component of the LMC response result in its delayed state. Within the motion detection chennel, the STMD processes the signals emitted by the Tm3 and Tm1 neurons, with the result displayed in Figure 10a,b. We find that both the real small target ($x=170$) and the fake targets in the background produce strong responses in the STMD. This indicates that the STMD is unable to effectively distinguish between the small target and background noise, suggesting the need to introduce additional features to aid in the classification.

4.3. The Working Mechanism of Contrast Detection Channel

In our work, we design a contrast information detection channel to help extract the contrast information of small features and combine it with a motion detection channel for discriminating small moving objects and background false positives. To confirm the validity of adding channels, we first explain the working mechanism of the contrast detection channel, then assess the performance including and excluding the contrast detection channel. Figure 11a,b gives the contrast information of the small target and fake feature, where Figure 11a,b denotes the contrast outputs to the small target and fake feature on time $t_0=500$ ms, and Figure 11c,d represents the change curve of contrast during the period. As can be seen, the contrast information of the small object and fake feature in fixed time $t_0=500$ ms is cluttered and it is helpless to identify small target motion and fake feature motion. Meanwhile, both the contrast fluctuation curves, of the true object and background false positives, across time are discriminative. More specifically, the contrast fluctuation curve of the small object exhibits a clear trend of ups and downs over time; on the contrary, the contrast change curve of a fake feature remains invariable.

As a result, small objects can be separated from background fake features based on the intrinsic differences in contrast variation over time. The contrast variance is applied to characterize these inherent differences in the contrast change curve, helping us identify small moving objects among the background fake features. Figure 12a,c presents the trajectories of small objects (blue curve) and fake features (red curve) recorded by our approach, excluding the contrast detection channel with diverse detection parameters $\beta$.

It can be easily observed that these trajectory results consistently include a high number of false positives trajectories, and the number of false positives trajectories gradually decreases as the threshold parameter increases. Meanwhile, the motion trajectories of small targets and fake features recorded by the $\mathcal{CI}$STMD model including the contrast detection channel across various thresholds parameters $\beta$ are presented in Figure 12d–f. As observed, a significant number of fake features is eliminated by the contrast detection channel at different thresholds. As shown in Figure 13a–c, the trajectories of movement identified by the ESTMD, DSTMD, and $\mathcal{F}$STMD methods are presented. It is evident that their logged data are exceedingly convoluted because of the multitude of spurious attributes, which renders the detection of small targets from these results unfeasible. It is clear that, owing to the interference of numerous false features, the original detection results are extremely chaotic, making it difficult to accurately extract the small target. By adjusting the contrast variance threshold $\gamma$, the $\mathcal{CI}$STMD effectively suppresses false features, improving the purity of the detection results. As shown in Figure 13d–f, as the value of $\gamma$ increases, the detected motion trajectories become clearer, and the interference gradually decreases. We further show the contrast variance curves of a small moving object and background fake feature over time t in Figure 14a,b. As shown in Figure 14a,b, the contrast variance of the small object exhibits a trend of initially increasing and then stabilizing, ultimately maintaining a high level, but the contrast variance to the fake feature is close to zero and hardly changes over the time t. We present the output of the STMD after thresholding in Figure 14c,d. By comparing the STMD output results after thresholding, we found that the false positive responses generated by the background were effectively suppressed, while the responses of small targets were retained. The results from the above experiments indicate that an appropriate threshold allows us to differentiate small moving objects from background fake features.

5. Comparative Analysis

In this section, we execute some performance comparison experiments to highlight the advantages of $\mathcal{CI}$STMD network. To quantitatively assess the model’s detection capability, we assess the identification capabilities of four current STMD-based frameworks, namely, $\mathcal{CI}$STMD, ESTMD [16], DSTMD [17], and $\mathcal{F}$STMD [21]. We will evaluate each model’s detection performance by plotting the ROC curve of $D_A-F_A$, and the detection rate $D_A$ and false alarm rate $F_A$ are depicted as follows:

(18)$D_R={\displaystyle \frac{T_D}{T_N}},F_A={\displaystyle \frac{B_D}{I_N}},$

where $T_D$, $T_N$, $B_D$, and $I_N$ represent the number of true detections, number of targets, number of false detections, and number of images, respectively.

First, we conduct a comparative study of the proposed visual network with the ESTMD, DSTMD, and $\mathcal{F}$STMD models across different video simulation variables, including object velocities, sizes, brightness, background movement velocities, movement orientations, and backdrop images. The information regarding the simulation frame sequences can be found in Table 2. In Figure 15a, the receiver operating characteristic (ROC) curves that depict the detection performance of the comparison models on small objects in the simulated image sequences are presented. As we can see, the performance of $\mathcal{CI}$STMD surpasses that of the other three models. More precisely, $\mathcal{CI}$STMD yields a higher detection rate at different $F_A$ levels. In Figure 15b–f, we provide the ROC curves of the comparison networks under different simulation video parameter conditions, where $F_A=5$ to maintain a fair comparison. As exhibited in Figure 15b, the ROC curves of the comparison models for detecting various sizes object in the simulation image sequence are given. The $\mathcal{CI}$STMD model clearly shows superior detection capabilities over the other three models when evaluated using different object sizes, varying from $1\times 1$ to $11\times 11$ pixels. As presented in Figure 15c, $\mathcal{CI}$STMD performs best according to different object brightnesses. More exactly, the ROC curves of the four models all decrease when the object brightness ranges from 0 to 0.15. Figure 15d demonstrates the ROC curves for different moving object velocities. It can be seen that $\mathcal{CI}$STMD significantly outperforms the ESTMD, DSTMD, and $\mathcal{F}$STMD models. We can find that $\mathcal{CI}$STMD has a higher $D_A$ compared to the other visual networks, while the $F_A$ is low when target velocity increases from 0 to 300 pixels per second. As shown in Figure 15e,f, the $\mathcal{CI}$STMD model has a higher performance level than the ESTMD, DSTMD, and $\mathcal{F}$STMD models in diverse conditions of background velocity and movement orientations.

As presented in Figure 16, we also present the ROC curve of the contrastive model to identify true small objects in various complex movement scenarios. Figure 16a,c,e presents a representative frame from each of three video sequences, wherein a small object can be observed moving within a complex, real-world background. The arrows $V_B$ and $V_T$ signify the movement orientations of the dynamic backdrop and the true object, respectively. Figure 16b,d,f depicts the ROC curves for four models, each tasked with the detection of small objects within complex and fluctuating background circumstances. Our proposed visual neural network exhibits superior performance in comparison to the other three models.

In addition, we also present the effectiveness of the $\mathcal{CI}$STMD model to detect small objects in real datasets. Figure 17 shows the different image frames of real datasets and the detection performance of four models in detecting small objects in real datasets. Figure 17a–c shows image frames 300 and 600 of three different real datasets and depicts the ROC curves of comparison models when applied to a variety of real-world datasets. The results demonstrate that the $\mathcal{CI}$STMD model exhibits superior detection performance compared to the ESTMD, DSTMD, and $\mathcal{F}$STMD models, while maintaining the same false alarm rate.

We further conducted comparative experiments of our model with methods such as optical flow, background subtraction, feedback STMD [20], fractional-order STMD [22], and apg-STMD [23]. The experimental results are shown in Table 3. When $F_A=5$, our model demonstrates outstanding performance in detection, with an average detection rate of $0.81$, which is significantly higher than that of other methods (all below 0.7). This indicates that the $\mathcal{CI}$STMD visual network has stronger robustness in detecting small target movements in complex dynamic environments.

Finally, we compared the precision and F1 scores of different models in Table 4 and Table 5. The results show that our model performs excellently in suppressing background false positives, with an average precision as high as 0.09, which is significantly higher than the average level of other models (all below 0.05). In addition, our model also shows an advantage in terms of F1 score. Our average F1 score is higher than 0.1, reaching 0.1705, while the F1 scores of other methods are all below 0.1. This indicates that our model has a better ability to identify moving targets. The above experiments indicate that the $\mathcal{CI}$STMD model is more robust and stable when dealing with complex and varying background environments and different types of targets.

6. Conclusions

In this paper, we propose a visual network that distinguishes small moving objects in dynamic, complex environments by utilizing the temporal feature differences in contrast between the target and background false positives. The visual neural network consists of two channels: one for motion information detection and another for extracting local contrast information. The network extracts target position information and motion trajectories through the motion information detection channel, and obtains the contrast information of motion features through the contrast information extraction channel. Subsequently, the network uses the target motion trajectory to determine the contrast trajectory and calculates the contrast variance. Finally, by utilizing the contrast feature fluctuations between the target and background false positives and applying a threshold, false positives are suppressed, enabling the detection of small target motion. To assess the effectiveness of our approach, we conducted a series of experiments on both synthetic and real-world datasets. The empirical results clearly indicate that our visual network significantly outperforms existing methods in terms of suppressing false alarms and improving detection accuracy. Contrast information proves highly effective in small target detection when there is a marked disparity between the target and the background. Nonetheless, the model’s detection capabilities are significantly undermined when the target and background have similar brightness levels. Therefore, in the future, it may be worthwhile to consider integrating other visual information, such as entropy, to address the issue of difficulty in detecting small targets when their brightness aligns with that of the background.

Author Contributions

Methodology, J.L.; validation, J.L.; investigation, H.M.; data curation, J.L. and H.M.; writing—original draft preparation, J.L.; writing—review and editing, J.L., H.M. and D.G.; visualization, J.L.; supervision, H.M. and D.G.; project administration, H.M. and D.G. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Not applicable.

Informed Consent Statement

Not applicable.

Data Availability Statement

The data presented in this study are available on request from the corresponding author.

Conflicts of Interest

Author Deming Gong was employed by the company Mltor Numerical Control Technology Limited Company. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Footnotes

Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.

Figures and Tables

Figure 1. (a) When an object is located at a significant distance from the imaging device, it occupies just a single pixel or a small number of pixels and lacks distinct physical features. The white circle of the image represents the small target (flying plane), and the top right corner red circle of the image represents the larger view of the small target. (b) The diagram of neuron information connection of the proposed visual network. The proposed visual network involves two parts, where one is the motion information detection channel, and the other is the contrast information detection channel. The external visual information is processed via ommatidia, LMC, TM3, TM1, and STMD neurons for detecting small targets’ motion in the motion information detection channel. Meanwhile, the contrast information of the object is extracted by AMC and T1 neurons in the contrast information detection pathway. Integrating contrast information and motion information in STMD neurons allows the detection of small objects’ motion.

Figure 2. (a) The working mechanism of the motion information detection channel. When a small target passes through a pixel, the ommatidium output first decreases and then increases, eventually returning to its initial level. This change is caused by the entry and exit of the small target, respectively. The LMC cell perceives the luminance change and transmits the signal to Tm3 and Tm1 neurons. The Tm3 neuron responds immediately, while the Tm1 neuron has a delayed response. Finally, the STMD produces a strong response by multiplying the outputs of Tm3 and Tm1 neurons at the same time position. (b) The working mechanism of contrast information detection channel. When the small target passes through a pixel, the AMC cells first compute the brightness information of the target region and the surrounding background region, and then transmit this information to the T1 neurons to calculate the contrast information of the target.

Figure 3. The working mechanism of the [Forumla omitted. See PDF.]STMD model. The [Forumla omitted. See PDF.]STMD model first obtains the positions of all features in extracting motion information by the STMD visual neural pathway and obtains the contrast information by calculating the contrast of all pixels. Then, the [Forumla omitted. See PDF.]STMD model estimates the motion trajectory of possible targets by the position information of all features, and records the contrast information traces based on motion trajectory and contrast information. Finally, based on contrast information traces, we calculate the variance of each trace and apply the variance to help us distinguish the small object and false positives.

Figure 4. (a) [Forumla omitted. See PDF.]STMD model working diagram. Visual information is processed through a cascade of different types of neurons in the nervous system, ultimately leading to visual perception. (b) The left image shows the motion of a small object and a small-target-like object in a complicated background. The contrast between the small target and the background exhibits temporal variation, while the small-target-like object, being hidden in the background, maintains almost constant contrast with the background. The upper right figure displays that the STMD neurons generate different contrast response curves to time-varying and time-fixed contrast features. The lower right figure displays the variances of different contrast responses, in which the variance curve of the time-varying contrast target is higher, and the variance curve of the time-fixed contrast target is lower. Therefore, we can select contrast response variance as a criterion to help us distinguish the small target and false positives.

Figure 5. Upper figure depicts the process of extracting the motion trajectory of small features. We continuously record the positions of small objects, and when two consecutively detected positions are close enough, we consider them to belong to the same trajectory. Using this approach, we can track the movement path of an object over a period of time. The following figure displays the complete trajectory extracted by the system.

Figure 6. (a) The spatial dimensions of the small target and its surrounding local neighborhood, where w and h denote the width and height of the target, respectively. (b–e) The STMD model’s responsiveness to the dynamics of objects with assorted Weber contrast, movement velocity, width, and height attributes.

Figure 7. (a) A [Forumla omitted. See PDF.] pixel frame sampled from the synthetic dataset. A small, dark object (the target) is observed moving in a direction opposite to the complex background. Both the target and background exhibit constant velocities of 250 and 150 pixels per second, respectively. The markers [Forumla omitted. See PDF.] and [Forumla omitted. See PDF.] denote the direction of movement for the true object and the backdrop, respectively. (b) The trajectory of the small target over a duration of 800 ms.

Figure 8. (a) Input brightness profile. (b) Input brightness profile of fake target. (c) Ommatidial neural response. (d) Ommatidial neural response of fake target. (e) LMC neuron response. (f) LMC neuron response of fake target.

Figure 9. (a) Tm3 neural response. (b) Tm3 neural response of fake target (dotted line). (c) Tm1 neural response. (d) Tm1 neural response of fake target (dotted line).

Figure 10. (a) STMD neural response. (b) STMD neural response of fake target (dotted line).

Figure 11. (a,b) Contrast information of the small target and fake features at a specified time point, [Forumla omitted. See PDF.] ms. (c,d) Temporal evolution of contrast information of the small target and fake features, observed over the time interval [Forumla omitted. See PDF.] ms.

Figure 12. (a–c) Motion trajectories tracked by the [Forumla omitted. See PDF.]STMD model without the incorporation of a contrast information detection channel, under various thresholds [Forumla omitted. See PDF.] of [Forumla omitted. See PDF.], and 100, respectively. (d–f) Motion trajectories tracked by the [Forumla omitted. See PDF.]STMD model with the integration of a contrast information detection channel, under various thresholds [Forumla omitted. See PDF.] of [Forumla omitted. See PDF.], and 100, and a fixed contrast information standard deviation threshold [Forumla omitted. See PDF.] of [Forumla omitted. See PDF.].

Figure 13. (a–c) Motion trajectories tracked by three different models: ESTMD, DSTMD, and [Forumla omitted. See PDF.]STMD. (d–f) The trajectories of movement, as sensed by the [Forumla omitted. See PDF.]STMD approach, are investigated across a range of contrast information variance parameters [Forumla omitted. See PDF.], at [Forumla omitted. See PDF.], and [Forumla omitted. See PDF.].

Figure 14. The contrast information variances under different time. (a) The contrast information variances of the small target. (b) The contrast information variances of the fake feature. (c,d) Output signal after applying a contrast threshold.

Figure 15. (a) The receiver operating characteristic (ROC) curve of the [Forumla omitted. See PDF.]STMD model applied to the initial test video sequence. (b–f) The ROC curves that detail the outcomes of four different methodologies on the original sequence across multiple object size variations, brightness scales, motion velocities, and background motion velocities, with a constant false alarm rate of 5.

Figure 16. (a,c,e) A sample frame from each of three test video sequences. A small moving object can be observed against a complex, real-world background. The arrows [Forumla omitted. See PDF.] and [Forumla omitted. See PDF.] signify the movement orientation of the background and object, respectively. (a) Synthetic video one; (c) synthetic video two; (e) synthetic video three. (b,d,f) The receiver operating characteristic (ROC) curves illustrating the performance of four different methods on three test video sequences.

Figure 17. Representative frames extracted from three real-world datasets and the receiver operating characteristic (ROC) curves illustrating the performance of four different methods on three real-world datasets. (a) Real video one; (b) real video two; (c) real video three.

Table 1

Model coefficients.

Equations	Parameters
(1)	${\sigma }_1=1$
(3)	$n_1=2,{\tau }_1=3,n_2=6,{\tau }_2=9$
(7)	$n_3=5,{\tau }_3=25$
(10)	${\sigma }_2=1.5,{\sigma }_3=3$
(11)	$\phi =1,\psi =3,e=1,\rho =0$
(13)	$k=5,z=25$

Table 2

Parameters of the generated data.

Video Argument	Original Video	Test 1	Test 2	Test 3	Test 4	Test 5
Object size range	$5\times 5$	$1\times 1\sim 11\times 11$	$5\times 5$	$5\times 5$	$5\times 5$	$5\times 5$
Object brightness range	0	0	$0\sim 0.15$	0	0	0
Object velocity range (pixels per second)	250	250	250	$0\sim 300$	250	250
Background velocity range (pixels per second)	150	150	150	150	$100\sim 200$	$100\sim 200$
Movement orientation of background	Towards the right	Towards the right	Towards the right	Towards the right	Towards the right	Towards the left
Background picture	Figure 7a	Figure 7a	Figure 7a	Figure 7a	Figure 7a	Figure 7a

Table 3

Detection rate of different methods ($F_A=5$).

Methods	Background Subtraction	Optical Flow	DSTMD	ESTMD	$\mathcal{F}$STMD	Feedback STMD	Frac-STMD	apg-STMD	$\mathcal{CI}$STMD
Initial video	0.49	0.44	0.36	0.50	0.57	0.66	0.55	0.76	0.87
Synthetic video one	0.50	0.44	0.60	0.69	0.71	0.71	0.73	0.57	0.89
Synthetic video two	0.50	0.43	0.57	0.73	0.74	0.77	0.74	0.75	0.81
Synthetic video three	0.52	0.50	0.7	0.78	0.74	0.83	0.59	0.57	1
Real video one	0.44	0.46	0.08	0.28	0.28	0.19	0.07	0.18	0.56
Real video two	0.51	0.45	0.4	0.55	0.55	0.32	0.33	0.37	0.75
Real video three	0.58	0.48	0.33	0.82	0.81	0.43	0.56	0.49	0.80
Mean	0.51	0.46	0.43	0.62	0.56	0.56	0.51	0.52	0.81

Table 4

Precision of different methods.

Video	Background Subtraction	Optical Flow	DSTMD	ESTMD	$\mathcal{F}$STMD	Feedback STMD	Frac-STMD	apg-STMD	$\mathcal{CI}$STMD
Initial video	0.0010	0.0007	0.0094	0.0088	0.0086	0.0086	0.0165	0.0231	0.1041
Synthetic video one	0.0011	0.0008	0.0101	0.0094	0.0094	0.0095	0.0290	0.0166	0.1723
Synthetic video two	0.0011	0.0009	0.0088	0.0081	0.0079	0.0081	0.0185	0.0129	0.0634
Synthetic video three	0.0011	0.001	0.0204	0.0163	0.0163	0.0178	0.1547	0.0150	0.1814
Real video one	0.0021	0.0005	0.0021	0.0040	0.0040	0.0028	0.0201	0.0134	0.0414
Real video two	0.0008	0.0006	0.0229	0.0244	0.0241	0.0247	0.0407	0.0204	0.0492
Real video three	0.0009	0.0007	0.0230	0.0273	0.0269	0.0236	0.0566	0.0195	0.0663
Mean	0.0012	0.0007	0.0138	0.0140	0.0138	0.0135	0.0480	0.0172	0.0968

Table 5

F1 scores of different methods.

Video	Background Subtraction	Optical Flow	DSTMD	ESTMD	$\mathcal{F}$STMD	Feedback STMD	Frac-STMD	apg-STMD	$\mathcal{CI}$STMD
Initial video	0.0020	0.0016	0.0187	0.0175	0.0170	0.0170	0.0324	0.0451	0.1885
Synthetic video one	0.0022	0.0017	0.02	0.0186	0.0186	0.0187	0.0563	0.0308	0.2939
Synthetic video two	0.0021	0.0018	0.0175	0.0160	0.0157	0.0161	0.0363	0.0255	0.1185
Synthetic video three	0.0022	0.0020	0.04	0.0321	0.0322	0.0350	0.2602	0.0290	0.3071
Real video one	0.0022	0.0005	0.0041	0.0080	0.0079	0.0056	0.0392	0.0263	0.0728
Real video two	0.0018	0.0012	0.0446	0.0476	0.0470	0.0481	0.0779	0.0204	0.0921
Real video three	0.0019	0.0014	0.0445	0.0529	0.0521	0.0456	0.1061	0.0382	0.1209
Mean	0.0020	0.0015	0.027	0.0275	0.0272	0.0265	0.0807	0.0308	0.1705

References

1. Escobar-Alvarez, H.D.; Ohradzansky, M.; Keshavan, J.; Ranganathan, B.N.; Humbert, J.S. Bio-inspired approaches for autonomous small-object detection and avoidance. IEEE Trans. Robot; 2019; 35, pp. 1220-1232. [DOI: https://dx.doi.org/10.1109/TRO.2019.2922472]

2. Li, J.; Liang, X.; Wei, Y.; Xu, T.; Feng, J.; Yan, S. Perceptual generative adversarial networks for small object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition; Honolulu, HI, USA, 21–26 July 2017; pp. 1222-1230.

3. Garcia-Garcia, B.; Bouwmans, T.; Silva, A.J.; Imler, E.; Olberg, R.; Leonardo, A. Background subtraction in real applications: Challenges, current models and future directions. Comput. Sci. Rev.; 2020; 517, pp. 333-338. [DOI: https://dx.doi.org/10.1016/j.cosrev.2019.100204]

4. Kalsotra, R.; Arora, S. Background subtraction for moving object detection: Explorations of recent developments and challenges. Comput. Sci. Rev.; 2022; 38, pp. 4151-4178. [DOI: https://dx.doi.org/10.1007/s00371-021-02286-0]

5. Shuigen, W.; Zhen, C.; Hua, D. Motion detection based on temporal difference method and optical flow field. Proceedings of the 2009 Second International Symposium on Electronic Commerce and Security; Nanchang, China, 22–24 May 2009; pp. 85-88.

6. Fortun, D.; Bouthemy, P.; Kervrann, C. Optical flow modeling and computation: A survey. Comput. Vis. Image Underst.; 2015; 134, pp. 1-21. [DOI: https://dx.doi.org/10.1016/j.cviu.2015.02.008]

7. Kačan, M.; Turčinović, F.; Bojanjac, D.; Bosiljevac, M. Deep learning approach for object classification on raw and reconstructed gbsar data. Remote Sens.; 2022; 14, 5673. [DOI: https://dx.doi.org/10.3390/rs14225673]

8. Hou, J.; Ding, H.; Lin, W.; Liu, W.; Fang, Y. Distilling knowledge from object classification to aesthetics assessment. IEEE Trans. Circuits Syst. Video Technol.; 2022; 32, pp. 7386-7402. [DOI: https://dx.doi.org/10.1109/TCSVT.2022.3186307]

9. Ghasemi Darehnaei, Z.; Rastegar Fatemi, S.M.J.; Mirhassani, S.M.; Fouladian, M. Ensemble deep learning using faster r-cnn and genetic algorithm for vehicle detection in uav images. IETE J. Res.; 2023; 69, pp. 5102-5111. [DOI: https://dx.doi.org/10.1080/03772063.2021.1962418]

10. Liang, B.; Luo, H. MEANet: An effective and lightweight solution for salient object detection in optical remote sensing images. Expert Syst. Appl.; 2024; 238, pp. 887-904. [DOI: https://dx.doi.org/10.1016/j.eswa.2023.121778]

11. Liu, J.J.; Hou, Q.; Liu, Z.A.; Cheng, M.M. Poolnet+: Exploring the potential of pooling for salient object detection. IEEE Trans. Pattern Anal. Mach. Intell.; 2022; 45, pp. 887-904. [DOI: https://dx.doi.org/10.1109/TPAMI.2021.3140168]

12. Mischiati, M.; Lin, H.T.; Herold, P.; Imler, E.; Olberg, R.; Leonardo, A. Internal models direct dragonfly interception steering. Nature; 2015; 517, pp. 333-338. [DOI: https://dx.doi.org/10.1038/nature14045]

13. Barnett, P.D.; Nordström, K.; O’carroll, D.C. Retinotopic organization of small-field-target-detecting neurons in the insect visual system. Curr. Biol.; 2007; 17, pp. 569-578. [DOI: https://dx.doi.org/10.1016/j.cub.2007.02.039] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/17363248]

14. Keleş, M.F.; Frye, M.A. Object-detecting neurons in Drosophila. Curr. Biol.; 2017; 27, pp. 680-687. [DOI: https://dx.doi.org/10.1016/j.cub.2017.01.012] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/28190726]

15. Nordström, K.; Barnett, P.D.; O’Carroll, D.C. Insect detection of small targets moving in visual clutter. PLoS Biol.; 2006; 4, e54. [DOI: https://dx.doi.org/10.1371/journal.pbio.0040054] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/16448249]

16. Wiederman, S.D.; Shoemaker, P.A.; O’Carroll, D.C. A model for the detection of moving targets in visual clutter inspired by insect physiology. PloS ONE; 2008; 3, e2784. [DOI: https://dx.doi.org/10.1371/journal.pone.0002784]

17. Wang, H.; Peng, J.; Yue, S. A directionally selective small target motion detecting visual neural network in cluttered backgrounds. IEEE Trans. Cybern.; 2018; 50, pp. 1541-1555. [DOI: https://dx.doi.org/10.1109/TCYB.2018.2869384]

18. Wiederman, S.D.; O’Carroll, D.C. Biologically inspired feature detection using cascaded correlations of off and on channels. J. Artif. Intell. Soft.; 2013; 3, pp. 5-14. [DOI: https://dx.doi.org/10.2478/jaiscr-2014-0001]

19. Wiederman, S.D.; O’Carroll, D.C. Biomimetic target detection: Modeling 2nd order correlation of off and on channels. Proceedings of the 2013 IEEE Symposium on Computational Intelligence for Multimedia, Signal and Vision Processing (CIMSIVP); Singapore, 16–19 April 2013; pp. 16-21.

20. Wang, H.; Wang, H.; Zhao, J.; Hu, C.; Peng, J.; Yue, S. A time-delay feedback neural network for discriminating small, fast-moving targets in complex dynamic environments. IEEE Trans. Neural Netw. Learn Syst.; 2021; 34, pp. 316-330. [DOI: https://dx.doi.org/10.1109/TNNLS.2021.3094205]

21. Ling, J.; Wang, H.; Xu, M.; Chen, H.; Li, H.; Peng, J. Mathematical study of neural feedback roles in small target motion detection. Front. Neurorobot.; 2022; 16, 984430. [DOI: https://dx.doi.org/10.3389/fnbot.2022.984430]

22. Xu, M.; Wang, H.; Chen, H.; Li, H.; Peng, J. A fractional-order visual neural model for small target motion detection. Neurocomputing; 2023; 550, 126459. [DOI: https://dx.doi.org/10.1016/j.neucom.2023.126459]

23. Wang, H.; Zhao, J.; Wang, H.; Hu, C.; Peng, J.; Yue, S. Attention and prediction-guided motion detection for low-contrast small moving targets. IEEE Trans. Cybern.; 2022; 53, pp. 6340-6352. [DOI: https://dx.doi.org/10.1109/TCYB.2022.3170699]

24. Chen, H.; Fan, B.; Li, H.; Peng, J. Rigid propagation of visual motion in the insect’s neural system. Neural Netw.; 2025; 181, 106874. [DOI: https://dx.doi.org/10.1016/j.neunet.2024.106874] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/39522416]

25. Billah, M.A.; Faruque, I.A. Modeling Small-Target Motion Detector Neurons as Switched Systems with Dwell Time Constraints. Proceedings of the 2022 American Control Conference (ACC); Atlanta, GA, USA, 8–10 June 2022; pp. 3192-3197.

26. Uzair, M.; Finn, A.; Brinkworth, R.S. Efficient Sampling of Bayer Pattern for Long Range Small Target Detection in Color Images. Proceedings of the 2023 38th International Conference on Image and Vision Computing New Zealand (IVCNZ); Palmerston North, New Zealand, 29–30 November 2023; pp. 1-5.

27. Uzair, M.; Brinkworth, R.S.; Finn, A. Detecting small size and minimal thermal signature targets in infrared imagery using biologically inspired vision. Neuron; 2021; 21, 1812. [DOI: https://dx.doi.org/10.3390/s21051812] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/33807741]

28. Uzair, M.; Brinkworth, R.S.A.; Finn, A. A bio-inspired spatiotemporal contrast operator for small and low-heat-signature target detection in infrared imagery. Neuron; 2021; 33, pp. 7311-7324. [DOI: https://dx.doi.org/10.1007/s00521-020-05206-w]

29. Freifeld, L.; Clark, D.A.; Schnitzer, M.J.; Horowitz, M.A.; Clandinin, T.R. GABAergic lateral interactions tune the early stages of visual processing in Drosophila. Neuron; 2013; 78, pp. 1075-1089. [DOI: https://dx.doi.org/10.1016/j.neuron.2013.04.024]

30. Behnia, R.; Clark, D.A.; Carter, A.G.; Clandinin, T.R.; Desplan, C. Processing properties of ON and OFF pathways for Drosophila motion detection. Nature; 2014; 512, pp. 427-430. [DOI: https://dx.doi.org/10.1038/nature13427]

31. Ling, J.; Wu, H. A Small Target Motion Detection Algorithm in Complex Dynamic Environment. J. Appl. Numer. Optim.; 2024; 6, pp. 411-427.

32. Tuthill, J.C.; Nern, A.; Holtz, S.L.; Rubin, G.M.; Reiser, M.B. Contributions of the 12 neuron classes in the fly lamina to motion vision. Neuron; 2013; 79, pp. 128-140. [DOI: https://dx.doi.org/10.1016/j.neuron.2013.05.024]

33. Clark, D.A.; Demb, J.B. Parallel computations in insect and mammalian visual motion processing. Curr. Biol.; 2016; 26, pp. R1062-R1072. [DOI: https://dx.doi.org/10.1016/j.cub.2016.08.003]

34. Stöckl, A.L.; Ribi, W.A.; Warrant, E.J. Adaptations for nocturnal and diurnal vision in the hawkmoth lamina. J. Comp. Neurol.; 2016; 524, pp. 160-175. [DOI: https://dx.doi.org/10.1002/cne.23832]

35. Rivera-Alvidrez, Z.; Lin, I.; Higgins, C.M. A neuronally based model of contrast gain adaptation in fly motion vision. Vis. Neurosci.; 2011; 28, pp. 419-431. [DOI: https://dx.doi.org/10.1017/S095252381100023X]

36. Rind, F.C.; Wernitznig, S.; Pölt, P.; Zankel, A.; Gütl, D.; Sztarker, J.; Leitinger, G. Two identified looming detectors in the locust: Ubiquitous lateral connections among their inputs contribute to selective responses to looming objects. Sci. Rep.; 2016; 6, 35525. [DOI: https://dx.doi.org/10.1038/srep35525] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/27774991]

37. Rind, F.C.; Bramwell, D.I. Neural network based on the input organization of an identified neuron signaling impending collision. J. Neurophysiol.; 1996; 75, pp. 967-985. [DOI: https://dx.doi.org/10.1152/jn.1996.75.3.967] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/8867110]

38. Maisak, M.S.; Haag, J.; Ammer, G.; Serbe, E.; Meier, M.; Leonhardt, A.; Schilling, T.; Bahl, A.; Rubin, G.M.; Nern, A. et al. A directional tuning map of Drosophila elementary motion detectors. Nature; 2013; 500, pp. 212-216. [DOI: https://dx.doi.org/10.1038/nature12320] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/23925246]

39. Perry, M.; Konstantinides, N.; Pinto-Teixeira, F.; Desplan, C. Generation and evolution of neural cell types and circuits: Insights from the Drosophila visual system. Annu. Rev. Genet.; 2017; 51, pp. 501-527. [DOI: https://dx.doi.org/10.1146/annurev-genet-120215-035312]

40. Yue, S.; Rind, F.C. Collision detection in complex dynamic scenes using an LGMD-based visual neural network with feature enhancement. IEEE Trans. Neural Netw.; 2006; 17, pp. 705-716.

41. Yue, S.; Rind, F.C. Redundant neural vision systems—Competing for collision recognition roles. IEEE Trans. Auton. Ment. Dev.; 2013; 5, pp. 173-186.

42. Hassenstein, B.; Reichardt, W. Systemtheoretische analyse der zeit-, reihenfolgen-und vorzeichenauswertung bei der bewegungsperzeption des rüsselkäfers chlorophanus. Z. Naturforsch. B; 1956; 11, pp. 513-524. [DOI: https://dx.doi.org/10.1515/znb-1956-9-1004]

43. Clark, D.A.; Bursztyn, L.; Horowitz, M.A.; Schnitzer, M.J.; Clandinin, T.R. Defining the computational structure of the motion detector in Drosophila. Neuron; 2011; 70, pp. 1165-1177. [DOI: https://dx.doi.org/10.1016/j.neuron.2011.05.023]

44. Warrant, E.J. Matched filtering and the ecology of vision in insects. Ecol. Anim. Senses Matched Filters Econ. Sens.; 2016; 11, pp. 143-167.

45. Han, J.; Liang, K.; Zhou, B.; Zhu, X.; Zhao, J.; Zhao, L. Infrared small target detection utilizing the multiscale relative local contrast measure. IEEE Geosci. Remote Sens. Lett.; 2018; 15, pp. 612-616. [DOI: https://dx.doi.org/10.1109/LGRS.2018.2790909]

46. Xia, C.; Li, X.; Zhao, L.; Shu, R. Infrared small target detection based on multiscale local contrast measure using local energy factor. IEEE Geosci. Remote Sens. Lett.; 2019; 17, pp. 157-161. [DOI: https://dx.doi.org/10.1109/LGRS.2019.2914432]

47. Guan, X.; Peng, Z.; Huang, S.; Chen, Y. Gaussian scale-space enhanced local contrast measure for small infrared target detection. IEEE Geosci. Remote Sens. Lett.; 2019; 17, pp. 327-331. [DOI: https://dx.doi.org/10.1109/LGRS.2019.2917825]

48. Zhao, X.; Zhang, L.; Pang, Y.; Lu, H.; Zhang, L. A single stream network for robust and real-time RGB-D salient object detection. Proceedings of the Computer Vision–ECCV 2020: 16th European Conference; Glasgow, UK, 23–28 August 2020; pp. 646-662.

49. Liu, W.; Ren, G.; Yu, R.; Guo, S.; Zhu, J.; Zhang, L. Image-adaptive YOLO for object detection in adverse weather conditions. Proceedings of the AAAI Conference on Artificial Intelligence; Palo Alto, CA, USA, 22 February–1 March 2022; pp. 1792-1800.

50. Zhang, W.; Zhuang, P.; Sun, H.H.; Li, G.; Kwong, S.; Li, C. Underwater image enhancement via minimal color loss and locally adaptive contrast enhancement. IEEE Trans. Image Process; 2022; 31, pp. 3997-4010. [DOI: https://dx.doi.org/10.1109/TIP.2022.3177129] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/35657839]

51. Zhang, L.; Peng, Z. Infrared small target detection based on partial sum of the tensor nuclear norm. Remote Sens.; 2019; 11, 382. [DOI: https://dx.doi.org/10.3390/rs11040382]

52. Nie, J.; Qu, S.; Wei, Y.; Zhang, L.; Deng, L. An infrared small target detection method based on multiscale local homogeneity measure. Infrared Phys. Technol.; 2018; 90, pp. 186-194. [DOI: https://dx.doi.org/10.1016/j.infrared.2018.03.006]

53. Zhang, H.; Zhang, L.; Yuan, D.; Chen, H. Infrared small target detection based on local intensity and gradient properties. Infrared Phys. Technol.; 2018; 89, pp. 88-96. [DOI: https://dx.doi.org/10.1016/j.infrared.2017.12.018]

54. Dai, Y.; Wu, Y.; Zhou, F.; Barnard, K. Asymmetric contextual modulation for infrared small target detection. Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision; Waikoloa, HI, USA, 3–8 January 2021; pp. 950-959.

55. Li, B.; Xiao, C.; Wang, L.; Wang, Y.; Lin, Z.; Li, M.; An, W.; Guo, Y. Dense nested attention network for infrared small target detection. IEEE Trans. Image Process; 2022; 32, pp. 1745-1758. [DOI: https://dx.doi.org/10.1109/TIP.2022.3199107]

56. Kong, X.; Yang, C.; Cao, S.; Li, C.; Peng, Z. Infrared small target detection via nonconvex tensor fibered rank approximation. IEEE Trans. Geosci. Remote Sens.; 2021; 60, pp. 1-21. [DOI: https://dx.doi.org/10.1109/TGRS.2021.3068465]

57. Borst, A.; Helmstaedter, M. Common circuit design in fly and mammalian motion vision. Nat. Neurosci.; 2015; 18, pp. 1067-1076. [DOI: https://dx.doi.org/10.1038/nn.4050]

58. Sanes, J.R.; Zipursky, S.L. Design principles of insect and vertebrate visual systems. Neuron; 2010; 66, pp. 15-36. [DOI: https://dx.doi.org/10.1016/j.neuron.2010.01.018]

59. De Vries, B.; Príncipe, J. A theory for neural networks with time delays. Proceedings of the Advances in Neural Information Processing Systems 3; Denver, CO, USA, 26–29 November 1990; pp. 162-168.

60. Joesch, M.; Schnell, B.; Raghu, S.V.; Reiff, D.F.; Borst, A. ON and OFF pathways in Drosophila motion vision. Nature; 2010; 468, pp. 300-304. [DOI: https://dx.doi.org/10.1038/nature09545]

61. O’Carroll, D. Feature-detecting neurons in dragonflies. Nature; 1993; 362, pp. 541-543. [DOI: https://dx.doi.org/10.1038/362541a0]

62. Gonzalez-Bellido, P.T.; Peng, H.; Yang, J.; Georgopoulos, A.P.; Olberg, R.M. Eight pairs of descending visual neurons in the dragonfly give wing motor centers accurate population vector of prey direction. Proc. Natl. Acad. Sci. USA; 2013; 110, pp. 696-701. [DOI: https://dx.doi.org/10.1073/pnas.1210489109] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/23213224]

63. Lessios, N.; Rutowski, R.L.; Cohen, J.H.; Sayre, M.E.; Strausfeld, N.J. Multiple spectral channels in branchiopods. I. Vision in dim light and neural correlates. J. Exp. Biol.; 2018; 221, jeb165860. [DOI: https://dx.doi.org/10.1242/jeb.165860]

64. Rogers, S.M.; Ott, S.R. Differential activation of serotonergic neurons during short-and long-term gregarization of desert locusts. Proc. Biol. Sci.; 2015; 282, 20142062. [DOI: https://dx.doi.org/10.1098/rspb.2014.2062]

65. Yamaguchi, S.; Heisenberg, M. Photoreceptors and neural circuitry underlying phototaxis in insects. Fly; 2011; 5, pp. 333-336. [DOI: https://dx.doi.org/10.4161/fly.5.4.16419]

66. Straw, A.D. Vision egg: An open-source library for realtime visual stimulus generation. Front. Neuroinform.; 2008; 2, 339. [DOI: https://dx.doi.org/10.3389/neuro.11.004.2008] [PubMed: https://www.ncbi.nlm.nih.gov/pubmed/19050754]

67. RIST Data Set. 2020; Available online: https://sites.google.com/view/hongxinwang-personalsite/download/ (accessed on 6 April 2020).