Content area
With the continuous development of the social economy, the ways people obtain news information are becoming increasingly diversified, but with that comes too much data. The research on Extracting useful information from too much data is extremely effective. Given these needs and deficiencies, this paper introduces a conditional random field knowledge recognition algorithm, designing a Segmentation analysis model for audience news with the segmentation technology of key frames and shots by sorting the business logic of automatic news Segmentation, realizing the analysis of the news video picture., and then analyze the news Segmentation to ensure that the production and dissemination of news programs are intelligent and smart. The simulation experiment results show that the conditional random field knowledge recognition algorithm is effective and can effectively support the analysis of automatic news Segmentation.
Introduction
With the continuous development of the social economy, the ways of news dissemination have become more and more diversified [1, 2]. People can get news videos in a timely and convenient way through PCs and mobile devices [1, 3]. In the meanwhile, this method of acquisition has brought about an explosion of new data. However, news information has become the main content of new media or micro-media and the main platform and channel to attract user traffic [4]. The effective processing of news data is actually to extract effective information from excess data information [5]. According to the news segmentation, the independent division of news content can be realized, which can be divided into “oral broadcast + content” to realize complete fragments, realize specific news segmentation, and further realize effective classification and data analysis of news and other news video services, which can provide users with the ability for quick search and access to news content, and realize effective classification. Meanwhile, it can also provide users with interesting sections or news according to users' browsing habits [6, 7].
Because of these needs and research bottlenecks, Scholars in the industry have also conducted corresponding research, such as the study of intelligent video Segmentation technology, which aims to improve the communication efficiency and effects of new media technologies. In the era of big data, information has become more diverse and larger in volume. Whether it is actively acquired or passively received, more and more news information can be obtained every day. Different users have different acceptance of news types and interest sectors. Therefore, traditional TV push news can only be passively accepted. In the mobile Internet era, users have more autonomy and choice [8]. How to push the sections and news of interest to users according to their interests, segment too many long news items into multiple key news items, change the traditional manual editing method, deliver news feeds more accurately, and improve users’ stickiness is one of the more and more important research directions.
Given these needs and research bottlenecks, based on the conditional random field knowledge recognition algorithm oriented to TV news, this paper designs the corresponding news program segmentation model orientated at TV news with news push as a cut-in point through sorting the business logic of automatic news Segmentation, to realize the effective analysis and segment of news pictures, to ensure the accuracy and effectiveness of news feeds, aiming at enhancing user stickiness in the era of new media.
Conditional random field knowledge recognition algorithm
The video can be regarded as a specific video frame image. The specific color histogram is used to optimize the matching of blocks. The specific images are segmented based on different thresholds so that the specific news disassembly can be realized [9]. The algorithm is shown in the flow chart in Fig. 1.
[See PDF for image]
Fig. 1
The flow chart of the intelligent Segmentation of TV news programs
Before implementing the conditional random field algorithm, a data preprocessing step is required. First, the text data are cleaned and marked to ensure that the input text format meets the model requirements, and then the text is divided into mark sequences, such as part-of-speech tagging, entity tagging, etc. Next, the performance of the conditional random field algorithm is highly dependent on the selection of features. Based on the task requirements, this study designed a rich feature set, which can include parts of speech, word boundaries, contextual information, etc.; the goal of feature extraction is to construct a feature vector for each tag. The third step is to conduct model training. This step mainly uses the training data set to train the conditional random field, and uses optimization algorithms, such as gradient descent, maximum entropy optimization, etc., to optimize the parameters of the conditional random field. Finally, the trained conditional random field model is used to perform label prediction on the new text sequence. Among them, methods such as the Viterbi algorithm are used to find the most likely tag sequence, and the formula is as follows:
1
In conditional random fields, parameters usually include weight vectors and transition matrices. The selection of specific parameters will vary depending on the task, but generally include weight vectors (feature weights) and transition matrices. The weight vector (feature weight) has a corresponding weight, which is used to measure its contribution to the model. The initial value of the weight is determined based on experience or cross-validation methods; the transition matrix defines the transition probability between different markers, which can be determined by the maximum Determined by methods such as likelihood estimation. The conditional probability distribution of a conditional random field is defined as follows:
2
Among them, is the marker sequence, is the input sequence, is the normalization factor, which normalizes the probability distribution, is the feature weight, and is the feature function.
Color model conversion
The traditional RGB color model is converted to the HSV model, and the specific calculations are as follows:
Let , define r′, g′, and b′:
3
Then
4
5
where , , , .The specific steps are as follows:
The visual discrimination ability of the human eye can divide the specific H color model into 8 parts with no interval, and the remaining two components are divided into 3 parts each.
Based on step 1, according to the size of the range, the color and subjective perception are quantitatively divided. The specific division is as follows:
6
7
On this basis, the corresponding feature vector is constructed to perform effective quantification according to the above formula, realize specific color quantitative analysis, and merge each color component into a specific one-dimensional feature vector to achieve effective comparison.
8
Among them, QS and QV are the two components of the quantization stages of components S and V, respectively, so the original formula can be expressed as
9
With the specific posterior probability as the confidence unit of the specific conditional random field, the specific confidence estimation calculation formula is shown in formula 10:
10
In the formula, the specific speech observation sequence can be expressed by . The specific cumulative calculation of the calculation of news dismantling is shown in formula (11):
11
Of course, this method may need to calculate the specific sequence probability value in the specific news Segmentation, but in practical applications, this cannot be achieved [10, 11]. As shown in Fig. 2:
[See PDF for image]
Fig. 2
Frame diagram of hidden conditional random field
Through the non-linear relationship, the data probability is used to realize the distribution, and then the specific conditional probability can be calculated by formula (12):
12
The conversion is carried out based on formula (12), as shown in formula (13) and formula (14):
13
14
Compared with the traditional conditional random field, the conditional random field knowledge recognition algorithm fused with hidden units can model data in a complex environment for accurate data prediction and analysis [12, 13]. Typical methods include maximum conditional probability, perceptron, etc. Among them, perceptron criteria can be used to obtain a more obvious model with less computational complexity in a specific environment. Therefore, this paper selects perceptron for model training. At the same time, as for specific feature selection, the conditional random field knowledge recognition algorithm has greater flexibility and can add or delete specific features without considering the redundancy of feature information [14].
In combining features, this article uses n-gram grammar mode to illustrate the mode of context combining features. The specific definition is shown in Table 1.
Table 1. Context example of syllable “yi”
Relative position | − 3 | − 2 | − 1 | 0 | 1 | 2 | 3 |
Syllable sequence | Liang | ge | ren | yi | qi | Shang | Jie |
According to the results in Table 1, the syllable can be realized as “two people go to the street together”.
Combine separately according to the corresponding combination mode; the conditional random field knowledge recognition algorithm is used to realize specific sorting analysis and intelligent combination, to remove other redundant data, and clarify the specific optimal combination mode [15, 16], as shown in Table 2:
Table 2. Optimal context feature combination mode
Feature type | Integrated mode |
|---|---|
Atonal syllable | − 1/0,0/1, − 1/0/1 |
Duration | − 1/0, 0/1 |
Normalized posterior probability | − 1/0, 0/1 |
Language model fallback behavior | − 1/0, 0/1 |
Part of speech | − 2/− 1/0 |
Distance between words | − 1/0/1 |
In a specific experiment, the number of cluster categories is shown in Fig. 3:
[See PDF for image]
Fig. 3
Categories of each knowledge source
Color histogram and block matching algorithm
Specific histograms are used to determine effective information. This article introduces a specific block-matching method to expand the color and analyze the histogram value. The entire image is divided into 3*3 modules to achieve effective analysis and comparison of adjacent frames, to achieve the information comparison of the color histogram, to improve the corresponding position information, and the specific accuracy rate [17, 18]. The specific method of image sub-module division is shown in Fig. 4:
[See PDF for image]
Fig. 4
Schematic diagram of image sub-block division
Judgment and selection of lens segmentation threshold
According to the corresponding flowchart, the frames before and after the shot can be calculated to determine whether the histogram difference is greater than the set threshold. If it exceeds a specific threshold, it needs to be selected again to ensure the accuracy rate and validity of the calculation during operation.
By subtracting the effective dimensions of the histogram of the specific sub-block, the specific histogram can be obtained, as shown in Fig. 5:
[See PDF for image]
Fig. 5
Histogram of the difference between the first sub-block of the 44th frame and the first sub-block of the 46th frame
The same formula can be used to calculate the difference value of the sub-block. After the difference value histogram is calculated, the formula (15) can be used to perform the specific calculation of the difference value histogram between two frames.
15
Among them, the value of the histogram is represented by H, the frame number is represented by n, and the pixel value is represented by k. After the specific two-frame calculations, the histogram of the difference between adjacent frames of the entire video sequence can be obtained, as shown in Fig. 6:
[See PDF for image]
Fig. 6
Difference histogram of all frames
It can be seen from the results in Fig. 6 that there is a large difference in 44 frames, and there may be a lens switch. Therefore, when comparing the corresponding lens transformation value with the actual switch parameter, you can find that the specific threshold setting of 0.3 is relatively accurate, so this paper selects 0.3 as the specific threshold.
Design and implementation of intelligent segmentation software system
Based on the above data analysis, this paper designs a corresponding computer vision library. The specific operation process is as follows:
Using “Open File” can effectively realize the feature segmentation of specific news video materials. After selecting a fixed video, the specific height, width, video frame, current frame, and other related parameters of the news video can be effectively displayed on the specific large screen; in the meantime, the specific first frame of the news video can be previewed.
The “setting” operation is mainly for setting the height and width of the news video.
“Play Video” is mainly used to play the corresponding video effectively. Once the specific shot segmentation operation is completed, in the actual news broadcast, if there is a corresponding shot change, the video will have a certain reaction time to pause and then continue to play. In the specific process, the parameter display can be used as the specific frame.
The “Reset” button can reset the video playback position to the first frame.
By operating the “previous frame”, the specific news video position can be adjusted to the previous frame.
By operating the “Next Frame” button, the specific news video position can be adjusted to the next frame.
“Shot segmentation” uses specific segmentation elements to achieve specific news material retrieval, which can achieve specific HSV model operations.
The “View Shot Segmentation Key Frame” button can switch the key frame after the shot segmentation is over and display its frame sequence on the right side of the button.
The function of the “Video Segmentation” button is to segment the selected news material, and the algorithm used is the same as the shot segmentation function.
The “View keyframes for Segmentation” button can be used to view the saved key frames of video Segmentation in the Segmentation keyframe display area after Segmentation is finished and display its frame sequence on the right side of the button.
The “Play button in the Segmentation Play Area” can be used to browse the selected news material after segmenting the article.
Experimental data and evaluation criteria
This paper uses the following three values to evaluate news Segmentation: news boundary point recall rate, news boundary point accuracy rate, and news boundary point value F1 = 2 × recall rate × accuracy rate/(recall rate + accuracy rate).
This study uses user experiment design to evaluate the essential differences and absolute advantages between the automatic news segmentation analysis algorithm combined with conditional random field knowledge and traditional software in terms of user experience and human–computer interaction. The researchers recruited a large number of users to ensure that factors such as age, gender, news reading experience, etc. were covered. A total of 50 participants were divided into experimental groups and control groups. The experimental group used an automatic news segmentation analysis algorithm combined with conditional random field knowledge. The control group used traditional automatic news segmentation and analysis software. By simulating real news reading scenarios, participants were asked to complete the following tasks under two different software: reading an article containing multiple news topics, extracting news fragments of interest in the article, and evaluating segmentation accuracy and usage experience. During the experiment, the researchers recorded the time it took the user to complete the task and analyzed the real-time performance of the algorithm (as shown in the Table 3). After the user completed the task, they filled in a user experience questionnaire, including software ease of use, ease of operation, etc. Evaluation (as shown in the Table 4). Finally, standard news segmentation labels are used to compare with user segmentation results, and accuracy, recall and F1 scores are calculated (as shown in the Table 5).
Table 3. Task completion time statistics
Participants | Experimental group (seconds) | Control group (seconds) |
|---|---|---|
1 | 62 | 75 |
2 | 68 | 82 |
– | – | – |
50 | 58 | 80 |
Table 4. User experience questionnaire
Evaluation items | Score of the experimental group (out of 5 points) | Control group score (out of 5 points) |
|---|---|---|
Usability | 4.3 | 3.5 |
Ease of operation | 4.5 | 3.2 |
Interface friendliness | 4.2 | 3.8 |
Table 5. Segmentation accuracy assessment
Evaluation index | Experimental group (%) | Control group (%) |
|---|---|---|
Accuracy rate | 92 | 80 |
Recall rate | 95 | 75 |
F1 score | 93 | 77 |
It has advantages in real-time performance. In the user experience questionnaire analysis, the experimental group received higher scores in terms of ease of use, operational convenience and interface friendliness, indicating that the conditional random field algorithm provides a better user experience. In terms of segmentation accuracy evaluation, the experimental group was significantly better than the control group in accuracy, recall and F1 score, indicating that the conditional random field algorithm has higher accuracy in automatic news segmentation. Through the experimental results, this study concluded that the automatic news segmentation software combined with the conditional random field knowledge recognition algorithm performs well in terms of user experience and segmentation accuracy, and has obvious advantages over traditional software. Future research can further optimize the algorithm to adapt to more complex contexts and extend it to practical application scenarios.
Experimental results and analysis
Overall experimental results and analysis
In this paper, the demarcation points obtained based on the test of the data mentioned above set are 1205 demarcation points, of which 1122 are correct demarcation points. The experimental results are: recall rate 0.8; accuracy rate 0.9; F1 value 0.9.
There are the following reasons leading to missed detection in the experiment of the algorithm: Since there is no absolute standard threshold to define the length of the mute period between news items, the length of the mute segment of the news item demarcation point is less than the threshold, and it accounts for larger proportion in the algorithm's missed detection.
The influence of the mute section length threshold on the experimental results
Different mute segment thresholds are adopted to carry out the experimental results for this algorithm, as shown in Fig. 7.
[See PDF for image]
Fig. 7
The influence of the mute section length threshold on the experimental results
When the threshold is around 50, the F1 value is at an overall high level.
The influence of the limiting conditions of the mute section on the experimental results
In this section, comparative experiments are used to prove the influence of the mute section restriction conditions on the experimental results, and the results are shown in Fig. 8.
[See PDF for image]
Fig. 8
The influence of the limit conditions of the mute section on the experimental results
Dealing with the influence of audio glitches on the experimental results
In this section, the effect of processing audio glitches on the experimental results is obtained through comparative experiments, as shown in Fig. 9.
[See PDF for image]
Fig. 9
The effect of processing audio glitches on the experimental results
Although not dealing with the glitch phenomenon can detect some missed news item demarcation points, it also caused more false detections.
Comparative analysis of other algorithm results
Conditional random fields are a probabilistic graphical model that have been widely used in sequence labeling tasks. In the study, CRF was used to perform knowledge recognition on news texts. The algorithm improves the accurate identification of knowledge entities by modeling the relationship between observation features and state features, as well as taking into account the global information of the label sequence. In recent years, deep learning methods have achieved remarkable results in the field of natural language processing. In this study, the conditional random field algorithm is compared with deep learning-based methods such as recurrent neural network (RNN) and long short-term memory network (LSTM), which are expected to improve the accuracy of news segmentation by capturing long-term dependencies in text.. Traditional rule-based methods rely on prior knowledge and hand-crafted rules. The researchers compared the CRF algorithm with these methods to evaluate whether the model can perform more robustly in different domains and contexts. Statistical methods are usually based on statistical analysis of text, such as word frequency statistics, TF-IDF, etc. This study compares the CRF algorithm with these methods to verify its performance in information-intensive news texts. This study uses indicators such as accuracy, recall, and F1 score to evaluate the performance of the above methods. Performance data for each method on these metrics were obtained through cross-validation or using a specific test set (See Table 6).
Table 6. Comparison result
Method | Accuracy | Recall | F1 score |
|---|---|---|---|
Conditional random field (CRF) | 0.85 | 0.88 | 0.86 |
Deep learning method | 0.82 | 0.84 | 0.83 |
A rule-based approach | 0.75 | 0.78 | 0.76 |
By comparing the performance of different methods in terms of accuracy, recall rate and F1 score, it is found that the conditional random field algorithm performs well in this task. Its high accuracy and F1 score indicate its applicability for news segmentation tasks. In contrast, deep learning methods may be too complex in certain situations, while rule-based methods and statistical methods perform relatively poorly in complex contexts.
The comparison results of the algorithm in this paper and other algorithm. The selected data sets are all news simulcast programs, as shown in Fig. 10.
[See PDF for image]
Fig. 10
Comparative experimental analysis results
Traditional method 1 uses four episodes of about 128-min news broadcasts as the experimental data set. The algorithm is also based on the characteristics of the host and the audio mute segment. By default, the host’s announcement is always a separate news item. Even if a detailed video report of related news follows the host's broadcast, it is considered two different news items. Under this premise, a higher recall rate is obtained but with a lower accuracy rate. Traditional method 2 also conducted experiments on four-episode news broadcasts, using host characteristics, theme subtitle characteristics, and audio mute segment characteristics to cut the story unit in the news video, resulting in good experimental results, but the boundary precision of the news demarcation point is in seconds.
Software performance test and result analysis
The recall and precision rates of the shot segmentation algorithm and news Segmentation algorithm were tested, respectively, and the test results are shown in Figs. 11 and 12.
[See PDF for image]
Fig. 11
Test results of the lens segmentation algorithm
[See PDF for image]
Fig. 12
Test results of news disassembly algorithm
It can be seen from the results that the conditional random field knowledge recognition algorithm is effective, can effectively detect specific shot switching, and better support the automatic Segmentation of news programs.
Conclusions
With the continuous development of the social economy, the ways to obtain news are gradually diversified, such as using mobile apps, TV news, and radio. During the algorithm development process, researchers put users' actual news viewing needs and experiences at the core to ensure that the research is closer to users' expectations and actual application scenarios. Specifically, in the early stages of the study, the researchers conducted extensive user needs research to understand users' interests, preferences, and expectations when watching news videos. A large amount of feedback and suggestions on users’ news viewing are collected through questionnaire surveys, user interviews, etc. Based on the results of user demand research, this article established a user interest model to clarify user preferences for different news topics and forms. This model plays a guiding role in the design of key frame extraction, recognition and processing algorithms. At each stage of algorithm development, this study simulates the actual viewing process of users and pays attention to factors such as key frame selection and processing speed to ensure that the algorithm not only meets technical requirements but also provides a smooth viewing experience that meets user expectations. Researchers will also continue to collect user feedback and incorporate this feedback into the algorithm improvement process. Through regular user testing and feedback loops, we ensure that algorithm performance is consistent with user expectations and continuously optimize the user experience. In addition to this, the research also focuses on the interpretability of the algorithm so that users can understand the process of key frame extraction and recognition. Efforts should be made to improve user participation in the decision-making process of the algorithm, allowing users to more proactively customize and adjust the behavior of the algorithm to meet personalized viewing needs. Based on the conditional random field knowledge recognition algorithm, this paper analyzes the business logic flow by sorting out the automatic news segmentation and news business model automatically. The corresponding segmentation of shots is performed through color histogram and HSV model to realize the segmentation technology of key frames and shots, design an analysis model for audience news, analyze news video pictures, and then analyze news Segmentation to ensure the production and dissemination of news show are smart and intelligent.
Author contributions
CX contributed to Writing-original draft preparation, conceptualization, supervision, project administration.
Funding
This study did not receive any funding in any form.
Data availability
The data used to support the findings of this study are available from the corresponding author upon request.
Declarations
Conflict of interest
The authors declare no conflicts of interest.
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
References
1. Osti, F; Ceruti, A; Liverani, A; Caligiana, G. Semi-automatic design for disassembly strategy planning: an augmented reality approach. Proc. Manuf.; 2017; 11, pp. 1481-1488.
2. Vongbunyong, S; Kara, S; Pagnucco, M. Learning and revision in cognitive robotics disassembly automation. Robot. Comput. Integr. Manuf.; 2015; 34, pp. 79-94. [DOI: https://dx.doi.org/10.1016/j.rcim.2014.11.003]
3. Marin, A; Denimal, E; Bertheau, L; Guyot, S; Journaux, L; Molin, P. Automatic counting of intra-cellular Ribonucleo-protein aggregates in saccharomyces Cerevisiae using a textural approach. Microsc. Microanal. Microanal.; 2019; 25,
4. Tucker, P.S., Scanlan, A.T., Dalbo, V.J.: Chronic kidney disease influences multiple systems: describing the relationship between oxidative stress, inflammation, kidney damage, and concomitant disease. Oxid. Med. Cell Longev 2015 (2015)
5. Hefler, M. Worldwide News and Comment; 2017; BMJ Publishing Group Ltd:
6. Cahen, D; Crabtree, G; Yalisove, SM. Energy quarterly: news and analysis on materials solutions to energy challenges. MRS Bull.; 2011; 36,
7. Whittingham, S.: Energy quarterly news and analysis on materials solutions to energy challenge. In: MRS Bulletin, vol. 42, no. 6. Cambridge Univ Press 32 Avenue of the Americas, New York, pp. 10013–2473, USA, p. 413 (2017)
8. Bettacchioli, E; Nafai, S; Renaudineau, Y. News and meta-analysis regarding anti-Beta 2 glycoprotein I antibodies and their determination. Clin. Immunol.; 2019; 205, pp. 106-115. [DOI: https://dx.doi.org/10.1016/j.clim.2019.06.002]
9. Li, X; Xie, H; Song, Y; Zhu, S; Li, Q; Wang, FL. Does summarization help stock prediction? A news impact analysis. IEEE Intell. Syst.; 2015; 30,
10. Zhang, C; Wu, X; Shyu, M-L; Peng, Q. Integration of visual temporal information and textual distribution information for news web video event mining. IEEE Trans. Hum. Mach. Syst.; 2015; 46,
11. Li, J; Ma, S. Survival Analysis in Medicine and Genetics; 2013; CRC Press: [DOI: https://dx.doi.org/10.1201/b14978]
12. Wasfy, JH; Zigler, CM; Choirat, C; Wang, Y; Dominici, F; Yeh, RW. Readmission rates after passage of the hospital readmissions reduction program: a pre–post analysis. Ann. Intern. Med.; 2017; 166,
13. Yablonovitch, E. News and analysis on materials solutions to energy challenges: www. mrs. org/energy-quarterly. MRS Bull.; 2015; 40,
14. Patton, EW; Moniz, MH; Hughes, LS; Buis, L; Howell, J. National network television news coverage of contraception: a content analysis. Contraception; 2017; 95,
15. Lam, TC; Kwan, E; Luo, H; Yip, PS. Emergency admissions and bad news disclosure as precipitators of suicide: a territory-wide cohort analysis of 458 oncology patients who completed suicides. Ann. Oncol.Oncol.; 2019; 30, v661. [DOI: https://dx.doi.org/10.1093/annonc/mdz261]
16. Lord, J et al. Prenatal exome sequencing analysis in fetal structural anomalies detected by ultrasonography (PAGE): a cohort study. The Lancet; 2019; 393,
17. Abraham, A; Zhang, AJ; Ahn, R; Woodbridge, A; Korenstein, D; Keyhani, S. Media content analysis of marijuana’s health effects in news coverage. J. Gen. Intern. Med.; 2018; 33, pp. 1438-1440. [DOI: https://dx.doi.org/10.1007/s11606-018-4492-9]
18. Blair, B; Zimny-Schmitt, D; Rudd, MA. US news media coverage of pharmaceutical pollution in the aquatic environment: a content analysis of the problems and solutions presented by actors. Environ. Manage.; 2017; 60, pp. 314-322. [DOI: https://dx.doi.org/10.1007/s00267-017-0881-9]
© The Author(s), under exclusive licence to Springer-Verlag London Ltd., part of Springer Nature 2024.