Convolutional Neural Network Based Structured Data Field Granularity Classification Method

Abstract

The data warehouse provides data support for enterprise decision-making and online analysis. In the process of building a data warehouse, many heterogeneous source system data needs to be integrated. In the integration process, these heterogeneous data needs to be classified and put into different topics. The diversity of the source systems of large and medium-sized enterprises poses difficulties for granularity at the field-level automated classification. However, the accuracy of previous methods cannot satisfy users. Therefore, this paper proposes a neural network-based classification technology to classify the data in the granularity field. This method adopts a sampling method to construct the characteristics of the field and innovates a novel classification framework based on the database field on the basis of the CNN network. Accurately achieve 89% by testing the data in the TPC-DS’s dimension tables, and achieve 93% accuracy in real-world data testing. This method was validated in the actual environment of the three banks in China and achieved satisfactory results.