Content area
Oilfield data is characterized by complex types, large volumes, and significant noise interference, so data cleansing has become a key procedure for improving data quality. However, the traditional data cleansing process needs to deal with multiple types of problems, such as outliers, duplicate data, and missing values in turn, and the processing steps are complex and inefficient. Therefore, an integrated data cleansing and function fitting method is established. The fine-mesh data density analysis method is utilized to cleanse outliers and duplicate data, and the automated segmented fitting method is used for missing data imputation. For the real-time data generated during drilling or well logging, data cleansing is realized through grid partitioning and data density analysis, and the cleansing ratio is controlled by data density threshold and grid spacing. After data cleansing, based on similar standards, the cleansed data is segmented, and the fitting function type of each segment is determined to fill in the missing data, and data outputs with any frequency can be obtained. For the analysis of the hook load data measured by sensors at the drilling site and obtained from rig floor monitors or remote centers, the data cleansing percentage reaches 98.88% after two-stage cleansing, which still retains the original trend of the data. After data cleansing, the cleansed data are modeled through the automated segmented fitting method, with Mean Absolute Percentage Errors (MAPEs) less than 3.66% and coefficient of determination (R2) values greater than 0.94. Through the integrated data processing mechanism, the workflow can synchronously eliminate outliers and redundant data and fill in the missing values, thereby dynamically adapting to the data requirements of numerical simulation and intelligent analysis and significantly improving the efficiency of on-site data processing and decision-making reliability in the oilfield.