A Monte Carlo and Kernel Density Estimation based virtual sample generation method for small data modeling problem

In early industrial production, due to the limited resources, enterprises need to use the limited data to analyze the production status and product quality in order to reduce the waste of resources and funds. This requires building a model with high accuracy. Due to the small amount of data, the accuracy of the model based on small samples is low. The technology of generating virtual sample is often used, according to the information interval between sample data to fill in it with an effective way to expand the amount of sample data. A novel kernel density estimation based on distribution with sample output variables is proposed. Monte Carlo sampling is used to fill the gap between sample distribution and realize the uniform distribution of samples. Combined with Bagging-RBF neural network and bat algorithm (BA), effective virtual samples are generated. Two experiments, MLCC and PTA, show that the virtual samples are more effective.

I. Introduction

With the continuous progress of modern science and technology, big data sets emerge, but many big data sets have some problems, such as: low value density. In the situation of high precision and reliability, there are not many useful data needed. Secondly, depth learning requires a data corresponding to a label. Although the technical content is not high, it needs a lot of manpower and time, so it is very expensive. Based on the shortcomings of big data, mining effective information with small samples is particularly important.