Context Extraction in Unsupervised Entity Resolution

Modern Entity Resolution (ER) system leverage metadata about the reference data to facilitate processing and making equivalence decisions. This requires that each source of input data must be pre-processed individually to conform to a common metadata alignment. During this alignment, the data is also preprocessed to detect and correct errors to optimize results. After metadata alignment and data cleansing, the sources are merged into a single dataset and submitted to an ER process. This is a costly process and requires additional passes of the input data prior to ER decisioning. This paper formally defines the concept of context within ER to replace the need for metadata alignment and preprocessing. During research into Unsupervised Data Curation (UDC), it was found that there are four (4) points within ER processes for which context can be extracted and used to make decisions to perform the error corrections in flight during processing with no need for metadata. Each of these types of context are formally defined as “Global”, “Block”, “Interim Cluster” and “Final Cluster” in this paper.