An Ensemble-based approach for assigning text to correct Harmonized system code

The world customs organization (WCO) administers the Harmonized System (HS) that provides guidance on common nomenclature to describe traded goods by names and numbers. Inaccurate or incomplete usage of HS codes lead to regulatory violations by the company for which the goods are traded. Shipment often gets delayed because of the inaccuracies encountered. The customs often classify these products into ‘other’ category which turns out to be expensive as high tariff rates are charged. Hence a system is required for accurate assignment of text-description to HS code as per WCO manual. In order to develop an accurate commodity classification engine this paper has provided solutions under two scenarios. Under the first scenario where sufficient and reliable training data is available, we build a BERT-transformer based hierarchical conditional multi-class classification model that outperforms the flat classification model by approximately 16 percentage on a publicly available data. As the HS codes are hierarchical in nature, building a model at each digit gives a clear edge. The reason is that the algorithm arrives at the final class by traversing all the possibilities at each digit level conditional upon the digit chosen at previous level. Under the second scenario, where training data is unreliable as is often the case due to unverified user input stored in the backend, we demonstrate how correct assignment can be arrived at by modelling the WCO manual as a NER based knowledge-graph and then computing the similarity of example text description with the nodes and edges of the graph. We assign the text-description to a HS code based on the highest similarity measure. The knowledge-graph also introduces AI based auditing functionality in the system which is a novelty and addresses questions of responsible AI.