CMCNet: Colorization-Aware Mix-Uncertainty-Adaptive Consistency Network for Semi-Supervised Fruit Counting

Fruit counting is a fundamental and challenging task of automatic fruit yield estimation in the field of intelligent agriculture. In recent years, to relieve the burden of data annotation, semi-supervised counting methods have been studied. Though significant progress has been achieved, the state-of-the-art method estimates the uncertainty of binary segmentation to guide the consistency training of density maps, being prone to deficient uncertainty estimation. Moreover, the method treats pixels with different difficulty equally in each training iteration, being troubled by inflexible consistency training which results in high supervision loss at the beginning of training and even causes network collapse. To alleviate the above limitations, in this paper, we propose a novel semi-supervised counting method CMCNet for fruit counting. CMCNet designs image colorization as an auxiliary task to estimate the uncertainty for density map consistency. Note that this work is the first effort to utilize image colorization for uncertainty estimation in semi-supervised counting. To obtain accurate uncertainty estimation for density map consistency, CMCNet estimates density uncertainty on density maps to depict the difficulty of fruit pixels from the semantic perspective, while using image colorization for constructing colorization uncertainty to measure the difficulty of part of fruit pixels and background pixels from the visual perspective. Then we obtain a comprehensive uncertainty by mixing density uncertainty and colorization uncertainty. Further, we propose a mix-uncertainty-adaptive consistency (MUAC) module for consistency training of density maps. With mix-uncertainty, uncertainty distribution is estimated. By adaptively adjusting the uncertainty threshold, harder pixels will be selected first and easier ones will be added into consistency training gradually. To evaluate the effectiveness of CMCNet, extensive experiments are conducted on two fruit datasets. Experimental results show that our CMCNet is superior to state-of-the-art semi-supervised counting methods.
Note to Practitioners
—This work is motivated by the emerging need for semi-supervised counting methods in fruit yield estimation. The difficulty of training semi-supervised counting methods with unlabeled images is exacerbated by the noisy supervision issue that pseudo-labels of unlabeled images are noisy. The proposed colorization-aware uncertainty estimation strategy and mix-uncertainty-adaptive consistency approach help the fruit planter sufficiently utilize the information of a large amount of unlabeled data and save the annotation cost in fruit quantity estimation. The proposed method is generic as it can be employed as a fruit yield estimation component of crop management support systems, while at the same time being applied to counting frameworks of other objects.