FS-Boost: Communication-Efficient Federated Subtree-Based Gradient Boosting Decision Trees

Federated learning (FL) is a secure and distributed machine learning method in which clients learn cooperatively without disclosing private data to others. some decision tree-based FL have been proposed that employ gradient boosting decision trees (GBDT). However, previously proposed GBDT-based FL methods require sharing the entire decision tree, including the tree structure and leaf weights, for each synchronization step in training. This process inevitably results in extensive communication. To solve this problem, we propose FS-Boost-Federated Subtree-based GBDT, a horizontal FL method that reduces the communication cost. Suppose the maximum depth of the decision tree is set to d. In that case, a conventional GBDT trains only a single decision tree of depth d within one training round. In contrast, FS- Boost utilizes subtrees from depth 1 to d-1 generated in the learning process in addition to the whole trees. Sharing still-growing subtrees with other clients reduces the total amount of communication cost through accelerating model convergence. Our experiment results indicate that FS-Boost significantly reduced the communication cost by at least half in most cases while maintaining the accuracy.