ROAR: A Benchmark for NFT Rarity Meters

Rarity meters are incorporated by industry and discursive by academia. Rarity, as an intuitive term, attracted numerous researchers to present their own view of it. While there is existing literature on comparing rarity meters, it requires access to NFT collection data, which can be challenging for researchers without a background in blockchain technology. This has created a demand for an easily accessible rarity meter benchmark. In this paper, we introduce the Rating over all Rarities (ROAR) benchmark, which includes data from one hundred popular NFT collections from the Ethereum blockchain, implemented a weighted correlation-based performance measurement function, as well as four state-of-the-art rarity meters (Rarity.tools, Kramer, OpenRarity, and NFTGo), along with a new rarity meter called ROAR. Our experiments show that the ROAR rarity meter, an ensemble of the other four meters, outperforms its competitors, with Rarity.tools and Kramer as runner-ups. The ROAR benchmark is a tool for examination and testing of rarity meter ideas, and we challenge readers to develop models that can outperform the ROAR rarity meter.