Agent-Based Document Expansion for Information Retrieval Based on Topic Modeling of Local Information

With the advent of data ecosystems finding information in distributed and federated catalogs and marketplaces becomes more and more important. One of the problems in data search and search in general is the mismatch between the terminology of users and of the searched items, be it dataset metadata or web pages. The paper proposes an agent-based approach to document expansion (ADE). The idea is to represent documents with agents that exploit local information collected from user searches and relevant signals to improve the representation of the document in a search index and subsequently to improve the search performance of the system. The agents collect terms from relevant queries and perform topic modeling on these terms and publish different variants expanded with the topic terms to the search index. We find that the approach achieves good improvement in search performance and is a valuable tool because is places no burden on the information retrieval pipeline and is complementary to other document expansion and information retrieval approaches.

I. Introduction

The proliferation of data from a variety of sources in research and industry enables more and more data-centric services and applications that are increasingly organized in data ecosystems [1]. The International Data Spaces [2] and Gaia-X¹

https://gaia-x.eu/

initiatives are manifestations of this trend. They establish frameworks that allow data producers to share data while preserving control over their assets. Bringing together needs of data users with the offerings of data providers is an important aspect in these data ecosystems. This matching process is performed by data catalogues, data marketplaces and data brokers.