Web Crawler based Caching Technique for efficient Downloading

With internet availability in almost all types of devices, downloading has become a widespread activity. Data is consumed when any content is downloaded from the web servers. The effectiveness of any internet service provider depends upon the internet connection speed and the time consumed to download a file. The internet connection speed is not always the same. The speed can sometimes be so slow that downloads are not possible. In a typical organizational setup, different types of files and software applications are downloaded regularly. Downloading the same file multiple times by different people is a wastage of resources. The proposed approach aims to decrease the search and download time. The primary purpose of this study is to ensure the availability of the file for download even if there is poor or no internet connectivity. The paper proposes a Web Crawler-based caching technique that helps resolve the issues mentioned above and makes downloading more efficient.