MetaWBC: POSIX-Compliant Metadata Write-Back Caching for Distributed File Systems | IEEE Conference Publication | IEEE Xplore

MetaWBC: POSIX-Compliant Metadata Write-Back Caching for Distributed File Systems

Publisher: IEEE

Abstract:

In parallel and distributed file systems, caching can improve data performance and metadata operations. Currently, most distributed file systems adopt a write-back data c...View more

Abstract:

In parallel and distributed file systems, caching can improve data performance and metadata operations. Currently, most distributed file systems adopt a write-back data cache for performance and a write-through metadata cache for simplifying consistency. However, with modern file systems scales and workloads, write-through metadata caching can impact overall file system performance, e.g., through lock contention and heavy RPC loads required for namespace synchronization and transaction serialization. This paper proposes a novel metadata write-back caching (MetaWBC) mechanism to improve the performance of metadata operations in distributed environments. To achieve extreme metadata performance, we developed a fast, lightweight, and POSIXcompatible memory file system as a metadata cache. Further, we designed a file caching state machine and included other performance optimizations. We coupled MetaWbc with Lustre and evaluated that MetaWbc can outperform the native parallel file system by up to 8x for metadata-intensive benchmarks, and up to 7x for realistic workloads in throughput.
Date of Conference: 13-18 November 2022
Date Added to IEEE Xplore: 23 February 2023
ISBN Information:

ISSN Information:

Publisher: IEEE
Conference Location: Dallas, TX, USA

I. Introduction

The notion of a cache hierarchy has long been central to the design of storage systems to improve their performance [1]. The faster, more expensive, and smaller performance layers can serve as caches for the slower, cheaper, and larger capacity layers [1], [2]. The foundation for caching in Linux is implemented in the Virtual File System (VFS) which enables a transparent access to files independent from the file system implementation. The Linux VFS therefore supports caching for many local and remote file systems through the page cache, the directory entry cache (dcache), and the inode cache (icache) [3]–[5]. Additionally, it ensures concurrency control and permission checking.

References

References is not available for this document.