A reconfigurable rack-scale interconnect architecture based on PCIe fabric

Today cloud computation center has become an important component of modern information system. How to achieve the cloud computation center resource sharing and improve resource utilization is an important research topic. In order to improve the resource utilization, rack-scale host (also called converged infrastructure system) is usually used by cloud computation center. This paper proposes a reconfigurable interconnect architecture and resource allocate algorithm for the rack-scale host. The architecture includes CPU node, IO node, PCIe switch node and Ethernet switch node. PCIe switch node is used to connect CPU and PCIe device, and Ethernet switch node is used to configure PCIe fabric. The resource allocation algorithm includes the following steps: request for CPU nodes and IO nodes, initialize PCIe devices, establish device configuration space map, discover new devices and allocate device address space, establish device address space map, establish a map of PCIe device interrupt vectors and CPU interrupt vector tables, and at last load device drivers. The interconnect between CPU and PCIe device is transparent to CPU and OS, so all CPU and OS that can connect to the PCIe device are supported.

I. Introduction

In cloud computation center, computing power and I/O resources are significantly varied over time. Workloads change as the number of users and the amount of data to be processed change. In this respect, it is important to share and allocate resource efficiently and dynamically. In order to improve the resource utilization of the cloud computation center, physical resources can be dynamically expanded or reduced according to the current workload. For PCIe interconnected clusters, different Device Lending strategies are proposed to share resource efficiently. In rack-scale host, I/O devices and CPUs are connected by the same PCIe fabric. In order to separate IO devices from the physical hosts where they reside in, the memory addressing capabilities inherent in the PCIe protocol can be leveraged. By separating the device from the host, any CPU in the rack-scale host can use the remote IO device as if it were a locally installed device. Device drivers or application software do not need to be modified any more.