Real Time Metering of Cloud Resource Reading Accurate Data Source Using Optimal Message Serialization and Format

In this paper the technology of collecting the logs and the logs message format is considered in addition to investigating multiple log data sources as an input to the cloud management system. A comparison between message exchange technologies (JSON, XML) is evaluated with the latest message format technology (Google Protocol Buffer) when used in combination with the message transmission protocols (XML-RPC, REST, Network Socket). In addition, the sampling rate that gives the accurate reading of resource usage is investigated, which is used to select among different log data sources to achieve the accurate log update time. Logs sampling rate of 1.0 second is found to be the best with "xentop" data source. The result of the experiment shows using Protocol Buffer with Socket protocol gives the best results in reducing message size. Network socket with JSON gives the best processing delay and traveling time.

I. Introduction

Cloud monitoring and logging play an important role in cloud management system. Logs are generated from multiple data sources in cloud environment such as Physical Machines (PMs), Virtual Machines (VMs), network devices and the cloud running applications services. Each single cloud component has numerous logs and events data source generators, which could be generated from low level frameworks such as Hypervisor (the virtualization software) up to the running applications on top of the VMs. Logs must be captured and filtered or aggregated based on the need of the cloud management system, which can be used to discover cloud status and future actions for cloud management system. Cloud management models can manage the resources dynamically via reconfiguring cloud resources using numerous approaches such as VM migration, storage migration, network reconfiguration, load balancing, affinity relation, hot spot avoidance, server consolidation, holistic approach and fault tolerance [1]. These resource reconfiguration methods can be generalized in a closed loop control model proposed as following [2]: Cloud resource monitoring, logs analyzing, resource planning and finally executing the planned decisions. Figure 1 depicts the closed loop cloud management system.