Adaptation of facial and body animation for MPEG-based architectures

While level of detail (LoD) methods for the representation of 3D models are efficient and established tools to manage the trade-off between speed and quality of the rendering, LoD for animation has not yet been intensively studied by the community, and especially virtual humans animation has not been focused in the past. Animation, a major step for immersive and credible virtual environments, involves heavy computations and as such, it needs a control on its complexity to be embedded into real-time systems. Today, it becomes even more critical and necessary to provide such a control with the emergence of powerful new mobile devices and their increasing use for cyberworlds. With the help of suitable middleware solutions, executables are becoming more and more multi-platform. However, the adaptation of content, for various network and terminal capabilities - as well as for different user preferences, is still a key feature that needs to be investigated. It would ensure the adoption of the "Multiple Target Devices Single Content" concept for virtual environments, and would in theory provide the possibility of such virtual worlds in any possible condition without the need for multiple content. It is on this issue that we focus, with a particular emphasis on 3D objects and animation. This paper presents some methods for adapting a virtual human's representation and animation stream, both for their skeleton-based body animation and their deformation based facial animation, we also discuss practical details for the integration of our methods into MPEG-21 and MPEG-4 architectures.

1. Introduction

Since the invention of the computer, content has been tailored towards a specific device, mainly by hand. Computer applications have generally been developed with specific computer's capabilities in mind, various sized video has been produced so that the user can select their choice based on the one most likely to run best, and even different formats have been provided in order that it can run on different types of machines. In recent years, this trend of multiple devices, multiple content (MDMC), is slowly shifting towards multiple devices, single content (MDSC), with great advantage both to the content provider and the end user. Firstly, only a single (usually high quality) content need be provided for an entire suite of devices, and secondly the user is provided with content that is optimised and fits not only the requirements of the device, but the network, and the user's own, often specific, preferences. This highly motivating goal can only be achieved within the framework of standardisation; and in this case the Motion Picture Experts Group (MPEG); this is mainly due to the extensibility and range of applications to which this can be taken. Though early work in MPEG-7, e.g. proposed by Heuer et al [1], adapts certain types of content, this kind of MDSC concept is being constructed under the framework of Digital Item Adaptation (DIA) - a Digital Item being the MPEG name given to any media type or content, adaptation because it is being scaled, rather than changed, from its original form into something that is tailored towards the context of the user and the user's equipment. DIA is part of the MPEG-21 framework [2], for content control, and is in its final stages of standardisation.