Sprite Tree: An Efficient Image-Based Representation for Networked Virtual Environments
COM2 Level 4
Executive Classroom, COM2-04-02
Abstract:
Networked virtual environments (NVEs) have become a popular and important class of remote rendering applications. But modern NVEs have several challenging features, including the increasing geometric complexity and concurrent users. Existing solutions, such as mesh-based and video-based approaches, rely heavily on the rendering capability of either the server or the client. Another suitable approach is image-based rendering, but a large number of images are not efficient to utilize. To resolve these challenges, we propose a new and efficient image-based representation for modern NVEs, called sprite tree. Sprite tree organizes multiple reference images efficiently and compactly for accelerating the rendering of complex virtual scenes.
This thesis addresses the following problems: (i) how to represent a complex virtual scene by a sprite tree with low redundancy; (ii) how to render a target view efficiently by using only the sprites similar to this view; (iii) how to assess the visual quality of the rendered target view; and, (iv) how to predict, prefetch, and cache the sprites to be used in the near future.
First, we introduce the basic insertion and rendering methods with sprite tree. We select and insert only dissimilar images based on the view similarity criteria, and use only visible sprites after the frustum and back-face culling for rendering a target view. Results show that sprite tree can efficiently organize the pixels from hundreds of distinctive reference images for the acceleration of rendering.
Second, we propose advanced insertion and rendering methods and also a more efficient no-reference visual quality assessment approach based on the sprite view similarity measure for sprite tree. Results show that the advanced methods largely reduce the lighting artifacts in the rendered images, and also reduce the redundancy and the tree size significantly with little loss of visual quality. We also find out that the no-reference visual quality assessment approach produces similar measures as the full-reference approach, making it useful for assessing a large number of images in real time.
Finally, we study the sprite tree in a client/server remote rendering system. We propose a view prediction scheme and the sprite access probability based on the statistical analysis of user traces in order to predict and prefetch the sprites needed in the near future. We also propose a sprite-based prefetching scheme that utilizes the sprite access probability to prefetch the more probably used sprites in the predicted views with higher priority. The same sprite access probability is also used to replace the least probably used sprites first. Results show that our prefetching scheme is much more efficient than the traditional view-based prefetching scheme.