Spatial Sensor Data Processing And Analysis For Mobile Media Applications
COM2 Level 4
Executive Classroom, COM2-04-02
Abstract:
Currently, an increasing number of user-generated videos (UGVs) are collected and uploaded to the Web ? a trend that is driven by the ubiquitous availability of smartphones and the advances in their camera technology. Additionally, with these sensor-equipped mobile devices, various spatial sensor data (e.g., data from GPS, digital compass, etc.) can be continuously acquired in conjunction with any captured video stream without any difficulty. Thus, it has become easy to record and fuse various contextual metadata with UGVs, such as the location and orientation of a camera. This has led to the emergence of large repositories of media contents that are automatically geo-tagged at the fine granularity of frames. Moreover, the collected spatial sensor information becomes a useful and powerful contextual feature to facilitate multimedia analysis and management in diverse media applications. Most sensor information collected from mobile devices, however, is not highly accurate due to two main reasons: (a) the varying surrounding environmental conditions during data acquisition, and (b) the use of low-cost, consumer-grade sensors in current mobile devices. To obtain the best performance from systems that utilize sensor data as important contextual information, highly accurate sensor data input is desirable and therefore sensor data correction algorithms and systems would be extremely useful.
In this dissertation we aim to enhance the accuracy of such noisy sensor data generated by smartphones during video recording, and utilize this emerging contextual information in media applications. For location sensor data refinements, we take two scenarios into consideration, pedestrian-attached sensors and vehicle-attached sensors. We propose two algorithms based on Kalman filtering and weighted linear least square regression for the pure location measurements, respectively. By leveraging the road network information from GIS (Geographic Information System), we also explore and improve the map-matching algorithm in our location data processing. For orientation data enhancements, we introduce a hybrid framework based on geospatial scene analysis and image processing techniques. After more accurate sensor data is obtained, we further investigate the possibility of applying sensor data analysis techniques to mobile systems and applications, such as key frame selection for 3D model reconstruction, camera motion characterization and video encoding.