Real-time Assistance in Photography using Social Media

Abstract:

In the last decade, we have seen significant improvement in the ease and cost of capturing multimedia content. However, the aesthetic quality of the content captured by an amateur user still needs substantial improvement. Camera devices have intelligent features, such as automatic focus, face detection, etc., to assist users in taking better photos, however, it remains a challenge for an amateur user to capture high-quality photographs. The complex nature of photography makes it difficult to provide real-time assistance to a user for capturing high-quality images. However, advancement in digital photography, sensor technology, wireless networks and social media provides us an opportunity to enhance the photography experience of users.

This doctoral research aims at providing real-time photography assistance to users by leveraging on camera sensors and social media content. The research in this thesis is focused on two different aspects of user experience in photography. The first part focuses on camera guidance and the second part is focused on location recommendation for photography.

In the first part, we developed computational models based on machine learning which can provide real-time camera guidance to users for capturing high-quality photographs. The proposed models utilize publicly available photographs along with social media cues and associated metadata for photography learning. In the first contribution, we focus on landmark photography where a feedback regarding scene composition and camera parameter settings is provided to a user while a photograph is being captured. We propose the idea of computing the photographic composition basis, eigenrules and baserules, to support our composition learning. As context is an important factor from a photography perspective, we also explore the role of user-context in photography recommendation. In the second contribution, we focus on group photography where we use the idea of spring-electric graph model and augment it with the concept of color energy from the literature of visual arts. The proposed model is applied in group photography utilizing social media images to provide real-time feedback to the user regarding the arrangement of people, their position on image frame and relative size.

In the second part, we focus on location recommendation for photography to improve the experience of users at tourist locations. As our third contribution, we propose ClickSmart, a viewpoint recommendation system which can provide real-time guidance based on the preview on user's camera, current time and user's geo-location. It makes use of publicly available geotagged images along with associated metadata for learning a recommendation model. We define view-cells, macro blocks in geospace, and propose the idea of popularity, quality and uniqueness of view-cells from viewpoint perspective. Finally, in the fourth contribution, we propose a photography trip recommendation method which guides a user in exploring any tourist location from the photography perspective. More specifically, a tour is recommended to a user based on Optimal Foraging Theory and social media images which provides a list of hot-spots to visit and corresponding stay time at each hot-spot for photography. We have conducted extensive experiments and user studies to demonstrate the effectiveness of the proposed methods.