PH.D DEFENCE - PUBLIC SEMINAR

Multimedia User Profiling in Online Social Networks

Speaker
Ms Geng Xue
Advisor
Dr Chua Tat Seng, Professor, School of Computing


10 Jan 2017 Tuesday, 02:00 PM to 03:30 PM

Executive Classroom, COM2-04-02

Abstract: Online Social Network Services (OSNs) have been evolving continuously while revolutionizing our lives over the past decade. They provide popular platforms to build social networks and enhance social relationships among people who share common interests, activities, backgrounds and real-life connections. Over the years, many types of OSNs have emerged, many of which are multimedia-based sites such as Pinterest, Flickr and Youtube. Furthermore, people have been sharing more and more multimedia contents over the years. For example, it has been reported that between April 2015 and November 2015, the amount of average daily video views on Facebook doubled from 4 billion video views per day to 8 billion. However, the exponentially increasing media contents will make it difficult for service providers to tailor media contents to accommodate specific individuals.

To address the above issue, we attempt to undertake the task of user profiling which is one of the fundamental tasks of personalization in OSNs. To the best of our knowledge, most existing approaches only focus on mining textual information to construct user profiles, overlooking the abundant shared media contents. Unfortunately, textual information may not provide complete and easy-to-grasp information to generate user profiles. Hence, we, taking Pinterest as an example, focuses on developing effective and efficient approaches to model user profiles, by exploring rich user-generated multimedia contents including images, texts, together with domain knowledge.

The task of profiling users based on their rich media interactions in OSNs poses several great challenges. First, how to mine the extremely heterogeneous and noisy media contents for user profiling; second, how to use domain knowledge to guide the media feature learning for human-understandable user profiles; third, how to use user-media interactions in OSNs to advance the task of modeling users; and how to integrate domain knowledge and social collective intelligence together to obtain efficient and effective user profiles for personalized services. To address the above challenges, this work first introduces a data-driven user profile ontology and exploits the relationships between concepts in the ontology to enhance media understanding for user profiling. The outcome is a human understandable user profile for efficient personalized services. The second part of this work, a deep learning model was presented to reveal the weak correlations of user-media connections for learning representative features of images and users simultaneously. The final part of this work presents a co-factorization approach to integrate the above multi-modal contents, domain knowledge and social user-media connections together into a framework to profile users in OSNs.

Through extensive experiments conducted on the large-scale real-world datasets, the experimental results have demonstrated that our study could yield significant gains in constructing effective user profiles based on the multimedia contents shared by users in online social networks.