Mitigating Filter Bubble in News Recommenders: A Framework of Interventions for Representation, Algorithms, and Evaluation
COM1 Level 3
MR1, COM1-03-19
Abstract:
News recommender platforms are the primary source of news for many people. However, their underlying algorithms, which optimise engagement through semantic similarity, can inadvertently create “filter bubbles”. In other words, there is a misalignment between this optimisation for short-term engagement and the societal goal of fostering an informed and diverse public discourse. To mitigating this socio-technical problem, the thesis proposes a multi-angled framework that intervenes at three layers of the recommendation stack, namely data representation, algorithmic ranking policy, and evaluation of the recommender algorithm.
First, at the representation layer, the thesis introduces ScaleEmb, an efficient method that “politicises embeddings” by encoding a diversity attribute, such as political slant, directly into the magnitude of a news article’s vector. Diverse yet relevant candidates can then be retrieved with standard, highly optimised nearest-neighbour search. ScaleEmb was validated using labelled news articles and a human study, and extended to multiple diversity attributes. Second, at the policy layer, the thesis proposes the Headline-Content Decoupling (HC-Dec) framework, a two-stage “Connect-and-Broaden” algorithm. It connects through engaging headlines to preserve relevance and then broadens exposure by re-ranking based on article content for viewpoint diversity, personalising the size of the nudge to each user’s tolerance using a multi-armed bandit approach. A longitudinal study with human participants shows that the adaptive policy increases the diversity of the articles users consume, with minimal cost to user engagement. Third, at the evaluation layer, the thesis proposes a persona-driven simulation sandbox, in which pre-trained LLMs, guided by behaviourally grounded personas, screens candidate algorithms before a costly human study. Validation studies show the simulator to be a useful comparator for ideological-exposure and diversity metrics.
Together, the methods, and their simulated and human-subject evaluations, contribute a principled and practical framework for news recommender systems that are personalised for users but also actively work to broaden user perspectives through greater diversity of news articles.