Fair Data Valuation For Collaborative Machine Learning
Dr Foo Chuan Sheng, Institute for Infocomm Research
COM1 Level 3
MR1, COM1-03-19
Abstract:
Collaborative machine learning presents as a promising solution for large-scale machine learning by leveraging the resources (i.e., data) of multiple agents (e.g., companies, organizations, individuals, teams etc.), introducing, at the same time, a new layer of complexity from the interaction among these agents. Since the resources of these agents are their data, ensuring their data is fairly valued in such collaborations is the key to incentivizing them to join the collaboration (e.g., so the agents do not feel exploited), thereby propelling the development and deployment of practical collaborative machine learning solutions. This thesis considers the problem of fair data valuation for collaborative machine learning in three parts.
This talk will focus on two works (i) model valuation and (ii) data distribution valuation where (i) discusses how we exploit a Dirichlet abstraction of trained classifiers to propose a fair and quantitative valuation method for these trained classifiers, called model Shapley. It differs from existing methods in that it does not require access to the training data, model architecture or other training/model information, except the model’s predictive outputs on a selected query set. Separately, (ii) studies the valuation of data distributions and utilizes a maximum-mean discrepancy-based formulation to design a novel data distribution valuation method. It differs from existing works in that it studies the underlying data distributions instead of the sampled discrete datasets, yielding a formal characterization of the effect of the data heterogeneity on the valuation.