Differentially private M-estimation via noisy optimization
We present a noisy composite gradient descent algorithm for differentially private statistical estimation in high dimensions. We begin by providing general rates of convergence for the parameter error of successive iterates under assumptions of local restricted strong convexity and local restricted smoothness. Our analysis is local, in that it ensures a linear rate of convergence when the initial iterate lies within a constant-radius region of the true parameter. At each iterate, multivariate Gaussian noise is added to the gradient in order to guarantee that the output satisfies Gaussian differential privacy. We then derive consequences of our theory for linear regression and mean estimation. Motivated by M-estimators used in robust statistics, we study loss functions which downweight the contribution of individual data points in such a way that the sensitivity of function gradients is guaranteed to be bounded, even without the usual assumption that our data lie in a bounded domain. We prove that the objective functions thus obtained indeed satisfy the restricted convexity and restricted smoothness conditions required for our general theory. We will also discuss the benefits of acceleration in optimization procedures, specifically a private version of the Frank-Wolfe algorithm, and its consequences for statistical estimation.

This is based on joint work with Marco Avella-Medina, Casey Bradshaw, Zheng Liu, and Laurentiu Marchis.
Date: 29 May 2025, 15:15
Venue: Mathematical Institute, Woodstock Road OX2 6GG
Venue Details: L2, Andrew Wiles Building
Speaker: Professor Po-Ling Loh (University of Cambridge)
Organising department: Department of Statistics
Organisers: Beverley Lane (Department of Statistics, University of Oxford), Professor Simon Myers (University of Oxford)
Organiser contact email address: events@stats.ox.ac.uk
Hosts: Professor Christl Donnelly (University of Oxford), Professor Simon Myers (University of Oxford)
Booking required?: Required
Booking url: https://forms.office.com/Pages/ResponsePage.aspx?id=G96VzPWXk0-0uv5ouFLPkTy86djjdtpBtsQPwHxYKRZUMTVGRU5CTFE0UVdPU1ZNRDJSVlBLUFhaRS4u
Booking email: events@stats.ox.ac.uk
Cost: No charge
Audience: Members of the University only
Editor: Beverley Lane