OxTalks is Changing
OxTalks will soon move to the new Halo platform and will become 'Oxford Events.' There will be a need for an OxTalks freeze. This was previously planned for Friday 14th November – a new date will be shared as soon as it is available (full details will be available on the Staff Gateway).
In the meantime, the OxTalks site will remain active and events will continue to be published.
If staff have any questions about the Oxford Events launch, please contact halo@digital.ox.ac.uk
Reproducing Expert Judgement with Shortened Surveys using Simulated Annealing
See more information at https://metrics-and-models.github.io/!
Surveys, screeners, and patient assessments are often shortened to decrease response burden and cost of administration. While there are many methods for shortening screeners, they often aim to reproduce the original score while inducing the smallest amount of measurement error. One consideration that is often overlooked is that these instruments are often used less for precise measurement of some latent construct, but instead for accurate prediction of an even more costly or time consuming diagnosis or expert judgment. As such, we present an alternative method that addresses these concerns through the use of a Markov Chain Monte Carlo algorithm with simulated annealing (MCMC-SA). To maintain ease of use of the eventual form, we use MCMC-SA to explore the combinatorial search space of short forms using a loss function that optimizes prediction based upon an unweighted sum score. This method is orders of magnitude more efficient than brute force search and has the advantage of optimizing for prediction of the desired outcome directly, instead of optimizing on the measurement of an intermediate outcome that may not be well aligned with the decision criteria of interest. When applied to a screener for alcohol use disorder, we demonstrate that under the unweighted sum score scoring constraint, shortened forms can more accurately predict expert judgments than full forms.
Date:
30 July 2025, 14:00
Venue:
Kindly register to our mailing list to receive the Teams invitation!
Speaker:
Klint (Kanopka)
Organising department:
Nuffield Department of Population Health
Organiser:
Metrics and Models Management Group (University of Oxford)
Organiser contact email address:
metrics_and_models_mgmt@maillist.ox.ac.uk
Part of:
Metrics and Models
Booking required?:
Not required
Booking url:
https://metrics-and-models.github.io/
Booking email:
metrics_and_models-subscribe@maillist.ox.ac.uk
Cost:
0
Audience:
Public
Editor:
Richard Rahal
