Bookmaking for Binary Outcomes: Prediction, Profits, and the IMV

Presented by Ben Domingue, Stanford University

Friday, February 24
11:00 a.m. ET
Virtual meeting - Webex meeting room

Understanding the "fit" of models designed to predict binary outcomes is a longstanding problem. We propose a flexible, portable, and intuitive metric for such scenarios: the InterModel Vigorish (IMV). The IMV is based on a series of bets involving weighted coins, well-characterized physical systems with tractable probabilities. The IMV has a number of desirable properties including an interpretable and portable scale and an appropriate sensitivity to outcome prevalence. We showcase its flexibility across examples spanning the social, biomedical, and physical sciences. We demonstrate how it can be used to provide straightforward interpretation of logistic regression coefficients and to provide insights about the value of different types of item response theory (IRT) models. The IMV allows for precise answers to questions about changes in model fit in a variety of settings in a manner that will be useful for furthering research with binary outcomes.

Speaker Bio:

Dr. Ben Domingue is an associate professor in the Graduate School of Education at Stanford University. He earned his BS and MA in Mathematics from the University of Texas at Austin and his PhD in Education from the University of Colorado Boulder. Dr. Domingue is interested in how student outcomes are leveraged to inform our understanding of student learning, teacher performance, and the efficacy of other programs. He has a particular interest in the technical issues that make it challenging to draw simple inferences from such student outcomes and the challenges of psychometric modeling in general.