Emmanuel Candes – Stanford University
Machine learning algorithms provide predictions with a self-reported confidence score, but they are frequently inaccurate and uncalibrated, limiting their use in sensitive applications. This talk introduces novel calibration techniques addressing two frequently discussed problems:
• Learn then Test calibrates machine learning models so that their predictions satisfy explicit, finite-sample statistical guarantees regardless of the underlying model and (unknown) data-generating distribution. The framework addresses, among other examples, false discovery rate control in multi-label classification, intersection-over-union control in instance segmentation, and the simultaneous control of the type-1 error of outlier detection and confidence set coverage in classification or regression.
• Adaptive conformal inference. One limitation of conformal inference — this is becoming a standard tool in uncertainty quantification — is the need for identically distributed samples so that future test samples look like the training samples we have seen before. What if this is not case? E.g. in most settings the distribution of observations can shift drastically—think of finance or economics where market behavior can change in response to new legislation or major world events. We introduce an algorithm which can maintain prediction coverage over time despite substantial changes in the data distribution; for instance, it maintains coverage of key economic variables during the 2008 financial crisis and the 2020 COVID-19 period.
Introduction
This conference – organized under the auspices of the Isaac Newton Institute “Mathematics of Deep Learning” Programme — brings together leading researchers along with other stakeholders in industry and society to discuss issues surrounding trustworthy artificial intelligence.
This conference will overview the state-of-the-art within the wide area of trustworthy artificial intelligence including machine learning accountability, fairness, privacy, and safety; it will overview emerging directions in trustworthy artificial intelligence, and engage with academia, industry, policy makers, and the wider public.
Add comment