Hurdles to model generalization and potential solutions to the problem.
Machine Learning has been a major success story in recent times, but we are still beginning to understand how these models work and what are are the risks of deploying them to public sectors. Neural Networks in particular are low bias high variance machines, so their inability to generalize as expected to unseen data is a notorious shortcoming for their practical usage. Making models transfer well to new domains or changing environments is still one of thew hardest challenges of machine learning.
Understanding the generalization performance of deep learning models is a core research objective of modern machine learning. Many empirical results appear counterintuitive,and remain largely unexplained by theory. ~ On the Benefits of Invariance in Neural Networks
About this Crash Course
This crash course tries to cover common reasons that result in failure to generalize to unseen environments including low variation and spurious correlations. Furthermore, it covers data-centric approaches inspired by invariance and causality, that can be used to tackle these problems. Finally, it includes some case studies which use experimental modeling to better understand the presented concepts.
Bookmark this page as we intend to keep adding more articles to the list below. The list of articles covering this topic are:
- From Model-centric to Data-centric Artificial Intelligence
Is all the craze over new and more complex model architectures justified? - Invariance, Causality, and Robust Deep Learning
Why do Neural networks fail to generalize to new environments, and how can this be fixed? - Domain Randomization: future of robust modeling
Learn how to leverage this tool to unlock the true potential of synthetic data for Machine Learning! - Rethinking Data Augmentations: A Causal Perspective
Data augmentation is a mainstream tool in Machine Learning, but do we really understand it ? - Systematic Approach to Robust Deep Learning
Upfront causal thinking to design Robust Neural Networks that work well in new environments - Case Study 1: Is Spurious Correlations the reason why Neural Networks fail on unseen data?
Problems caused by spurious correlations and data-centric approach to deal with them and train Robust Models. - Case Study 2: Is your Neural Network failing due to low variance in irrelevant features ?
Susceptibility of neural networks to overfit on irrelevant properties and importance of randomizing them. - Case Study 3: Training a Robust Classifier using Invariance by design Approach
Putting invariance be design approach to practice in an image classification setting.